2024-09-13T18:39:05.381 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T18:39:05.401 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T18:39:05.494 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7904006 branch: wip-rishabh-testing-20240910.100730-reef description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/openfiletable} email: null first_in_suite: false job_id: '7904006' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-rishabh-testing-20240910.100730-reef ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240910.100730-reef sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6439 sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 sleep_before_teardown: 0 subset: 964/1024 suite: fs suite_branch: wip-rishabh-testing-20240910.100730-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi171.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_openfiletable teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-13_14:42:16 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T18:39:05.494 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa; will attempt to use it 2024-09-13T18:39:05.495 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa/tasks 2024-09-13T18:39:05.495 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T18:39:05.496 INFO:teuthology.task.internal:Checking packages... 2024-09-13T18:39:05.517 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3' 2024-09-13T18:39:05.517 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T18:39:05.517 INFO:teuthology.packaging:ref: None 2024-09-13T18:39:05.517 INFO:teuthology.packaging:tag: None 2024-09-13T18:39:05.517 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240910.100730-reef 2024-09-13T18:39:05.517 INFO:teuthology.packaging:sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:39:05.517 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-rishabh-testing-20240910.100730-reef 2024-09-13T18:39:05.666 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-860-g6c6b083f-1focal 2024-09-13T18:39:05.667 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T18:39:05.675 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T18:39:05.675 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T18:39:05.720 INFO:teuthology.task.internal:Saving configuration 2024-09-13T18:39:05.736 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T18:39:05.753 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T18:39:05.785 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7904006', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 18:32:49.463749', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-13T18:39:05.814 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7904006', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 18:32:49.465119', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-13T18:39:05.814 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T18:39:05.822 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-13T18:39:05.822 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-13T18:39:05.822 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T18:39:05.937 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f201ff17be0>, signals=[15]) 2024-09-13T18:39:05.937 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T18:39:05.942 INFO:teuthology.task.internal:Opening connections... 2024-09-13T18:39:05.942 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-09-13T18:39:05.943 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:39:06.050 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T18:39:06.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:39:06.156 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T18:39:06.164 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-09-13T18:39:06.170 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-09-13T18:39:06.170 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-09-13T18:39:06.217 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="Ubuntu" 2024-09-13T18:39:06.217 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-13T18:39:06.218 INFO:teuthology.orchestra.run.smithi007.stdout:ID=ubuntu 2024-09-13T18:39:06.218 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE=debian 2024-09-13T18:39:06.218 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-13T18:39:06.218 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="20.04" 2024-09-13T18:39:06.218 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T18:39:06.218 INFO:teuthology.orchestra.run.smithi007.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T18:39:06.218 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T18:39:06.218 INFO:teuthology.orchestra.run.smithi007.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T18:39:06.218 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_CODENAME=focal 2024-09-13T18:39:06.218 INFO:teuthology.orchestra.run.smithi007.stdout:UBUNTU_CODENAME=focal 2024-09-13T18:39:06.219 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-09-13T18:39:06.250 DEBUG:teuthology.orchestra.run.smithi171:> uname -m 2024-09-13T18:39:06.258 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2024-09-13T18:39:06.258 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release 2024-09-13T18:39:06.306 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="Ubuntu" 2024-09-13T18:39:06.306 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-13T18:39:06.307 INFO:teuthology.orchestra.run.smithi171.stdout:ID=ubuntu 2024-09-13T18:39:06.307 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE=debian 2024-09-13T18:39:06.307 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-13T18:39:06.307 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="20.04" 2024-09-13T18:39:06.307 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T18:39:06.307 INFO:teuthology.orchestra.run.smithi171.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T18:39:06.307 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T18:39:06.308 INFO:teuthology.orchestra.run.smithi171.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T18:39:06.308 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_CODENAME=focal 2024-09-13T18:39:06.308 INFO:teuthology.orchestra.run.smithi171.stdout:UBUNTU_CODENAME=focal 2024-09-13T18:39:06.308 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2024-09-13T18:39:06.342 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T18:39:06.351 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T18:39:06.379 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T18:39:06.379 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-09-13T18:39:06.382 DEBUG:teuthology.orchestra.run.smithi171:> test '!' -e /home/ubuntu/cephtest 2024-09-13T18:39:06.387 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T18:39:06.411 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T18:39:06.411 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-09-13T18:39:06.413 DEBUG:teuthology.orchestra.run.smithi171:> test -z $(ls -A /var/lib/ceph) 2024-09-13T18:39:06.445 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T18:39:06.514 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T18:39:06.527 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T18:39:06.527 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T18:39:06.528 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T18:39:06.528 DEBUG:teuthology.orchestra.run.smithi171:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T18:39:06.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:39:06.534 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-09-13T18:39:06.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:39:06.535 DEBUG:teuthology.orchestra.run.smithi171:> uname -r 2024-09-13T18:39:06.541 INFO:teuthology.orchestra.run.smithi007.stdout:5.4.0-124-generic 2024-09-13T18:39:06.541 INFO:teuthology.task.kernel:Running kernel on smithi007: 5.4.0-124-generic 2024-09-13T18:39:06.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo apt-get clean 2024-09-13T18:39:06.542 INFO:teuthology.orchestra.run.smithi171.stdout:5.4.0-124-generic 2024-09-13T18:39:06.542 INFO:teuthology.task.kernel:Running kernel on smithi171: 5.4.0-124-generic 2024-09-13T18:39:06.542 DEBUG:teuthology.orchestra.run.smithi171:> sudo apt-get clean 2024-09-13T18:39:06.690 DEBUG:teuthology.orchestra.run.smithi171:> sudo apt-get update 2024-09-13T18:39:06.694 DEBUG:teuthology.orchestra.run.smithi007:> sudo apt-get update 2024-09-13T18:39:06.836 INFO:teuthology.orchestra.run.smithi171.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-13T18:39:06.851 INFO:teuthology.orchestra.run.smithi007.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-13T18:39:06.954 INFO:teuthology.orchestra.run.smithi171.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T18:39:06.962 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T18:39:07.037 INFO:teuthology.orchestra.run.smithi171.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-13T18:39:07.043 INFO:teuthology.orchestra.run.smithi007.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-13T18:39:07.136 INFO:teuthology.orchestra.run.smithi007.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-13T18:39:07.142 INFO:teuthology.orchestra.run.smithi171.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-13T18:39:07.235 INFO:teuthology.orchestra.run.smithi171.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-13T18:39:07.276 INFO:teuthology.orchestra.run.smithi171.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-13T18:39:07.278 INFO:teuthology.orchestra.run.smithi007.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-13T18:39:07.280 INFO:teuthology.orchestra.run.smithi171.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-13T18:39:07.280 INFO:teuthology.orchestra.run.smithi171.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-13T18:39:07.290 INFO:teuthology.orchestra.run.smithi007.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-13T18:39:07.293 INFO:teuthology.orchestra.run.smithi007.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-13T18:39:07.293 INFO:teuthology.orchestra.run.smithi007.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-13T18:39:07.294 INFO:teuthology.orchestra.run.smithi007.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-13T18:39:07.296 INFO:teuthology.orchestra.run.smithi171.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-13T18:39:07.314 INFO:teuthology.orchestra.run.smithi007.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-13T18:39:07.317 INFO:teuthology.orchestra.run.smithi007.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T18:39:07.317 INFO:teuthology.orchestra.run.smithi007.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-13T18:39:07.320 INFO:teuthology.orchestra.run.smithi171.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-13T18:39:07.321 INFO:teuthology.orchestra.run.smithi007.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-13T18:39:07.323 INFO:teuthology.orchestra.run.smithi171.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T18:39:07.323 INFO:teuthology.orchestra.run.smithi171.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-13T18:39:07.328 INFO:teuthology.orchestra.run.smithi007.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-13T18:39:07.329 INFO:teuthology.orchestra.run.smithi007.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-13T18:39:07.329 INFO:teuthology.orchestra.run.smithi007.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-13T18:39:07.330 INFO:teuthology.orchestra.run.smithi007.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-13T18:39:07.331 INFO:teuthology.orchestra.run.smithi171.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-13T18:39:07.337 INFO:teuthology.orchestra.run.smithi171.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-13T18:39:07.338 INFO:teuthology.orchestra.run.smithi171.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-13T18:39:07.338 INFO:teuthology.orchestra.run.smithi171.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-13T18:39:07.339 INFO:teuthology.orchestra.run.smithi171.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-13T18:39:07.339 INFO:teuthology.orchestra.run.smithi171.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-13T18:39:07.345 INFO:teuthology.orchestra.run.smithi171.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-13T18:39:07.345 INFO:teuthology.orchestra.run.smithi007.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-13T18:39:07.345 INFO:teuthology.orchestra.run.smithi007.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-13T18:39:07.391 INFO:teuthology.orchestra.run.smithi171.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-13T18:39:07.396 INFO:teuthology.orchestra.run.smithi007.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-13T18:39:07.531 INFO:teuthology.orchestra.run.smithi171.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-13T18:39:07.536 INFO:teuthology.orchestra.run.smithi007.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-13T18:39:07.916 INFO:teuthology.orchestra.run.smithi171.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-13T18:39:07.921 INFO:teuthology.orchestra.run.smithi007.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-13T18:39:07.944 INFO:teuthology.orchestra.run.smithi171.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-13T18:39:07.948 INFO:teuthology.orchestra.run.smithi007.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-13T18:39:07.958 INFO:teuthology.orchestra.run.smithi171.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-13T18:39:07.958 INFO:teuthology.orchestra.run.smithi171.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-13T18:39:07.959 INFO:teuthology.orchestra.run.smithi171.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-13T18:39:07.961 INFO:teuthology.orchestra.run.smithi007.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-13T18:39:07.961 INFO:teuthology.orchestra.run.smithi007.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-13T18:39:07.962 INFO:teuthology.orchestra.run.smithi007.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-13T18:39:08.052 INFO:teuthology.orchestra.run.smithi171.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-13T18:39:08.060 INFO:teuthology.orchestra.run.smithi007.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-13T18:39:08.070 INFO:teuthology.orchestra.run.smithi171.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T18:39:08.070 INFO:teuthology.orchestra.run.smithi171.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,230 kB] 2024-09-13T18:39:08.077 INFO:teuthology.orchestra.run.smithi007.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T18:39:08.078 INFO:teuthology.orchestra.run.smithi007.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-13T18:39:08.101 INFO:teuthology.orchestra.run.smithi007.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,230 kB] 2024-09-13T18:39:08.105 INFO:teuthology.orchestra.run.smithi171.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-13T18:39:08.124 INFO:teuthology.orchestra.run.smithi171.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-13T18:39:08.130 INFO:teuthology.orchestra.run.smithi007.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-13T18:39:08.133 INFO:teuthology.orchestra.run.smithi171.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-13T18:39:08.134 INFO:teuthology.orchestra.run.smithi171.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-13T18:39:08.135 INFO:teuthology.orchestra.run.smithi171.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-13T18:39:08.135 INFO:teuthology.orchestra.run.smithi171.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-13T18:39:08.141 INFO:teuthology.orchestra.run.smithi007.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-13T18:39:08.142 INFO:teuthology.orchestra.run.smithi007.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-13T18:39:08.142 INFO:teuthology.orchestra.run.smithi007.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-13T18:39:08.143 INFO:teuthology.orchestra.run.smithi007.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-13T18:39:08.165 INFO:teuthology.orchestra.run.smithi171.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-13T18:39:08.166 INFO:teuthology.orchestra.run.smithi171.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-13T18:39:08.166 INFO:teuthology.orchestra.run.smithi007.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-13T18:39:08.167 INFO:teuthology.orchestra.run.smithi171.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-13T18:39:08.167 INFO:teuthology.orchestra.run.smithi007.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-13T18:39:08.168 INFO:teuthology.orchestra.run.smithi007.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-13T18:39:08.247 INFO:teuthology.orchestra.run.smithi171.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-13T18:39:08.248 INFO:teuthology.orchestra.run.smithi171.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-13T18:39:08.248 INFO:teuthology.orchestra.run.smithi007.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-13T18:39:08.248 INFO:teuthology.orchestra.run.smithi007.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-13T18:39:08.249 INFO:teuthology.orchestra.run.smithi171.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-13T18:39:08.249 INFO:teuthology.orchestra.run.smithi007.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-13T18:39:08.250 INFO:teuthology.orchestra.run.smithi171.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-13T18:39:08.250 INFO:teuthology.orchestra.run.smithi007.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-13T18:39:11.507 INFO:teuthology.orchestra.run.smithi171.stdout:Fetched 21.8 MB in 3s (7,542 kB/s) 2024-09-13T18:39:11.893 INFO:teuthology.orchestra.run.smithi007.stdout:Fetched 21.8 MB in 3s (7,061 kB/s) 2024-09-13T18:39:12.983 INFO:teuthology.orchestra.run.smithi171.stdout:Reading package lists... 2024-09-13T18:39:13.009 DEBUG:teuthology.orchestra.run.smithi171:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T18:39:13.085 INFO:teuthology.orchestra.run.smithi171.stdout:Reading package lists... 2024-09-13T18:39:13.304 INFO:teuthology.orchestra.run.smithi171.stdout:Building dependency tree... 2024-09-13T18:39:13.304 INFO:teuthology.orchestra.run.smithi171.stdout:Reading state information... 2024-09-13T18:39:13.458 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2024-09-13T18:39:13.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T18:39:13.506 INFO:teuthology.orchestra.run.smithi171.stdout:The following additional packages will be installed: 2024-09-13T18:39:13.506 INFO:teuthology.orchestra.run.smithi171.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T18:39:13.506 INFO:teuthology.orchestra.run.smithi171.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-13T18:39:13.506 INFO:teuthology.orchestra.run.smithi171.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-13T18:39:13.507 INFO:teuthology.orchestra.run.smithi171.stdout:Suggested packages: 2024-09-13T18:39:13.508 INFO:teuthology.orchestra.run.smithi171.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-13T18:39:13.508 INFO:teuthology.orchestra.run.smithi171.stdout:Recommended packages: 2024-09-13T18:39:13.508 INFO:teuthology.orchestra.run.smithi171.stdout: thermald 2024-09-13T18:39:13.566 INFO:teuthology.orchestra.run.smithi171.stdout:The following NEW packages will be installed: 2024-09-13T18:39:13.566 INFO:teuthology.orchestra.run.smithi171.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T18:39:13.566 INFO:teuthology.orchestra.run.smithi171.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-13T18:39:13.567 INFO:teuthology.orchestra.run.smithi171.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-13T18:39:13.567 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2024-09-13T18:39:13.568 INFO:teuthology.orchestra.run.smithi171.stdout:The following packages will be upgraded: 2024-09-13T18:39:13.569 INFO:teuthology.orchestra.run.smithi171.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T18:39:13.622 INFO:teuthology.orchestra.run.smithi171.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T18:39:13.622 INFO:teuthology.orchestra.run.smithi171.stdout:Need to get 77.2 MB of archives. 2024-09-13T18:39:13.623 INFO:teuthology.orchestra.run.smithi171.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-13T18:39:13.623 INFO:teuthology.orchestra.run.smithi171.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-13T18:39:13.790 INFO:teuthology.orchestra.run.smithi007.stdout:Building dependency tree... 2024-09-13T18:39:13.790 INFO:teuthology.orchestra.run.smithi007.stdout:Reading state information... 2024-09-13T18:39:13.934 INFO:teuthology.orchestra.run.smithi171.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-13T18:39:14.003 INFO:teuthology.orchestra.run.smithi007.stdout:The following additional packages will be installed: 2024-09-13T18:39:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T18:39:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-13T18:39:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-13T18:39:14.004 INFO:teuthology.orchestra.run.smithi007.stdout:Suggested packages: 2024-09-13T18:39:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-13T18:39:14.004 INFO:teuthology.orchestra.run.smithi007.stdout:Recommended packages: 2024-09-13T18:39:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: thermald 2024-09-13T18:39:14.025 INFO:teuthology.orchestra.run.smithi171.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-13T18:39:14.066 INFO:teuthology.orchestra.run.smithi007.stdout:The following NEW packages will be installed: 2024-09-13T18:39:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T18:39:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-13T18:39:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-13T18:39:14.068 INFO:teuthology.orchestra.run.smithi007.stdout:The following packages will be upgraded: 2024-09-13T18:39:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T18:39:14.124 INFO:teuthology.orchestra.run.smithi007.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T18:39:14.124 INFO:teuthology.orchestra.run.smithi007.stdout:Need to get 77.2 MB of archives. 2024-09-13T18:39:14.124 INFO:teuthology.orchestra.run.smithi007.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-13T18:39:14.124 INFO:teuthology.orchestra.run.smithi007.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-13T18:39:14.454 INFO:teuthology.orchestra.run.smithi007.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-13T18:39:14.473 INFO:teuthology.orchestra.run.smithi171.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-13T18:39:14.473 INFO:teuthology.orchestra.run.smithi171.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-13T18:39:14.474 INFO:teuthology.orchestra.run.smithi171.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-13T18:39:14.548 INFO:teuthology.orchestra.run.smithi007.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-13T18:39:14.565 INFO:teuthology.orchestra.run.smithi171.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-13T18:39:14.576 INFO:teuthology.orchestra.run.smithi171.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-13T18:39:15.090 INFO:teuthology.orchestra.run.smithi007.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-13T18:39:15.090 INFO:teuthology.orchestra.run.smithi007.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-13T18:39:15.091 INFO:teuthology.orchestra.run.smithi007.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-13T18:39:15.186 INFO:teuthology.orchestra.run.smithi007.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-13T18:39:15.197 INFO:teuthology.orchestra.run.smithi007.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-13T18:39:15.265 INFO:teuthology.orchestra.run.smithi171.stdout:Fetched 77.2 MB in 1s (77.2 MB/s) 2024-09-13T18:39:15.463 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-13T18:39:15.996 INFO:teuthology.orchestra.run.smithi007.stdout:Fetched 77.2 MB in 1s (68.8 MB/s) 2024-09-13T18:39:16.187 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-13T18:39:17.200 INFO:teuthology.orchestra.run.smithi171.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-13T18:39:17.205 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T18:39:17.318 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:39:17.977 INFO:teuthology.orchestra.run.smithi007.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-13T18:39:17.982 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T18:39:18.119 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:39:19.285 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-13T18:39:19.300 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T18:39:19.391 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:39:20.146 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-13T18:39:20.160 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T18:39:20.265 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:39:20.513 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-13T18:39:20.529 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T18:39:20.587 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:39:21.507 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-13T18:39:21.523 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T18:39:21.565 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:39:25.700 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T18:39:25.848 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T18:39:26.194 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T18:39:26.320 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T18:39:26.457 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T18:39:26.581 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T18:39:26.673 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-13T18:39:26.689 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-13T18:39:26.732 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T18:39:26.966 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T18:39:27.093 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T18:39:27.455 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-13T18:39:27.469 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-13T18:39:27.546 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T18:39:32.218 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-13T18:39:32.240 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T18:39:32.284 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:39:32.968 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-13T18:39:32.990 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T18:39:33.068 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:39:35.003 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T18:39:35.124 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T18:39:35.483 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:39:35.933 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T18:39:36.086 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T18:39:36.438 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:39:36.713 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:39:37.901 INFO:teuthology.orchestra.run.smithi171.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-13T18:39:37.901 INFO:teuthology.orchestra.run.smithi171.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-13T18:39:38.057 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:39:38.192 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:39:39.250 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T18:39:39.388 INFO:teuthology.orchestra.run.smithi007.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-13T18:39:39.388 INFO:teuthology.orchestra.run.smithi007.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-13T18:39:39.392 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-13T18:39:39.510 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:39:39.519 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:39:39.645 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-13T18:39:39.787 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-13T18:39:39.914 INFO:teuthology.orchestra.run.smithi171.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:39:40.000 INFO:teuthology.orchestra.run.smithi171.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T18:39:40.000 INFO:teuthology.orchestra.run.smithi171.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-13T18:39:40.778 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T18:39:40.904 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-13T18:39:41.048 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:39:41.173 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-13T18:39:41.299 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-13T18:39:41.434 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T18:39:41.499 INFO:teuthology.orchestra.run.smithi007.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T18:39:41.499 INFO:teuthology.orchestra.run.smithi007.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-13T18:40:02.503 INFO:teuthology.orchestra.run.smithi171.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T18:40:02.504 INFO:teuthology.orchestra.run.smithi171.stdout:Sourcing file `/etc/default/grub' 2024-09-13T18:40:02.513 INFO:teuthology.orchestra.run.smithi171.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T18:40:02.528 INFO:teuthology.orchestra.run.smithi171.stdout:Generating grub configuration file ... 2024-09-13T18:40:02.849 INFO:teuthology.orchestra.run.smithi171.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T18:40:02.877 INFO:teuthology.orchestra.run.smithi171.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T18:40:03.278 INFO:teuthology.orchestra.run.smithi171.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T18:40:03.289 INFO:teuthology.orchestra.run.smithi171.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T18:40:03.797 INFO:teuthology.orchestra.run.smithi007.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T18:40:03.798 INFO:teuthology.orchestra.run.smithi007.stdout:Sourcing file `/etc/default/grub' 2024-09-13T18:40:03.808 INFO:teuthology.orchestra.run.smithi007.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T18:40:03.832 INFO:teuthology.orchestra.run.smithi007.stdout:Generating grub configuration file ... 2024-09-13T18:40:04.220 INFO:teuthology.orchestra.run.smithi007.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T18:40:04.245 INFO:teuthology.orchestra.run.smithi007.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T18:40:04.633 INFO:teuthology.orchestra.run.smithi007.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T18:40:04.645 INFO:teuthology.orchestra.run.smithi007.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T18:40:05.087 INFO:teuthology.orchestra.run.smithi171.stdout:done 2024-09-13T18:40:05.516 DEBUG:teuthology.orchestra.run.smithi171:> dpkg -s linux-image-generic 2024-09-13T18:40:05.587 INFO:teuthology.orchestra.run.smithi171.stdout:Package: linux-image-generic 2024-09-13T18:40:05.587 INFO:teuthology.orchestra.run.smithi171.stdout:Status: install ok installed 2024-09-13T18:40:05.587 INFO:teuthology.orchestra.run.smithi171.stdout:Priority: optional 2024-09-13T18:40:05.587 INFO:teuthology.orchestra.run.smithi171.stdout:Section: kernel 2024-09-13T18:40:05.587 INFO:teuthology.orchestra.run.smithi171.stdout:Installed-Size: 20 2024-09-13T18:40:05.588 INFO:teuthology.orchestra.run.smithi171.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T18:40:05.588 INFO:teuthology.orchestra.run.smithi171.stdout:Architecture: amd64 2024-09-13T18:40:05.588 INFO:teuthology.orchestra.run.smithi171.stdout:Source: linux-meta 2024-09-13T18:40:05.588 INFO:teuthology.orchestra.run.smithi171.stdout:Version: 5.4.0.195.193 2024-09-13T18:40:05.588 INFO:teuthology.orchestra.run.smithi171.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T18:40:05.588 INFO:teuthology.orchestra.run.smithi171.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T18:40:05.588 INFO:teuthology.orchestra.run.smithi171.stdout:Recommends: thermald 2024-09-13T18:40:05.588 INFO:teuthology.orchestra.run.smithi171.stdout:Description: Generic Linux kernel image 2024-09-13T18:40:05.588 INFO:teuthology.orchestra.run.smithi171.stdout: This package will always depend on the latest generic kernel image 2024-09-13T18:40:05.588 INFO:teuthology.orchestra.run.smithi171.stdout: available. 2024-09-13T18:40:05.588 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-13T18:40:05.589 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T18:40:05.589 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T18:40:05.589 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T18:40:05.589 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi171.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T18:40:05.589 DEBUG:teuthology.orchestra.run.smithi171:> sudo apt-get clean 2024-09-13T18:40:05.688 DEBUG:teuthology.orchestra.run.smithi171:> sudo apt-get update 2024-09-13T18:40:05.956 INFO:teuthology.orchestra.run.smithi171.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T18:40:05.956 INFO:teuthology.orchestra.run.smithi171.stdout:Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T18:40:06.027 INFO:teuthology.orchestra.run.smithi171.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T18:40:06.121 INFO:teuthology.orchestra.run.smithi171.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T18:40:06.693 INFO:teuthology.orchestra.run.smithi007.stdout:done 2024-09-13T18:40:07.193 DEBUG:teuthology.orchestra.run.smithi007:> dpkg -s linux-image-generic 2024-09-13T18:40:07.237 INFO:teuthology.orchestra.run.smithi007.stdout:Package: linux-image-generic 2024-09-13T18:40:07.237 INFO:teuthology.orchestra.run.smithi007.stdout:Status: install ok installed 2024-09-13T18:40:07.237 INFO:teuthology.orchestra.run.smithi007.stdout:Priority: optional 2024-09-13T18:40:07.237 INFO:teuthology.orchestra.run.smithi007.stdout:Section: kernel 2024-09-13T18:40:07.237 INFO:teuthology.orchestra.run.smithi007.stdout:Installed-Size: 20 2024-09-13T18:40:07.237 INFO:teuthology.orchestra.run.smithi007.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T18:40:07.237 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: amd64 2024-09-13T18:40:07.237 INFO:teuthology.orchestra.run.smithi007.stdout:Source: linux-meta 2024-09-13T18:40:07.237 INFO:teuthology.orchestra.run.smithi007.stdout:Version: 5.4.0.195.193 2024-09-13T18:40:07.237 INFO:teuthology.orchestra.run.smithi007.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T18:40:07.238 INFO:teuthology.orchestra.run.smithi007.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T18:40:07.238 INFO:teuthology.orchestra.run.smithi007.stdout:Recommends: thermald 2024-09-13T18:40:07.238 INFO:teuthology.orchestra.run.smithi007.stdout:Description: Generic Linux kernel image 2024-09-13T18:40:07.238 INFO:teuthology.orchestra.run.smithi007.stdout: This package will always depend on the latest generic kernel image 2024-09-13T18:40:07.238 INFO:teuthology.orchestra.run.smithi007.stdout: available. 2024-09-13T18:40:07.238 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-13T18:40:07.238 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T18:40:07.238 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T18:40:07.238 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T18:40:07.238 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T18:40:07.239 DEBUG:teuthology.orchestra.run.smithi007:> sudo apt-get clean 2024-09-13T18:40:07.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo apt-get update 2024-09-13T18:40:07.459 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T18:40:07.576 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T18:40:07.657 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T18:40:07.738 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T18:40:07.844 INFO:teuthology.orchestra.run.smithi171.stdout:Reading package lists... 2024-09-13T18:40:07.871 DEBUG:teuthology.orchestra.run.smithi171:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T18:40:07.945 INFO:teuthology.orchestra.run.smithi171.stdout:Reading package lists... 2024-09-13T18:40:08.163 INFO:teuthology.orchestra.run.smithi171.stdout:Building dependency tree... 2024-09-13T18:40:08.164 INFO:teuthology.orchestra.run.smithi171.stdout:Reading state information... 2024-09-13T18:40:08.400 INFO:teuthology.orchestra.run.smithi171.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-13T18:40:08.400 INFO:teuthology.orchestra.run.smithi171.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T18:40:08.402 DEBUG:teuthology.orchestra.run.smithi171:> dpkg -s linux-image-generic 2024-09-13T18:40:08.426 INFO:teuthology.orchestra.run.smithi171.stdout:Package: linux-image-generic 2024-09-13T18:40:08.426 INFO:teuthology.orchestra.run.smithi171.stdout:Status: install ok installed 2024-09-13T18:40:08.426 INFO:teuthology.orchestra.run.smithi171.stdout:Priority: optional 2024-09-13T18:40:08.426 INFO:teuthology.orchestra.run.smithi171.stdout:Section: kernel 2024-09-13T18:40:08.426 INFO:teuthology.orchestra.run.smithi171.stdout:Installed-Size: 20 2024-09-13T18:40:08.426 INFO:teuthology.orchestra.run.smithi171.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T18:40:08.426 INFO:teuthology.orchestra.run.smithi171.stdout:Architecture: amd64 2024-09-13T18:40:08.426 INFO:teuthology.orchestra.run.smithi171.stdout:Source: linux-meta 2024-09-13T18:40:08.426 INFO:teuthology.orchestra.run.smithi171.stdout:Version: 5.4.0.195.193 2024-09-13T18:40:08.426 INFO:teuthology.orchestra.run.smithi171.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T18:40:08.427 INFO:teuthology.orchestra.run.smithi171.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T18:40:08.427 INFO:teuthology.orchestra.run.smithi171.stdout:Recommends: thermald 2024-09-13T18:40:08.427 INFO:teuthology.orchestra.run.smithi171.stdout:Description: Generic Linux kernel image 2024-09-13T18:40:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: This package will always depend on the latest generic kernel image 2024-09-13T18:40:08.427 INFO:teuthology.orchestra.run.smithi171.stdout: available. 2024-09-13T18:40:08.427 DEBUG:teuthology.orchestra.run.smithi171:> mktemp 2024-09-13T18:40:08.433 INFO:teuthology.orchestra.run.smithi171.stdout:/tmp/tmp.EQqTD7Vnz7 2024-09-13T18:40:08.433 DEBUG:teuthology.orchestra.run.smithi171:> sudo cp /boot/grub/grub.cfg /tmp/tmp.EQqTD7Vnz7 2024-09-13T18:40:08.489 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0666 /tmp/tmp.EQqTD7Vnz7 2024-09-13T18:40:08.532 DEBUG:teuthology.orchestra.remote:smithi171:/tmp/tmp.EQqTD7Vnz7 is 9KB 2024-09-13T18:40:08.546 DEBUG:teuthology.orchestra.run.smithi171:> rm -fr /tmp/tmp.EQqTD7Vnz7 2024-09-13T18:40:08.552 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T18:40:08.609 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:40:08.609 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T18:40:08.609 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T18:40:08.636 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-13T18:40:08.636 DEBUG:teuthology.orchestra.run.smithi171:> sudo update-grub 2024-09-13T18:40:09.393 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2024-09-13T18:40:09.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T18:40:09.542 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2024-09-13T18:40:09.766 INFO:teuthology.orchestra.run.smithi007.stdout:Building dependency tree... 2024-09-13T18:40:09.766 INFO:teuthology.orchestra.run.smithi007.stdout:Reading state information... 2024-09-13T18:40:10.000 INFO:teuthology.orchestra.run.smithi171.stderr:Sourcing file `/etc/default/grub' 2024-09-13T18:40:10.009 INFO:teuthology.orchestra.run.smithi007.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-13T18:40:10.009 INFO:teuthology.orchestra.run.smithi007.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T18:40:10.011 DEBUG:teuthology.orchestra.run.smithi007:> dpkg -s linux-image-generic 2024-09-13T18:40:10.012 INFO:teuthology.orchestra.run.smithi171.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T18:40:10.024 INFO:teuthology.orchestra.run.smithi171.stderr:Generating grub configuration file ... 2024-09-13T18:40:10.033 INFO:teuthology.orchestra.run.smithi007.stdout:Package: linux-image-generic 2024-09-13T18:40:10.033 INFO:teuthology.orchestra.run.smithi007.stdout:Status: install ok installed 2024-09-13T18:40:10.033 INFO:teuthology.orchestra.run.smithi007.stdout:Priority: optional 2024-09-13T18:40:10.033 INFO:teuthology.orchestra.run.smithi007.stdout:Section: kernel 2024-09-13T18:40:10.033 INFO:teuthology.orchestra.run.smithi007.stdout:Installed-Size: 20 2024-09-13T18:40:10.033 INFO:teuthology.orchestra.run.smithi007.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T18:40:10.033 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: amd64 2024-09-13T18:40:10.034 INFO:teuthology.orchestra.run.smithi007.stdout:Source: linux-meta 2024-09-13T18:40:10.034 INFO:teuthology.orchestra.run.smithi007.stdout:Version: 5.4.0.195.193 2024-09-13T18:40:10.034 INFO:teuthology.orchestra.run.smithi007.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T18:40:10.034 INFO:teuthology.orchestra.run.smithi007.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T18:40:10.034 INFO:teuthology.orchestra.run.smithi007.stdout:Recommends: thermald 2024-09-13T18:40:10.034 INFO:teuthology.orchestra.run.smithi007.stdout:Description: Generic Linux kernel image 2024-09-13T18:40:10.034 INFO:teuthology.orchestra.run.smithi007.stdout: This package will always depend on the latest generic kernel image 2024-09-13T18:40:10.034 INFO:teuthology.orchestra.run.smithi007.stdout: available. 2024-09-13T18:40:10.034 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-13T18:40:10.039 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.GWwNEIq1lo 2024-09-13T18:40:10.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub/grub.cfg /tmp/tmp.GWwNEIq1lo 2024-09-13T18:40:10.097 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.GWwNEIq1lo 2024-09-13T18:40:10.162 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.GWwNEIq1lo is 9KB 2024-09-13T18:40:10.177 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.GWwNEIq1lo 2024-09-13T18:40:10.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T18:40:10.239 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:40:10.239 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T18:40:10.239 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T18:40:10.261 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-13T18:40:10.262 DEBUG:teuthology.orchestra.run.smithi007:> sudo update-grub 2024-09-13T18:40:10.277 INFO:teuthology.orchestra.run.smithi171.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T18:40:10.289 INFO:teuthology.orchestra.run.smithi171.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T18:40:10.639 INFO:teuthology.orchestra.run.smithi171.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T18:40:10.650 INFO:teuthology.orchestra.run.smithi171.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T18:40:11.380 INFO:teuthology.orchestra.run.smithi171.stderr:done 2024-09-13T18:40:11.383 DEBUG:teuthology.orchestra.run.smithi171:> sudo shutdown -r now 2024-09-13T18:40:11.944 INFO:teuthology.orchestra.run.smithi007.stderr:Sourcing file `/etc/default/grub' 2024-09-13T18:40:11.949 INFO:teuthology.orchestra.run.smithi007.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T18:40:11.963 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-09-13T18:40:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T18:40:12.274 INFO:teuthology.orchestra.run.smithi007.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T18:40:12.647 INFO:teuthology.orchestra.run.smithi007.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T18:40:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T18:40:13.678 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-09-13T18:40:13.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-09-13T18:40:41.399 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T18:40:41.400 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-13T18:40:41.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:40:43.683 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T18:40:43.683 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-13T18:40:43.684 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:40:59.965 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.171 2024-09-13T18:41:02.269 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-09-13T18:41:08.973 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-13T18:41:08.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:41:11.272 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-13T18:41:11.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:41:12.061 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.171 2024-09-13T18:41:14.337 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-09-13T18:41:24.069 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-13T18:41:24.070 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:41:26.340 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-13T18:41:26.341 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:41:58.067 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-09-13T18:41:58.687 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-13T18:41:58.687 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-195-generic"... 2024-09-13T18:41:58.687 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-09-13T18:41:58.694 INFO:teuthology.orchestra.run.smithi007.stdout:5.4.0-195-generic 2024-09-13T18:41:58.694 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-13T18:41:58.694 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T18:41:58.694 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T18:41:59.695 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T18:41:59.695 DEBUG:teuthology.orchestra.run.smithi007:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T18:41:59.792 INFO:teuthology.orchestra.run.smithi007.stdout:ttyS1 2024-09-13T18:41:59.808 DEBUG:teuthology.parallel:result is None 2024-09-13T18:42:24.080 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T18:42:39.082 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-13T18:42:39.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:42:39.335 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-09-13T18:42:39.899 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-13T18:42:39.899 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-195-generic"... 2024-09-13T18:42:39.899 DEBUG:teuthology.orchestra.run.smithi171:> uname -r 2024-09-13T18:42:39.950 INFO:teuthology.orchestra.run.smithi171.stdout:5.4.0-195-generic 2024-09-13T18:42:39.950 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-13T18:42:39.950 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T18:42:39.950 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T18:42:40.951 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T18:42:40.951 DEBUG:teuthology.orchestra.run.smithi171:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T18:42:41.052 INFO:teuthology.orchestra.run.smithi171.stdout:ttyS1 2024-09-13T18:42:41.081 DEBUG:teuthology.parallel:result is None 2024-09-13T18:42:41.082 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T18:42:41.089 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T18:42:41.089 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T18:42:41.092 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T18:42:41.098 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T18:42:41.134 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T18:42:41.164 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T18:42:41.165 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T18:42:41.167 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T18:42:41.189 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T18:42:41.196 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T18:42:41.196 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T18:42:41.199 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T18:42:41.216 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T18:42:41.216 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T18:42:41.225 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T18:42:41.227 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T18:42:41.228 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T18:42:41.235 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T18:42:41.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T18:42:41.272 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T18:42:41.287 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T18:42:41.297 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T18:42:41.297 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T18:42:41.299 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T18:42:41.306 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T18:42:41.313 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T18:42:41.361 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:42:41.361 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T18:42:41.388 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T18:42:41.396 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T18:42:41.447 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:42:41.447 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T18:42:41.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-09-13T18:42:41.478 DEBUG:teuthology.orchestra.run.smithi171:> sudo service rsyslog restart 2024-09-13T18:42:41.532 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T18:42:41.539 INFO:teuthology.task.internal:Starting timer... 2024-09-13T18:42:41.539 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T18:42:41.565 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T18:42:41.597 DEBUG:teuthology.task.selinux:Excluding smithi007: OS 'ubuntu' does not support SELinux 2024-09-13T18:42:41.597 DEBUG:teuthology.task.selinux:Excluding smithi171: OS 'ubuntu' does not support SELinux 2024-09-13T18:42:41.598 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T18:42:41.598 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-13T18:42:41.598 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T18:42:41.598 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T18:42:41.631 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T18:42:41.645 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T18:42:41.646 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi171.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T18:48:21.868 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi171.front.sepia.ceph.com')] 2024-09-13T18:48:21.869 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-13T18:48:21.871 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:48:21.979 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-09-13T18:48:22.102 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-13T18:48:22.103 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-13T18:48:22.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T18:48:22.231 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-09-13T18:48:22.355 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-13T18:48:22.355 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T18:48:22.365 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T18:48:22.365 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T18:48:22.366 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T18:48:22.368 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T18:48:22.368 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T18:48:22.400 INFO:teuthology.orchestra.run.smithi007.stdout:13 Sep 18:48:22 ntpd[13639]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-13T18:48:22.401 INFO:teuthology.orchestra.run.smithi007.stdout:13 Sep 18:48:22 ntpd[13639]: Command line: ntpd -gq 2024-09-13T18:48:22.401 INFO:teuthology.orchestra.run.smithi007.stdout:13 Sep 18:48:22 ntpd[13639]: proto: precision = 0.166 usec (-22) 2024-09-13T18:48:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T18:48:22.402 INFO:teuthology.orchestra.run.smithi007.stdout:13 Sep 18:48:22 ntpd[13639]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T18:48:22.402 INFO:teuthology.orchestra.run.smithi007.stdout:13 Sep 18:48:22 ntpd[13639]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T18:48:22.402 INFO:teuthology.orchestra.run.smithi007.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T18:48:22.403 INFO:teuthology.orchestra.run.smithi007.stdout:13 Sep 18:48:22 ntpd[13639]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T18:48:22.403 INFO:teuthology.orchestra.run.smithi007.stdout:13 Sep 18:48:22 ntpd[13639]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T18:48:22.403 INFO:teuthology.orchestra.run.smithi007.stdout:13 Sep 18:48:22 ntpd[13639]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T18:48:22.404 INFO:teuthology.orchestra.run.smithi007.stdout:13 Sep 18:48:22 ntpd[13639]: Listen normally on 3 ens1f0 172.21.15.7:123 2024-09-13T18:48:22.404 INFO:teuthology.orchestra.run.smithi007.stdout:13 Sep 18:48:22 ntpd[13639]: Listen normally on 4 lo [::1]:123 2024-09-13T18:48:22.404 INFO:teuthology.orchestra.run.smithi007.stdout:13 Sep 18:48:22 ntpd[13639]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15ea%4]:123 2024-09-13T18:48:22.404 INFO:teuthology.orchestra.run.smithi007.stdout:13 Sep 18:48:22 ntpd[13639]: Listening on routing socket on fd #22 for interface updates 2024-09-13T18:48:22.405 INFO:teuthology.orchestra.run.smithi171.stdout:13 Sep 18:48:22 ntpd[13695]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-13T18:48:22.405 INFO:teuthology.orchestra.run.smithi171.stdout:13 Sep 18:48:22 ntpd[13695]: Command line: ntpd -gq 2024-09-13T18:48:22.406 INFO:teuthology.orchestra.run.smithi171.stdout:13 Sep 18:48:22 ntpd[13695]: proto: precision = 0.116 usec (-23) 2024-09-13T18:48:22.407 INFO:teuthology.orchestra.run.smithi171.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T18:48:22.407 INFO:teuthology.orchestra.run.smithi171.stdout:13 Sep 18:48:22 ntpd[13695]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T18:48:22.407 INFO:teuthology.orchestra.run.smithi171.stdout:13 Sep 18:48:22 ntpd[13695]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T18:48:22.407 INFO:teuthology.orchestra.run.smithi171.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T18:48:22.408 INFO:teuthology.orchestra.run.smithi171.stdout:13 Sep 18:48:22 ntpd[13695]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T18:48:22.408 INFO:teuthology.orchestra.run.smithi171.stdout:13 Sep 18:48:22 ntpd[13695]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T18:48:22.409 INFO:teuthology.orchestra.run.smithi171.stdout:13 Sep 18:48:22 ntpd[13695]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T18:48:22.409 INFO:teuthology.orchestra.run.smithi171.stdout:13 Sep 18:48:22 ntpd[13695]: Listen normally on 3 enp3s0f1 172.21.15.171:123 2024-09-13T18:48:22.409 INFO:teuthology.orchestra.run.smithi171.stdout:13 Sep 18:48:22 ntpd[13695]: Listen normally on 4 lo [::1]:123 2024-09-13T18:48:22.409 INFO:teuthology.orchestra.run.smithi171.stdout:13 Sep 18:48:22 ntpd[13695]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6f6d%5]:123 2024-09-13T18:48:22.409 INFO:teuthology.orchestra.run.smithi171.stdout:13 Sep 18:48:22 ntpd[13695]: Listening on routing socket on fd #22 for interface updates 2024-09-13T18:48:23.402 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:23 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:23.402 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:23 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:23.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:23 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:23.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:23 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:23.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:23 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:23.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:23 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:23.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:23 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:23.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:23 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:23.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:23 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:48:23.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:23 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:48:23.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:23 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:23.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:23 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:23.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:23 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:48:23.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:23 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:48:23.405 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:23 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:23.405 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:23 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:24.406 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:24 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:24.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:24 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:24.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:24 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:24.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:24 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:24.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:24 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:24.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:24 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:24.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:24 ntpd[13695]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:48:24.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:24 ntpd[13695]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:48:24.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:24 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:24.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:24 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:24.408 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:24 ntpd[13695]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:48:24.408 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:24 ntpd[13695]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:48:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:25 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:25 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:25 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:25 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:25 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:25 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:25 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:48:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:25 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:48:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:25 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:25 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:25 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:48:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:25 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:48:26.405 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:26 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:26.406 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:26 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:26.406 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:26 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:26.406 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:26 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:26.409 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:26 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:26.410 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:26 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:26.410 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:26 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:26.410 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:26 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:26.410 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:26 ntpd[13695]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:48:26.410 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:26 ntpd[13695]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:48:26.410 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:26 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:26.410 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:26 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:26.410 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:26 ntpd[13695]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:48:26.410 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:26 ntpd[13695]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:48:26.410 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:26 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:26.410 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:26 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:26.411 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:26 ntpd[13695]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:48:26.411 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:26 ntpd[13695]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:48:27.402 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:27 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:27 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:27 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:27 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:27 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:27 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:27 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:27 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:27 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:48:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:27 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:48:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:27 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:27 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:27 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:48:27.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:27 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:48:27.404 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:27 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:27.404 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:27 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:27.404 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:27 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:48:27.404 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:27 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:48:28.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:28 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:28.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:28 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:28.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:28 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:28.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:28 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:28.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:28 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:28.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:28 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:28.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:28 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:28.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:28 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:28.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:28 ntpd[13695]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:48:28.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:28 ntpd[13695]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:48:28.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:28 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:28.408 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:28 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:28.408 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:28 ntpd[13695]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:48:28.408 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:28 ntpd[13695]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:48:28.408 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:28 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:28.408 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:28 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:28.408 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:28 ntpd[13695]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:48:28.409 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:28 ntpd[13695]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:48:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:29 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:29 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:29 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:29 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:29 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:29 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:29.402 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:29 ntpd[13639]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:29 ntpd[13639]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:29 ntpd[13639]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:48:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:29 ntpd[13639]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:48:29.403 INFO:teuthology.orchestra.run.smithi007.stdout:13 Sep 18:48:29 ntpd[13639]: ntpd: time slew +0.001512 s 2024-09-13T18:48:29.403 INFO:teuthology.orchestra.run.smithi007.stdout:ntpd: time slew +0.001512s 2024-09-13T18:48:29.403 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:29 ntpd[13639]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T18:48:29.404 INFO:teuthology.orchestra.run.smithi007.stderr:13 Sep 18:48:29 ntpd[13639]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T18:48:29.476 INFO:teuthology.orchestra.run.smithi007.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T18:48:29.476 INFO:teuthology.orchestra.run.smithi007.stdout:============================================================================== 2024-09-13T18:48:29.476 INFO:teuthology.orchestra.run.smithi007.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T18:48:29.476 INFO:teuthology.orchestra.run.smithi007.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T18:48:29.476 INFO:teuthology.orchestra.run.smithi007.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T18:48:29.476 INFO:teuthology.orchestra.run.smithi007.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T18:48:30.406 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:30 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:30.406 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:30 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:30.406 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:30 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:30.406 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:30 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:30.406 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:30 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:30.406 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:30 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:30.406 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:30 ntpd[13695]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T18:48:30.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:30 ntpd[13695]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T18:48:30.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:30 ntpd[13695]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T18:48:30.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:30 ntpd[13695]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T18:48:30.407 INFO:teuthology.orchestra.run.smithi171.stdout:13 Sep 18:48:30 ntpd[13695]: ntpd: time slew +0.000654 s 2024-09-13T18:48:30.407 INFO:teuthology.orchestra.run.smithi171.stdout:ntpd: time slew +0.000654s 2024-09-13T18:48:30.407 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:30 ntpd[13695]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T18:48:30.408 INFO:teuthology.orchestra.run.smithi171.stderr:13 Sep 18:48:30 ntpd[13695]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T18:48:30.481 INFO:teuthology.orchestra.run.smithi171.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T18:48:30.481 INFO:teuthology.orchestra.run.smithi171.stdout:============================================================================== 2024-09-13T18:48:30.481 INFO:teuthology.orchestra.run.smithi171.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T18:48:30.481 INFO:teuthology.orchestra.run.smithi171.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T18:48:30.481 INFO:teuthology.orchestra.run.smithi171.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T18:48:30.481 INFO:teuthology.orchestra.run.smithi171.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T18:48:30.482 INFO:teuthology.run_tasks:Running task install... 2024-09-13T18:48:30.491 DEBUG:teuthology.task.install:project ceph 2024-09-13T18:48:30.491 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}} 2024-09-13T18:48:30.491 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'} 2024-09-13T18:48:30.491 INFO:teuthology.task.install:Using flavor: default 2024-09-13T18:48:30.497 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T18:48:30.497 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-09-13T18:48:30.497 DEBUG:teuthology.orchestra.run.smithi007:> sudo apt-key list | grep Ceph 2024-09-13T18:48:30.498 DEBUG:teuthology.orchestra.run.smithi171:> sudo apt-key list | grep Ceph 2024-09-13T18:48:30.596 INFO:teuthology.orchestra.run.smithi171.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-13T18:48:30.598 INFO:teuthology.orchestra.run.smithi007.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-13T18:48:30.654 INFO:teuthology.orchestra.run.smithi171.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T18:48:30.655 INFO:teuthology.orchestra.run.smithi171.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T18:48:30.656 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-13T18:48:30.656 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-13T18:48:30.656 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:48:30.670 INFO:teuthology.orchestra.run.smithi007.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T18:48:30.670 INFO:teuthology.orchestra.run.smithi007.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T18:48:30.670 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-13T18:48:30.670 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-13T18:48:30.671 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:48:30.809 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default/ 2024-09-13T18:48:30.809 INFO:teuthology.task.install.deb:Package version is 18.2.4-860-g6c6b083f-1focal 2024-09-13T18:48:30.825 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default/ 2024-09-13T18:48:30.825 INFO:teuthology.task.install.deb:Package version is 18.2.4-860-g6c6b083f-1focal 2024-09-13T18:48:30.934 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:48:30.935 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T18:48:30.947 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:48:30.947 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T18:48:30.953 DEBUG:teuthology.orchestra.run.smithi171:> sudo apt-get update 2024-09-13T18:48:30.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo apt-get update 2024-09-13T18:48:31.124 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T18:48:31.142 INFO:teuthology.orchestra.run.smithi171.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T18:48:31.143 INFO:teuthology.orchestra.run.smithi171.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T18:48:31.148 INFO:teuthology.orchestra.run.smithi171.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T18:48:31.162 INFO:teuthology.orchestra.run.smithi007.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal InRelease 2024-09-13T18:48:31.165 INFO:teuthology.orchestra.run.smithi171.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T18:48:31.183 INFO:teuthology.orchestra.run.smithi007.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-13T18:48:31.193 INFO:teuthology.orchestra.run.smithi171.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal InRelease 2024-09-13T18:48:31.206 INFO:teuthology.orchestra.run.smithi007.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal Release.gpg 2024-09-13T18:48:31.215 INFO:teuthology.orchestra.run.smithi171.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-13T18:48:31.223 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T18:48:31.228 INFO:teuthology.orchestra.run.smithi007.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-13T18:48:31.239 INFO:teuthology.orchestra.run.smithi171.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal Release.gpg 2024-09-13T18:48:31.259 INFO:teuthology.orchestra.run.smithi171.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-13T18:48:31.260 INFO:teuthology.orchestra.run.smithi007.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main i386 Packages [3,433 B] 2024-09-13T18:48:31.294 INFO:teuthology.orchestra.run.smithi171.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main i386 Packages [3,433 B] 2024-09-13T18:48:31.304 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T18:48:31.386 INFO:teuthology.orchestra.run.smithi007.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T18:48:31.795 INFO:teuthology.orchestra.run.smithi171.stdout:Fetched 28.7 kB in 1s (45.2 kB/s) 2024-09-13T18:48:31.811 INFO:teuthology.orchestra.run.smithi007.stdout:Fetched 28.7 kB in 1s (41.9 kB/s) 2024-09-13T18:48:32.941 INFO:teuthology.orchestra.run.smithi171.stdout:Reading package lists... 2024-09-13T18:48:32.968 DEBUG:teuthology.orchestra.run.smithi171:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-860-g6c6b083f-1focal cephadm=18.2.4-860-g6c6b083f-1focal ceph-mds=18.2.4-860-g6c6b083f-1focal ceph-mgr=18.2.4-860-g6c6b083f-1focal ceph-common=18.2.4-860-g6c6b083f-1focal ceph-fuse=18.2.4-860-g6c6b083f-1focal ceph-test=18.2.4-860-g6c6b083f-1focal ceph-volume=18.2.4-860-g6c6b083f-1focal radosgw=18.2.4-860-g6c6b083f-1focal python3-rados=18.2.4-860-g6c6b083f-1focal python3-rgw=18.2.4-860-g6c6b083f-1focal python3-cephfs=18.2.4-860-g6c6b083f-1focal python3-rbd=18.2.4-860-g6c6b083f-1focal libcephfs2=18.2.4-860-g6c6b083f-1focal libcephfs-dev=18.2.4-860-g6c6b083f-1focal librados2=18.2.4-860-g6c6b083f-1focal librbd1=18.2.4-860-g6c6b083f-1focal rbd-fuse=18.2.4-860-g6c6b083f-1focal python3-cephfs=18.2.4-860-g6c6b083f-1focal cephfs-shell=18.2.4-860-g6c6b083f-1focal cephfs-top=18.2.4-860-g6c6b083f-1focal cephfs-mirror=18.2.4-860-g6c6b083f-1focal 2024-09-13T18:48:33.014 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2024-09-13T18:48:33.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-860-g6c6b083f-1focal cephadm=18.2.4-860-g6c6b083f-1focal ceph-mds=18.2.4-860-g6c6b083f-1focal ceph-mgr=18.2.4-860-g6c6b083f-1focal ceph-common=18.2.4-860-g6c6b083f-1focal ceph-fuse=18.2.4-860-g6c6b083f-1focal ceph-test=18.2.4-860-g6c6b083f-1focal ceph-volume=18.2.4-860-g6c6b083f-1focal radosgw=18.2.4-860-g6c6b083f-1focal python3-rados=18.2.4-860-g6c6b083f-1focal python3-rgw=18.2.4-860-g6c6b083f-1focal python3-cephfs=18.2.4-860-g6c6b083f-1focal python3-rbd=18.2.4-860-g6c6b083f-1focal libcephfs2=18.2.4-860-g6c6b083f-1focal libcephfs-dev=18.2.4-860-g6c6b083f-1focal librados2=18.2.4-860-g6c6b083f-1focal librbd1=18.2.4-860-g6c6b083f-1focal rbd-fuse=18.2.4-860-g6c6b083f-1focal python3-cephfs=18.2.4-860-g6c6b083f-1focal cephfs-shell=18.2.4-860-g6c6b083f-1focal cephfs-top=18.2.4-860-g6c6b083f-1focal cephfs-mirror=18.2.4-860-g6c6b083f-1focal 2024-09-13T18:48:33.041 INFO:teuthology.orchestra.run.smithi171.stdout:Reading package lists... 2024-09-13T18:48:33.113 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2024-09-13T18:48:33.241 INFO:teuthology.orchestra.run.smithi171.stdout:Building dependency tree... 2024-09-13T18:48:33.242 INFO:teuthology.orchestra.run.smithi171.stdout:Reading state information... 2024-09-13T18:48:33.336 INFO:teuthology.orchestra.run.smithi007.stdout:Building dependency tree... 2024-09-13T18:48:33.337 INFO:teuthology.orchestra.run.smithi007.stdout:Reading state information... 2024-09-13T18:48:33.435 INFO:teuthology.orchestra.run.smithi171.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T18:48:33.437 INFO:teuthology.orchestra.run.smithi171.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T18:48:33.437 INFO:teuthology.orchestra.run.smithi171.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T18:48:33.437 INFO:teuthology.orchestra.run.smithi171.stdout:The following additional packages will be installed: 2024-09-13T18:48:33.437 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T18:48:33.437 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-13T18:48:33.437 INFO:teuthology.orchestra.run.smithi171.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-13T18:48:33.438 INFO:teuthology.orchestra.run.smithi171.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-13T18:48:33.438 INFO:teuthology.orchestra.run.smithi171.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-13T18:48:33.439 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-13T18:48:33.440 INFO:teuthology.orchestra.run.smithi171.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T18:48:33.440 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T18:48:33.440 INFO:teuthology.orchestra.run.smithi171.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T18:48:33.440 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-13T18:48:33.440 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-13T18:48:33.440 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-13T18:48:33.440 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-13T18:48:33.440 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-13T18:48:33.440 INFO:teuthology.orchestra.run.smithi171.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T18:48:33.440 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-13T18:48:33.440 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2024-09-13T18:48:33.440 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T18:48:33.440 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T18:48:33.441 INFO:teuthology.orchestra.run.smithi171.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T18:48:33.441 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T18:48:33.441 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T18:48:33.441 INFO:teuthology.orchestra.run.smithi171.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-13T18:48:33.441 INFO:teuthology.orchestra.run.smithi171.stdout:Suggested packages: 2024-09-13T18:48:33.441 INFO:teuthology.orchestra.run.smithi171.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-13T18:48:33.441 INFO:teuthology.orchestra.run.smithi171.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-13T18:48:33.441 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-13T18:48:33.441 INFO:teuthology.orchestra.run.smithi171.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-13T18:48:33.441 INFO:teuthology.orchestra.run.smithi171.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-13T18:48:33.441 INFO:teuthology.orchestra.run.smithi171.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-13T18:48:33.441 INFO:teuthology.orchestra.run.smithi171.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-13T18:48:33.441 INFO:teuthology.orchestra.run.smithi171.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T18:48:33.441 INFO:teuthology.orchestra.run.smithi171.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-13T18:48:33.442 INFO:teuthology.orchestra.run.smithi171.stdout:Recommended packages: 2024-09-13T18:48:33.442 INFO:teuthology.orchestra.run.smithi171.stdout: btrfs-tools 2024-09-13T18:48:33.547 INFO:teuthology.orchestra.run.smithi007.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T18:48:33.548 INFO:teuthology.orchestra.run.smithi007.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T18:48:33.548 INFO:teuthology.orchestra.run.smithi007.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T18:48:33.549 INFO:teuthology.orchestra.run.smithi007.stdout:The following additional packages will be installed: 2024-09-13T18:48:33.549 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T18:48:33.550 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-13T18:48:33.550 INFO:teuthology.orchestra.run.smithi007.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-13T18:48:33.551 INFO:teuthology.orchestra.run.smithi007.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-13T18:48:33.551 INFO:teuthology.orchestra.run.smithi007.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-13T18:48:33.551 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-13T18:48:33.551 INFO:teuthology.orchestra.run.smithi007.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T18:48:33.551 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T18:48:33.551 INFO:teuthology.orchestra.run.smithi007.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T18:48:33.551 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-13T18:48:33.551 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-13T18:48:33.551 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-13T18:48:33.551 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-13T18:48:33.551 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-13T18:48:33.551 INFO:teuthology.orchestra.run.smithi007.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T18:48:33.551 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-13T18:48:33.552 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2024-09-13T18:48:33.552 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T18:48:33.552 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T18:48:33.552 INFO:teuthology.orchestra.run.smithi007.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T18:48:33.552 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T18:48:33.552 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T18:48:33.552 INFO:teuthology.orchestra.run.smithi007.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-13T18:48:33.552 INFO:teuthology.orchestra.run.smithi007.stdout:Suggested packages: 2024-09-13T18:48:33.553 INFO:teuthology.orchestra.run.smithi007.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-13T18:48:33.553 INFO:teuthology.orchestra.run.smithi007.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-13T18:48:33.553 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-13T18:48:33.553 INFO:teuthology.orchestra.run.smithi007.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-13T18:48:33.553 INFO:teuthology.orchestra.run.smithi007.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-13T18:48:33.553 INFO:teuthology.orchestra.run.smithi007.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-13T18:48:33.553 INFO:teuthology.orchestra.run.smithi007.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-13T18:48:33.553 INFO:teuthology.orchestra.run.smithi007.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T18:48:33.553 INFO:teuthology.orchestra.run.smithi007.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-13T18:48:33.553 INFO:teuthology.orchestra.run.smithi007.stdout:Recommended packages: 2024-09-13T18:48:33.553 INFO:teuthology.orchestra.run.smithi007.stdout: btrfs-tools 2024-09-13T18:48:33.564 INFO:teuthology.orchestra.run.smithi171.stdout:The following NEW packages will be installed: 2024-09-13T18:48:33.564 INFO:teuthology.orchestra.run.smithi171.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T18:48:33.565 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T18:48:33.565 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T18:48:33.565 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-09-13T18:48:33.565 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-09-13T18:48:33.565 INFO:teuthology.orchestra.run.smithi171.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-09-13T18:48:33.565 INFO:teuthology.orchestra.run.smithi171.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-09-13T18:48:33.568 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-09-13T18:48:33.568 INFO:teuthology.orchestra.run.smithi171.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T18:48:33.568 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T18:48:33.569 INFO:teuthology.orchestra.run.smithi171.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T18:48:33.569 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2024-09-13T18:48:33.569 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-09-13T18:48:33.569 INFO:teuthology.orchestra.run.smithi171.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-13T18:48:33.569 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T18:48:33.569 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T18:48:33.569 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T18:48:33.569 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T18:48:33.569 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-09-13T18:48:33.569 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-09-13T18:48:33.570 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T18:48:33.570 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T18:48:33.570 INFO:teuthology.orchestra.run.smithi171.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T18:48:33.570 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T18:48:33.570 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T18:48:33.570 INFO:teuthology.orchestra.run.smithi171.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-13T18:48:33.570 INFO:teuthology.orchestra.run.smithi171.stdout:The following packages will be upgraded: 2024-09-13T18:48:33.570 INFO:teuthology.orchestra.run.smithi171.stdout: librados2 librbd1 2024-09-13T18:48:33.633 INFO:teuthology.orchestra.run.smithi171.stdout:2 upgraded, 116 newly installed, 0 to remove and 295 not upgraded. 2024-09-13T18:48:33.633 INFO:teuthology.orchestra.run.smithi171.stdout:Need to get 200 MB of archives. 2024-09-13T18:48:33.633 INFO:teuthology.orchestra.run.smithi171.stdout:After this operation, 979 MB of additional disk space will be used. 2024-09-13T18:48:33.633 INFO:teuthology.orchestra.run.smithi171.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-09-13T18:48:33.651 INFO:teuthology.orchestra.run.smithi171.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-13T18:48:33.659 INFO:teuthology.orchestra.run.smithi171.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-09-13T18:48:33.685 INFO:teuthology.orchestra.run.smithi007.stdout:The following NEW packages will be installed: 2024-09-13T18:48:33.686 INFO:teuthology.orchestra.run.smithi007.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T18:48:33.686 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T18:48:33.686 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T18:48:33.686 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror cephfs-shell cephfs-top fonts-mathjax jq libcephfs-dev 2024-09-13T18:48:33.686 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc 2024-09-13T18:48:33.686 INFO:teuthology.orchestra.run.smithi007.stdout: libjs-underscore liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 2024-09-13T18:48:33.686 INFO:teuthology.orchestra.run.smithi007.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 2024-09-13T18:48:33.686 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 2024-09-13T18:48:33.686 INFO:teuthology.orchestra.run.smithi007.stdout: nvme-cli python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T18:48:33.686 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T18:48:33.686 INFO:teuthology.orchestra.run.smithi007.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T18:48:33.686 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cmd2 2024-09-13T18:48:33.687 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama python3-cryptography python3-decorator python3-google-auth 2024-09-13T18:48:33.687 INFO:teuthology.orchestra.run.smithi007.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-13T18:48:33.687 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T18:48:33.687 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T18:48:33.687 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T18:48:33.687 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T18:48:33.687 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2024-09-13T18:48:33.687 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru 2024-09-13T18:48:33.687 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T18:48:33.687 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T18:48:33.687 INFO:teuthology.orchestra.run.smithi007.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T18:48:33.687 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T18:48:33.687 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T18:48:33.687 INFO:teuthology.orchestra.run.smithi007.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-13T18:48:33.688 INFO:teuthology.orchestra.run.smithi007.stdout:The following packages will be upgraded: 2024-09-13T18:48:33.688 INFO:teuthology.orchestra.run.smithi171.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-13T18:48:33.688 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 librbd1 2024-09-13T18:48:33.714 INFO:teuthology.orchestra.run.smithi171.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-13T18:48:33.715 INFO:teuthology.orchestra.run.smithi171.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-860-g6c6b083f-1focal [3,453 kB] 2024-09-13T18:48:33.717 INFO:teuthology.orchestra.run.smithi171.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-13T18:48:33.732 INFO:teuthology.orchestra.run.smithi171.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-13T18:48:33.776 INFO:teuthology.orchestra.run.smithi171.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-13T18:48:33.778 INFO:teuthology.orchestra.run.smithi171.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-13T18:48:33.784 INFO:teuthology.orchestra.run.smithi171.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-13T18:48:33.789 INFO:teuthology.orchestra.run.smithi171.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-13T18:48:33.789 INFO:teuthology.orchestra.run.smithi171.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-13T18:48:33.790 INFO:teuthology.orchestra.run.smithi171.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-13T18:48:33.790 INFO:teuthology.orchestra.run.smithi171.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-13T18:48:33.790 INFO:teuthology.orchestra.run.smithi171.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-13T18:48:33.790 INFO:teuthology.orchestra.run.smithi171.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-13T18:48:33.793 INFO:teuthology.orchestra.run.smithi171.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-13T18:48:33.810 INFO:teuthology.orchestra.run.smithi171.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-13T18:48:33.810 INFO:teuthology.orchestra.run.smithi171.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-13T18:48:33.810 INFO:teuthology.orchestra.run.smithi171.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-13T18:48:33.811 INFO:teuthology.orchestra.run.smithi171.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T18:48:33.811 INFO:teuthology.orchestra.run.smithi171.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-13T18:48:33.812 INFO:teuthology.orchestra.run.smithi171.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-13T18:48:33.814 INFO:teuthology.orchestra.run.smithi171.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-13T18:48:33.814 INFO:teuthology.orchestra.run.smithi171.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-13T18:48:33.816 INFO:teuthology.orchestra.run.smithi171.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-13T18:48:33.827 INFO:teuthology.orchestra.run.smithi171.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-13T18:48:33.836 INFO:teuthology.orchestra.run.smithi171.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-13T18:48:33.836 INFO:teuthology.orchestra.run.smithi007.stdout:2 upgraded, 116 newly installed, 0 to remove and 295 not upgraded. 2024-09-13T18:48:33.836 INFO:teuthology.orchestra.run.smithi007.stdout:Need to get 200 MB of archives. 2024-09-13T18:48:33.836 INFO:teuthology.orchestra.run.smithi007.stdout:After this operation, 979 MB of additional disk space will be used. 2024-09-13T18:48:33.836 INFO:teuthology.orchestra.run.smithi007.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-860-g6c6b083f-1focal [3,453 kB] 2024-09-13T18:48:33.842 INFO:teuthology.orchestra.run.smithi171.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-13T18:48:33.842 INFO:teuthology.orchestra.run.smithi171.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-13T18:48:33.844 INFO:teuthology.orchestra.run.smithi171.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-13T18:48:33.844 INFO:teuthology.orchestra.run.smithi171.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-13T18:48:33.846 INFO:teuthology.orchestra.run.smithi171.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-13T18:48:33.849 INFO:teuthology.orchestra.run.smithi171.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-13T18:48:33.849 INFO:teuthology.orchestra.run.smithi171.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-13T18:48:33.851 INFO:teuthology.orchestra.run.smithi171.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-13T18:48:33.861 INFO:teuthology.orchestra.run.smithi171.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-13T18:48:33.863 INFO:teuthology.orchestra.run.smithi171.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-13T18:48:33.864 INFO:teuthology.orchestra.run.smithi171.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-13T18:48:33.878 INFO:teuthology.orchestra.run.smithi171.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-13T18:48:33.879 INFO:teuthology.orchestra.run.smithi171.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-13T18:48:33.879 INFO:teuthology.orchestra.run.smithi171.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-13T18:48:33.879 INFO:teuthology.orchestra.run.smithi007.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2024-09-13T18:48:33.881 INFO:teuthology.orchestra.run.smithi171.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-13T18:48:33.881 INFO:teuthology.orchestra.run.smithi171.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-13T18:48:33.881 INFO:teuthology.orchestra.run.smithi171.stdout:Get:46 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-860-g6c6b083f-1focal [4,041 kB] 2024-09-13T18:48:33.882 INFO:teuthology.orchestra.run.smithi171.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-13T18:48:33.884 INFO:teuthology.orchestra.run.smithi171.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T18:48:33.895 INFO:teuthology.orchestra.run.smithi171.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-13T18:48:33.898 INFO:teuthology.orchestra.run.smithi171.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-13T18:48:33.901 INFO:teuthology.orchestra.run.smithi171.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-13T18:48:33.912 INFO:teuthology.orchestra.run.smithi171.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-13T18:48:33.913 INFO:teuthology.orchestra.run.smithi171.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-13T18:48:33.913 INFO:teuthology.orchestra.run.smithi171.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-13T18:48:33.949 INFO:teuthology.orchestra.run.smithi171.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-860-g6c6b083f-1focal [747 kB] 2024-09-13T18:48:33.960 INFO:teuthology.orchestra.run.smithi171.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-860-g6c6b083f-1focal [388 kB] 2024-09-13T18:48:33.963 INFO:teuthology.orchestra.run.smithi007.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-13T18:48:33.967 INFO:teuthology.orchestra.run.smithi171.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-860-g6c6b083f-1focal [31.9 kB] 2024-09-13T18:48:33.967 INFO:teuthology.orchestra.run.smithi171.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-860-g6c6b083f-1focal [200 kB] 2024-09-13T18:48:33.978 INFO:teuthology.orchestra.run.smithi171.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-860-g6c6b083f-1focal [62.5 kB] 2024-09-13T18:48:33.979 INFO:teuthology.orchestra.run.smithi171.stdout:Get:60 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-860-g6c6b083f-1focal [368 kB] 2024-09-13T18:48:33.986 INFO:teuthology.orchestra.run.smithi171.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-860-g6c6b083f-1focal [6,634 kB] 2024-09-13T18:48:33.993 INFO:teuthology.orchestra.run.smithi171.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-13T18:48:33.997 INFO:teuthology.orchestra.run.smithi007.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2024-09-13T18:48:34.008 INFO:teuthology.orchestra.run.smithi171.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-13T18:48:34.085 INFO:teuthology.orchestra.run.smithi171.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-13T18:48:34.106 INFO:teuthology.orchestra.run.smithi171.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-13T18:48:34.107 INFO:teuthology.orchestra.run.smithi171.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-13T18:48:34.107 INFO:teuthology.orchestra.run.smithi171.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-13T18:48:34.107 INFO:teuthology.orchestra.run.smithi171.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-13T18:48:34.107 INFO:teuthology.orchestra.run.smithi171.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-13T18:48:34.108 INFO:teuthology.orchestra.run.smithi171.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-13T18:48:34.108 INFO:teuthology.orchestra.run.smithi171.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-13T18:48:34.109 INFO:teuthology.orchestra.run.smithi171.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-13T18:48:34.110 INFO:teuthology.orchestra.run.smithi171.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-13T18:48:34.121 INFO:teuthology.orchestra.run.smithi171.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-860-g6c6b083f-1focal [113 kB] 2024-09-13T18:48:34.122 INFO:teuthology.orchestra.run.smithi007.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-13T18:48:34.122 INFO:teuthology.orchestra.run.smithi171.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-860-g6c6b083f-1focal [478 kB] 2024-09-13T18:48:34.124 INFO:teuthology.orchestra.run.smithi171.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-13T18:48:34.127 INFO:teuthology.orchestra.run.smithi171.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-13T18:48:34.131 INFO:teuthology.orchestra.run.smithi171.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-860-g6c6b083f-1focal [25.2 MB] 2024-09-13T18:48:34.141 INFO:teuthology.orchestra.run.smithi171.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-13T18:48:34.143 INFO:teuthology.orchestra.run.smithi171.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-13T18:48:34.143 INFO:teuthology.orchestra.run.smithi171.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-13T18:48:34.147 INFO:teuthology.orchestra.run.smithi171.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-13T18:48:34.150 INFO:teuthology.orchestra.run.smithi171.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-09-13T18:48:34.150 INFO:teuthology.orchestra.run.smithi171.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-13T18:48:34.167 INFO:teuthology.orchestra.run.smithi171.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-13T18:48:34.170 INFO:teuthology.orchestra.run.smithi171.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-13T18:48:34.171 INFO:teuthology.orchestra.run.smithi171.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-13T18:48:34.196 INFO:teuthology.orchestra.run.smithi007.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-860-g6c6b083f-1focal [4,041 kB] 2024-09-13T18:48:34.214 INFO:teuthology.orchestra.run.smithi171.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-13T18:48:34.214 INFO:teuthology.orchestra.run.smithi171.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-13T18:48:34.214 INFO:teuthology.orchestra.run.smithi171.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-13T18:48:34.215 INFO:teuthology.orchestra.run.smithi171.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-13T18:48:34.227 INFO:teuthology.orchestra.run.smithi171.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T18:48:34.227 INFO:teuthology.orchestra.run.smithi171.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-13T18:48:34.228 INFO:teuthology.orchestra.run.smithi171.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-13T18:48:34.228 INFO:teuthology.orchestra.run.smithi171.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-13T18:48:34.229 INFO:teuthology.orchestra.run.smithi171.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-13T18:48:34.252 INFO:teuthology.orchestra.run.smithi007.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-13T18:48:34.268 INFO:teuthology.orchestra.run.smithi007.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-13T18:48:34.329 INFO:teuthology.orchestra.run.smithi007.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-13T18:48:34.496 INFO:teuthology.orchestra.run.smithi007.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-860-g6c6b083f-1focal [747 kB] 2024-09-13T18:48:34.525 INFO:teuthology.orchestra.run.smithi007.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-13T18:48:34.534 INFO:teuthology.orchestra.run.smithi007.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-13T18:48:34.556 INFO:teuthology.orchestra.run.smithi007.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-860-g6c6b083f-1focal [388 kB] 2024-09-13T18:48:34.559 INFO:teuthology.orchestra.run.smithi007.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-13T18:48:34.567 INFO:teuthology.orchestra.run.smithi007.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-13T18:48:34.568 INFO:teuthology.orchestra.run.smithi007.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-13T18:48:34.573 INFO:teuthology.orchestra.run.smithi007.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-13T18:48:34.574 INFO:teuthology.orchestra.run.smithi007.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-13T18:48:34.586 INFO:teuthology.orchestra.run.smithi007.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-13T18:48:34.587 INFO:teuthology.orchestra.run.smithi007.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-13T18:48:34.609 INFO:teuthology.orchestra.run.smithi007.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-13T18:48:34.612 INFO:teuthology.orchestra.run.smithi007.stdout:Get:22 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-860-g6c6b083f-1focal [31.9 kB] 2024-09-13T18:48:34.613 INFO:teuthology.orchestra.run.smithi007.stdout:Get:23 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-860-g6c6b083f-1focal [200 kB] 2024-09-13T18:48:34.617 INFO:teuthology.orchestra.run.smithi007.stdout:Get:24 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-860-g6c6b083f-1focal [62.5 kB] 2024-09-13T18:48:34.619 INFO:teuthology.orchestra.run.smithi007.stdout:Get:25 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-860-g6c6b083f-1focal [368 kB] 2024-09-13T18:48:34.623 INFO:teuthology.orchestra.run.smithi007.stdout:Get:26 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-860-g6c6b083f-1focal [6,634 kB] 2024-09-13T18:48:34.674 INFO:teuthology.orchestra.run.smithi007.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-13T18:48:34.674 INFO:teuthology.orchestra.run.smithi007.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-13T18:48:34.674 INFO:teuthology.orchestra.run.smithi007.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-13T18:48:34.676 INFO:teuthology.orchestra.run.smithi007.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T18:48:34.677 INFO:teuthology.orchestra.run.smithi007.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-13T18:48:34.677 INFO:teuthology.orchestra.run.smithi007.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-13T18:48:34.679 INFO:teuthology.orchestra.run.smithi007.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-13T18:48:34.680 INFO:teuthology.orchestra.run.smithi007.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-13T18:48:34.688 INFO:teuthology.orchestra.run.smithi007.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-13T18:48:34.731 INFO:teuthology.orchestra.run.smithi171.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-860-g6c6b083f-1focal [9,609 kB] 2024-09-13T18:48:34.769 INFO:teuthology.orchestra.run.smithi007.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-13T18:48:34.770 INFO:teuthology.orchestra.run.smithi007.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-13T18:48:34.778 INFO:teuthology.orchestra.run.smithi007.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-13T18:48:34.778 INFO:teuthology.orchestra.run.smithi007.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-13T18:48:34.779 INFO:teuthology.orchestra.run.smithi007.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-13T18:48:34.780 INFO:teuthology.orchestra.run.smithi007.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-13T18:48:34.781 INFO:teuthology.orchestra.run.smithi007.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-13T18:48:34.784 INFO:teuthology.orchestra.run.smithi007.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-13T18:48:34.851 INFO:teuthology.orchestra.run.smithi007.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-13T18:48:34.854 INFO:teuthology.orchestra.run.smithi007.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-13T18:48:34.889 INFO:teuthology.orchestra.run.smithi171.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-860-g6c6b083f-1focal [238 kB] 2024-09-13T18:48:34.895 INFO:teuthology.orchestra.run.smithi171.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-860-g6c6b083f-1focal [147 kB] 2024-09-13T18:48:34.897 INFO:teuthology.orchestra.run.smithi171.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-860-g6c6b083f-1focal [1,576 kB] 2024-09-13T18:48:34.932 INFO:teuthology.orchestra.run.smithi007.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-13T18:48:34.935 INFO:teuthology.orchestra.run.smithi007.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-13T18:48:34.936 INFO:teuthology.orchestra.run.smithi171.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-860-g6c6b083f-1focal [6,641 kB] 2024-09-13T18:48:34.938 INFO:teuthology.orchestra.run.smithi007.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-13T18:48:34.938 INFO:teuthology.orchestra.run.smithi007.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-13T18:48:34.939 INFO:teuthology.orchestra.run.smithi007.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-13T18:48:34.940 INFO:teuthology.orchestra.run.smithi007.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-13T18:48:34.941 INFO:teuthology.orchestra.run.smithi007.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-13T18:48:34.942 INFO:teuthology.orchestra.run.smithi007.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-13T18:48:34.981 INFO:teuthology.orchestra.run.smithi007.stdout:Get:54 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-860-g6c6b083f-1focal [113 kB] 2024-09-13T18:48:34.983 INFO:teuthology.orchestra.run.smithi007.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-860-g6c6b083f-1focal [478 kB] 2024-09-13T18:48:35.008 INFO:teuthology.orchestra.run.smithi007.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-860-g6c6b083f-1focal [25.2 MB] 2024-09-13T18:48:35.013 INFO:teuthology.orchestra.run.smithi007.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-13T18:48:35.018 INFO:teuthology.orchestra.run.smithi007.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T18:48:35.090 INFO:teuthology.orchestra.run.smithi171.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-860-g6c6b083f-1focal [24.5 MB] 2024-09-13T18:48:35.095 INFO:teuthology.orchestra.run.smithi007.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-13T18:48:35.097 INFO:teuthology.orchestra.run.smithi007.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-13T18:48:35.100 INFO:teuthology.orchestra.run.smithi007.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-13T18:48:35.102 INFO:teuthology.orchestra.run.smithi007.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-13T18:48:35.103 INFO:teuthology.orchestra.run.smithi007.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-13T18:48:35.104 INFO:teuthology.orchestra.run.smithi007.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-13T18:48:35.426 INFO:teuthology.orchestra.run.smithi007.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-13T18:48:35.455 INFO:teuthology.orchestra.run.smithi007.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-13T18:48:35.457 INFO:teuthology.orchestra.run.smithi007.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-13T18:48:35.526 INFO:teuthology.orchestra.run.smithi007.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-13T18:48:35.526 INFO:teuthology.orchestra.run.smithi007.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-13T18:48:35.527 INFO:teuthology.orchestra.run.smithi007.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-13T18:48:35.527 INFO:teuthology.orchestra.run.smithi007.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-13T18:48:35.528 INFO:teuthology.orchestra.run.smithi007.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-13T18:48:35.528 INFO:teuthology.orchestra.run.smithi007.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-13T18:48:35.528 INFO:teuthology.orchestra.run.smithi007.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-13T18:48:35.589 INFO:teuthology.orchestra.run.smithi007.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-13T18:48:35.590 INFO:teuthology.orchestra.run.smithi007.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-13T18:48:35.670 INFO:teuthology.orchestra.run.smithi007.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-13T18:48:35.676 INFO:teuthology.orchestra.run.smithi007.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-13T18:48:35.679 INFO:teuthology.orchestra.run.smithi007.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-13T18:48:35.682 INFO:teuthology.orchestra.run.smithi007.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-13T18:48:35.683 INFO:teuthology.orchestra.run.smithi007.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-13T18:48:35.686 INFO:teuthology.orchestra.run.smithi007.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-13T18:48:35.688 INFO:teuthology.orchestra.run.smithi007.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2024-09-13T18:48:35.688 INFO:teuthology.orchestra.run.smithi007.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-13T18:48:35.763 INFO:teuthology.orchestra.run.smithi007.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-13T18:48:35.768 INFO:teuthology.orchestra.run.smithi007.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-13T18:48:35.825 INFO:teuthology.orchestra.run.smithi171.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-860-g6c6b083f-1focal [13.7 kB] 2024-09-13T18:48:35.826 INFO:teuthology.orchestra.run.smithi171.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-860-g6c6b083f-1focal [857 kB] 2024-09-13T18:48:35.834 INFO:teuthology.orchestra.run.smithi007.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-13T18:48:35.839 INFO:teuthology.orchestra.run.smithi171.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-860-g6c6b083f-1focal [2,296 kB] 2024-09-13T18:48:35.855 INFO:teuthology.orchestra.run.smithi171.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-860-g6c6b083f-1focal [239 kB] 2024-09-13T18:48:35.857 INFO:teuthology.orchestra.run.smithi171.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-860-g6c6b083f-1focal [143 kB] 2024-09-13T18:48:35.858 INFO:teuthology.orchestra.run.smithi171.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-860-g6c6b083f-1focal [7,015 kB] 2024-09-13T18:48:35.945 INFO:teuthology.orchestra.run.smithi007.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-13T18:48:36.000 INFO:teuthology.orchestra.run.smithi007.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-13T18:48:36.000 INFO:teuthology.orchestra.run.smithi007.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-13T18:48:36.002 INFO:teuthology.orchestra.run.smithi007.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-13T18:48:36.020 INFO:teuthology.orchestra.run.smithi007.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T18:48:36.021 INFO:teuthology.orchestra.run.smithi007.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-13T18:48:36.022 INFO:teuthology.orchestra.run.smithi007.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-13T18:48:36.022 INFO:teuthology.orchestra.run.smithi007.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-13T18:48:36.022 INFO:teuthology.orchestra.run.smithi007.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-13T18:48:36.061 INFO:teuthology.orchestra.run.smithi171.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-860-g6c6b083f-1focal [7,894 kB] 2024-09-13T18:48:36.193 INFO:teuthology.orchestra.run.smithi007.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-860-g6c6b083f-1focal [9,609 kB] 2024-09-13T18:48:36.251 INFO:teuthology.orchestra.run.smithi171.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-860-g6c6b083f-1focal [13.9 kB] 2024-09-13T18:48:36.251 INFO:teuthology.orchestra.run.smithi171.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-860-g6c6b083f-1focal [47.1 MB] 2024-09-13T18:48:36.478 INFO:teuthology.orchestra.run.smithi007.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-860-g6c6b083f-1focal [238 kB] 2024-09-13T18:48:36.482 INFO:teuthology.orchestra.run.smithi007.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-860-g6c6b083f-1focal [147 kB] 2024-09-13T18:48:36.485 INFO:teuthology.orchestra.run.smithi007.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-860-g6c6b083f-1focal [1,576 kB] 2024-09-13T18:48:36.541 INFO:teuthology.orchestra.run.smithi007.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-860-g6c6b083f-1focal [6,641 kB] 2024-09-13T18:48:36.810 INFO:teuthology.orchestra.run.smithi007.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-860-g6c6b083f-1focal [24.5 MB] 2024-09-13T18:48:37.790 INFO:teuthology.orchestra.run.smithi007.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-860-g6c6b083f-1focal [13.7 kB] 2024-09-13T18:48:37.790 INFO:teuthology.orchestra.run.smithi007.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-860-g6c6b083f-1focal [857 kB] 2024-09-13T18:48:37.804 INFO:teuthology.orchestra.run.smithi007.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-860-g6c6b083f-1focal [2,296 kB] 2024-09-13T18:48:37.820 INFO:teuthology.orchestra.run.smithi007.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-860-g6c6b083f-1focal [239 kB] 2024-09-13T18:48:37.822 INFO:teuthology.orchestra.run.smithi007.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-860-g6c6b083f-1focal [143 kB] 2024-09-13T18:48:37.823 INFO:teuthology.orchestra.run.smithi007.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-860-g6c6b083f-1focal [7,015 kB] 2024-09-13T18:48:37.824 INFO:teuthology.orchestra.run.smithi171.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-860-g6c6b083f-1focal [120 kB] 2024-09-13T18:48:37.826 INFO:teuthology.orchestra.run.smithi171.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.2.4-860-g6c6b083f-1focal [271 kB] 2024-09-13T18:48:37.831 INFO:teuthology.orchestra.run.smithi171.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.2.4-860-g6c6b083f-1focal [33.0 kB] 2024-09-13T18:48:37.831 INFO:teuthology.orchestra.run.smithi171.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.2.4-860-g6c6b083f-1focal [22.8 kB] 2024-09-13T18:48:37.832 INFO:teuthology.orchestra.run.smithi171.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-860-g6c6b083f-1focal [37.4 kB] 2024-09-13T18:48:37.833 INFO:teuthology.orchestra.run.smithi171.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-860-g6c6b083f-1focal [13.5 MB] 2024-09-13T18:48:38.007 INFO:teuthology.orchestra.run.smithi007.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-860-g6c6b083f-1focal [7,894 kB] 2024-09-13T18:48:38.214 INFO:teuthology.orchestra.run.smithi007.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-860-g6c6b083f-1focal [13.9 kB] 2024-09-13T18:48:38.215 INFO:teuthology.orchestra.run.smithi007.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-860-g6c6b083f-1focal [47.1 MB] 2024-09-13T18:48:38.223 INFO:teuthology.orchestra.run.smithi171.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-860-g6c6b083f-1focal [108 kB] 2024-09-13T18:48:38.892 INFO:teuthology.orchestra.run.smithi171.stdout:Fetched 200 MB in 5s (42.9 MB/s) 2024-09-13T18:48:39.015 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-13T18:48:39.072 INFO:teuthology.orchestra.run.smithi171.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-13T18:48:39.077 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-09-13T18:48:39.114 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-09-13T18:48:39.239 INFO:teuthology.orchestra.run.smithi007.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-860-g6c6b083f-1focal [120 kB] 2024-09-13T18:48:39.241 INFO:teuthology.orchestra.run.smithi007.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephfs-mirror amd64 18.2.4-860-g6c6b083f-1focal [271 kB] 2024-09-13T18:48:39.245 INFO:teuthology.orchestra.run.smithi007.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 18.2.4-860-g6c6b083f-1focal [33.0 kB] 2024-09-13T18:48:39.246 INFO:teuthology.orchestra.run.smithi007.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 cephfs-top all 18.2.4-860-g6c6b083f-1focal [22.8 kB] 2024-09-13T18:48:39.247 INFO:teuthology.orchestra.run.smithi007.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-860-g6c6b083f-1focal [37.4 kB] 2024-09-13T18:48:39.300 INFO:teuthology.orchestra.run.smithi007.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-860-g6c6b083f-1focal [13.5 MB] 2024-09-13T18:48:39.459 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T18:48:39.475 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-13T18:48:39.489 INFO:teuthology.orchestra.run.smithi007.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-860-g6c6b083f-1focal [108 kB] 2024-09-13T18:48:39.550 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T18:48:39.879 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-cmd2. 2024-09-13T18:48:39.895 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-09-13T18:48:39.937 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-09-13T18:48:40.168 INFO:teuthology.orchestra.run.smithi007.stdout:Fetched 200 MB in 6s (34.4 MB/s) 2024-09-13T18:48:40.266 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-13T18:48:40.283 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-13T18:48:40.315 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-13T18:48:40.349 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T18:48:40.372 INFO:teuthology.orchestra.run.smithi007.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-13T18:48:40.377 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2024-09-13T18:48:40.443 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2024-09-13T18:48:40.729 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T18:48:40.733 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T18:48:40.744 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-13T18:48:40.753 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-13T18:48:40.812 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T18:48:40.816 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T18:48:41.078 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-cmd2. 2024-09-13T18:48:41.097 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2024-09-13T18:48:41.149 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2024-09-13T18:48:41.174 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T18:48:41.191 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-13T18:48:41.232 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T18:48:41.514 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-13T18:48:41.533 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-13T18:48:41.614 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T18:48:41.695 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T18:48:41.711 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T18:48:41.797 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:48:42.010 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T18:48:42.027 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-13T18:48:42.068 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T18:48:42.349 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T18:48:42.364 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T18:48:42.396 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T18:48:42.415 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../005-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-13T18:48:42.432 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:48:42.446 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T18:48:42.794 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T18:48:42.811 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T18:48:42.853 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:48:42.977 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T18:48:42.997 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../006-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T18:48:43.045 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:48:43.273 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-13T18:48:43.289 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-13T18:48:43.331 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T18:48:43.580 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T18:48:43.598 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../007-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T18:48:43.663 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:48:43.698 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../010-librbd1_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:43.811 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking librbd1 (18.2.4-860-g6c6b083f-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T18:48:44.058 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T18:48:44.075 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../008-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T18:48:44.166 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:48:44.567 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../011-librados2_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:44.570 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-13T18:48:44.587 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../009-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-13T18:48:44.628 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T18:48:44.716 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking librados2 (18.2.4-860-g6c6b083f-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T18:48:45.014 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../010-librbd1_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:45.173 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking librbd1 (18.2.4-860-g6c6b083f-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T18:48:45.464 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T18:48:45.481 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../012-libcephfs2_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:45.521 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libcephfs2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:45.841 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-rados. 2024-09-13T18:48:45.859 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../013-python3-rados_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:45.885 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../011-librados2_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:45.926 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-rados (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:46.011 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking librados2 (18.2.4-860-g6c6b083f-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T18:48:46.311 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T18:48:46.327 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:48:46.369 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-ceph-argparse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:46.655 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T18:48:46.671 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../015-python3-cephfs_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:46.714 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-cephfs (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:46.765 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T18:48:46.783 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../012-libcephfs2_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:46.843 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libcephfs2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:47.017 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T18:48:47.036 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../016-python3-ceph-common_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:48:47.075 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:47.218 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-rados. 2024-09-13T18:48:47.235 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../013-python3-rados_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:47.276 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-rados (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:47.437 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T18:48:47.454 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2024-09-13T18:48:47.514 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-13T18:48:47.654 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T18:48:47.671 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../014-python3-ceph-argparse_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:48:47.712 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-ceph-argparse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:47.774 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T18:48:47.791 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../018-python3-rbd_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:47.857 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-rbd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:48.006 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T18:48:48.023 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../015-python3-cephfs_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:48.064 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-cephfs (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:48.212 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-13T18:48:48.231 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-13T18:48:48.270 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T18:48:48.417 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T18:48:48.437 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../016-python3-ceph-common_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:48:48.476 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:48.649 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-13T18:48:48.667 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-13T18:48:48.707 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T18:48:48.829 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T18:48:48.849 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../017-python3-prettytable_0.7.2-5_all.deb ... 2024-09-13T18:48:48.887 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-13T18:48:49.019 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T18:48:49.036 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-13T18:48:49.069 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T18:48:49.165 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T18:48:49.181 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../018-python3-rbd_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:49.223 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-rbd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:49.473 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package librgw2. 2024-09-13T18:48:49.489 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../022-librgw2_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:49.531 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking librgw2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:49.593 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-13T18:48:49.613 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../019-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-13T18:48:49.651 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T18:48:50.012 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-13T18:48:50.030 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../020-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-13T18:48:50.087 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T18:48:50.463 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T18:48:50.480 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../023-python3-rgw_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:50.526 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-rgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:50.565 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T18:48:50.585 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../021-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-13T18:48:50.707 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T18:48:50.880 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package liboath0. 2024-09-13T18:48:50.895 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-13T18:48:50.938 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-13T18:48:51.085 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package librgw2. 2024-09-13T18:48:51.103 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../022-librgw2_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:51.160 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking librgw2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:51.258 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T18:48:51.277 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../025-libradosstriper1_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:51.316 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libradosstriper1 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:51.779 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package ceph-common. 2024-09-13T18:48:51.795 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../026-ceph-common_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:51.828 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:52.071 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T18:48:52.089 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../023-python3-rgw_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:52.129 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-rgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:52.482 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package liboath0. 2024-09-13T18:48:52.501 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../024-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-13T18:48:52.573 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-13T18:48:52.918 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T18:48:52.937 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../025-libradosstriper1_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:52.976 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libradosstriper1 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:53.447 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-common. 2024-09-13T18:48:53.465 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../026-ceph-common_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:53.505 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:54.327 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package ceph-base. 2024-09-13T18:48:54.343 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../027-ceph-base_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:54.395 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking ceph-base (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:55.750 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T18:48:55.767 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-13T18:48:55.808 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-13T18:48:55.986 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-base. 2024-09-13T18:48:56.001 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../027-ceph-base_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:48:56.069 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-base (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:48:56.203 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T18:48:56.220 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2024-09-13T18:48:56.285 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-13T18:48:56.633 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T18:48:56.649 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2024-09-13T18:48:56.691 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-13T18:48:57.019 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T18:48:57.035 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-13T18:48:57.077 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-13T18:48:57.348 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T18:48:57.368 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../028-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-13T18:48:57.398 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-mako. 2024-09-13T18:48:57.406 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-13T18:48:57.415 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-13T18:48:57.456 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T18:48:57.769 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T18:48:57.788 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../029-python3-natsort_7.0.1-1_all.deb ... 2024-09-13T18:48:57.802 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T18:48:57.821 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T18:48:57.849 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-13T18:48:57.860 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T18:48:58.164 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T18:48:58.180 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../030-python3-logutils_0.3.3-7_all.deb ... 2024-09-13T18:48:58.238 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T18:48:58.246 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-13T18:48:58.258 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-13T18:48:58.297 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T18:48:58.585 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T18:48:58.602 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../031-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-13T18:48:58.617 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-webob. 2024-09-13T18:48:58.637 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-13T18:48:58.641 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-13T18:48:58.675 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T18:48:58.910 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-mako. 2024-09-13T18:48:58.929 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../032-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-13T18:48:58.969 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T18:48:59.012 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-13T18:48:59.028 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-13T18:48:59.070 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T18:48:59.306 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T18:48:59.323 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../033-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T18:48:59.364 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T18:48:59.407 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-bs4. 2024-09-13T18:48:59.426 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2024-09-13T18:48:59.483 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-13T18:48:59.700 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T18:48:59.721 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../034-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-13T18:48:59.758 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T18:48:59.844 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T18:48:59.860 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-13T18:48:59.905 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T18:49:00.095 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-webob. 2024-09-13T18:49:00.112 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../035-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-13T18:49:00.153 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T18:49:00.223 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T18:49:00.242 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2024-09-13T18:49:00.281 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-13T18:49:00.489 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-13T18:49:00.509 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../036-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-13T18:49:00.547 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T18:49:00.610 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-paste. 2024-09-13T18:49:00.629 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-13T18:49:00.668 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T18:49:00.884 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-bs4. 2024-09-13T18:49:00.902 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../037-python3-bs4_4.8.2-1_all.deb ... 2024-09-13T18:49:00.941 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-13T18:49:01.014 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T18:49:01.030 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-13T18:49:01.105 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T18:49:01.328 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T18:49:01.346 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../038-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-13T18:49:01.415 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T18:49:01.440 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T18:49:01.456 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-13T18:49:01.498 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-13T18:49:01.790 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T18:49:01.807 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-6_all.deb ... 2024-09-13T18:49:01.835 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T18:49:01.848 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-13T18:49:01.851 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2024-09-13T18:49:01.893 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-13T18:49:02.201 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-paste. 2024-09-13T18:49:02.218 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../040-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-13T18:49:02.230 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T18:49:02.249 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-13T18:49:02.259 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T18:49:02.288 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T18:49:02.571 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T18:49:02.587 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-13T18:49:02.629 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T18:49:02.684 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T18:49:02.702 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-13T18:49:02.744 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T18:49:02.974 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T18:49:02.990 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../042-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-13T18:49:03.032 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-13T18:49:03.082 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T18:49:03.099 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:49:03.140 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking ceph-mgr-modules-core (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:03.361 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T18:49:03.377 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../043-python3-webtest_2.0.32-2_all.deb ... 2024-09-13T18:49:03.419 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-13T18:49:03.530 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T18:49:03.545 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:03.588 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:03.788 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T18:49:03.805 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../044-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-13T18:49:03.872 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T18:49:03.983 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T18:49:04.002 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../048-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-13T18:49:04.041 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T18:49:04.248 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T18:49:04.264 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../045-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-13T18:49:04.305 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T18:49:04.375 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-cryptography. 2024-09-13T18:49:04.391 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../049-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-13T18:49:04.433 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T18:49:04.617 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T18:49:04.635 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../046-ceph-mgr-modules-core_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:49:04.675 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mgr-modules-core (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:04.838 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T18:49:04.855 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../050-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-13T18:49:04.896 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-13T18:49:05.080 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T18:49:05.098 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../047-libsqlite3-mod-ceph_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:05.146 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:05.491 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T18:49:05.508 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../048-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-13T18:49:05.541 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T18:49:05.819 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-cryptography. 2024-09-13T18:49:05.838 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../049-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-13T18:49:05.869 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T18:49:06.118 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T18:49:06.138 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../051-ceph-mgr_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:06.177 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking ceph-mgr (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:06.697 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T18:49:06.715 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../052-ceph-mon_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:06.754 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking ceph-mon (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:07.115 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T18:49:07.131 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../050-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-13T18:49:07.173 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-13T18:49:07.526 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T18:49:07.543 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../051-ceph-mgr_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:07.626 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mgr (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:07.717 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T18:49:07.733 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../053-ceph-osd_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:07.776 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking ceph-osd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:08.146 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T18:49:08.165 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../052-ceph-mon_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:08.204 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mon (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:09.125 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T18:49:09.144 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../053-ceph-osd_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:09.182 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-osd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:09.955 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package ceph. 2024-09-13T18:49:09.974 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../054-ceph_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:10.013 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:10.326 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T18:49:10.341 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../055-ceph-fuse_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:10.375 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking ceph-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:10.854 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T18:49:10.871 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../056-ceph-mds_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:10.912 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking ceph-mds (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:11.433 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package cephadm. 2024-09-13T18:49:11.451 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../057-cephadm_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:11.452 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph. 2024-09-13T18:49:11.470 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../054-ceph_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:11.516 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:11.527 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:11.845 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T18:49:11.864 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../058-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-13T18:49:11.897 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T18:49:11.902 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-13T18:49:11.917 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../055-ceph-fuse_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:11.955 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:12.273 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T18:49:12.289 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-13T18:49:12.331 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T18:49:12.358 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T18:49:12.375 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../056-ceph-mds_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:12.408 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mds (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:12.702 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T18:49:12.719 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:49:12.793 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking ceph-mgr-cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:12.970 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package cephadm. 2024-09-13T18:49:12.987 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../057-cephadm_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:13.028 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:13.164 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-jwt. 2024-09-13T18:49:13.182 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-13T18:49:13.222 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T18:49:13.373 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T18:49:13.389 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../058-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-13T18:49:13.431 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-13T18:49:13.559 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-attr. 2024-09-13T18:49:13.575 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../062-python3-attr_19.3.0-2_all.deb ... 2024-09-13T18:49:13.617 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-13T18:49:13.817 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T18:49:13.837 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../059-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-13T18:49:13.875 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T18:49:13.954 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T18:49:13.973 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../063-python3-pluggy_0.13.0-2_all.deb ... 2024-09-13T18:49:14.012 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-13T18:49:14.229 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T18:49:14.249 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../060-ceph-mgr-cephadm_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:49:14.303 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mgr-cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:14.349 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-py. 2024-09-13T18:49:14.365 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../064-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-13T18:49:14.432 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T18:49:14.675 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-jwt. 2024-09-13T18:49:14.691 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../061-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-13T18:49:14.733 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T18:49:14.786 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-13T18:49:14.803 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../065-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-13T18:49:14.844 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T18:49:15.078 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-attr. 2024-09-13T18:49:15.095 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../062-python3-attr_19.3.0-2_all.deb ... 2024-09-13T18:49:15.136 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-13T18:49:15.166 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-packaging. 2024-09-13T18:49:15.185 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../066-python3-packaging_20.3-1_all.deb ... 2024-09-13T18:49:15.224 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-13T18:49:15.497 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T18:49:15.517 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../063-python3-pluggy_0.13.0-2_all.deb ... 2024-09-13T18:49:15.544 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T18:49:15.555 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-13T18:49:15.560 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../067-python3-pytest_4.6.9-1_all.deb ... 2024-09-13T18:49:15.602 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-13T18:49:15.965 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T18:49:15.984 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T18:49:16.048 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T18:49:16.050 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-py. 2024-09-13T18:49:16.070 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../064-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-13T18:49:16.108 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T18:49:16.335 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libjs-underscore. 2024-09-13T18:49:16.354 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../069-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-13T18:49:16.393 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T18:49:16.462 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-13T18:49:16.478 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../065-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-13T18:49:16.520 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T18:49:16.704 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-13T18:49:16.722 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../070-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-13T18:49:16.762 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T18:49:16.831 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-packaging. 2024-09-13T18:49:16.848 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../066-python3-packaging_20.3-1_all.deb ... 2024-09-13T18:49:16.889 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-13T18:49:17.108 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-routes. 2024-09-13T18:49:17.124 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../071-python3-routes_2.4.1-2_all.deb ... 2024-09-13T18:49:17.166 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-13T18:49:17.225 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T18:49:17.243 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../067-python3-pytest_4.6.9-1_all.deb ... 2024-09-13T18:49:17.283 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-13T18:49:17.528 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T18:49:17.544 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:49:17.619 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking ceph-mgr-dashboard (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:17.637 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T18:49:17.657 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../068-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T18:49:17.695 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T18:49:17.973 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libjs-underscore. 2024-09-13T18:49:17.992 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../069-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-13T18:49:18.031 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T18:49:18.350 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-13T18:49:18.367 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../070-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-13T18:49:18.408 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T18:49:18.753 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-routes. 2024-09-13T18:49:18.769 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../071-python3-routes_2.4.1-2_all.deb ... 2024-09-13T18:49:18.811 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-13T18:49:19.118 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-decorator. 2024-09-13T18:49:19.136 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../073-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-13T18:49:19.156 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T18:49:19.172 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../072-ceph-mgr-dashboard_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:49:19.197 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T18:49:19.239 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mgr-dashboard (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:19.519 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-13T18:49:19.537 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../074-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-13T18:49:19.579 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T18:49:19.933 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-scipy. 2024-09-13T18:49:19.949 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../075-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-13T18:49:19.991 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-13T18:49:20.664 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-decorator. 2024-09-13T18:49:20.680 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../073-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-13T18:49:20.738 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T18:49:21.083 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-13T18:49:21.100 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../074-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-13T18:49:21.141 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T18:49:21.485 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-scipy. 2024-09-13T18:49:21.505 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../075-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-13T18:49:21.551 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-13T18:49:21.736 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-13T18:49:21.752 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../076-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-13T18:49:21.794 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T18:49:22.398 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T18:49:22.418 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../077-python3-joblib_0.14.0-3_all.deb ... 2024-09-13T18:49:22.448 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-13T18:49:22.863 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T18:49:22.880 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../078-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-13T18:49:22.921 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T18:49:23.236 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-13T18:49:23.255 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../076-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-13T18:49:23.294 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T18:49:23.646 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T18:49:23.664 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../079-ceph-mgr-diskprediction-local_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:49:23.704 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:23.857 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T18:49:23.873 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../077-python3-joblib_0.14.0-3_all.deb ... 2024-09-13T18:49:23.948 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-13T18:49:24.343 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T18:49:24.360 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../078-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-13T18:49:24.405 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T18:49:24.857 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T18:49:24.874 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../080-python3-cachetools_4.0.0-1_all.deb ... 2024-09-13T18:49:24.932 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-13T18:49:25.147 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T18:49:25.164 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../079-ceph-mgr-diskprediction-local_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:49:25.205 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:25.354 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T18:49:25.374 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../081-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-13T18:49:25.412 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T18:49:25.762 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T18:49:25.779 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../082-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-13T18:49:25.820 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T18:49:26.207 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T18:49:26.226 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../083-python3-rsa_4.0-3_all.deb ... 2024-09-13T18:49:26.264 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-13T18:49:26.399 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T18:49:26.416 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../080-python3-cachetools_4.0.0-1_all.deb ... 2024-09-13T18:49:26.457 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-13T18:49:26.610 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T18:49:26.627 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../084-python3-google-auth_1.5.1-2_all.deb ... 2024-09-13T18:49:26.668 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-13T18:49:26.921 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T18:49:26.937 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../081-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-13T18:49:26.979 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T18:49:27.013 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-blinker. 2024-09-13T18:49:27.033 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../085-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-13T18:49:27.071 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T18:49:27.367 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T18:49:27.384 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../082-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-13T18:49:27.417 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-13T18:49:27.425 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T18:49:27.432 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../086-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-13T18:49:27.508 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T18:49:27.754 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T18:49:27.774 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../083-python3-rsa_4.0-3_all.deb ... 2024-09-13T18:49:27.812 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-13T18:49:27.861 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T18:49:27.880 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-13T18:49:27.919 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T18:49:28.160 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T18:49:28.177 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../084-python3-google-auth_1.5.1-2_all.deb ... 2024-09-13T18:49:28.218 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-13T18:49:28.267 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T18:49:28.284 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../088-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-13T18:49:28.326 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T18:49:28.564 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-blinker. 2024-09-13T18:49:28.583 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../085-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-13T18:49:28.647 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T18:49:28.671 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T18:49:28.688 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../089-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-13T18:49:28.729 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T18:49:28.975 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-13T18:49:28.992 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../086-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-13T18:49:29.033 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T18:49:29.311 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T18:49:29.330 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:49:29.369 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:29.405 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T18:49:29.423 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../087-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-13T18:49:29.464 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T18:49:29.747 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T18:49:29.764 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../091-libonig5_6.9.4-1_amd64.deb ... 2024-09-13T18:49:29.805 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-13T18:49:29.808 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T18:49:29.824 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../088-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-13T18:49:29.866 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T18:49:30.183 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T18:49:30.200 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../092-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:49:30.211 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T18:49:30.228 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../089-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-13T18:49:30.241 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T18:49:30.269 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T18:49:30.536 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package jq. 2024-09-13T18:49:30.555 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../093-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:49:30.594 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T18:49:30.858 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T18:49:30.872 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package socat. 2024-09-13T18:49:30.875 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../090-ceph-mgr-k8sevents_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:49:30.890 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../094-socat_1.7.3.3-2_amd64.deb ... 2024-09-13T18:49:30.915 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:30.930 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-13T18:49:31.266 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T18:49:31.268 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T18:49:31.283 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-13T18:49:31.286 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../091-libonig5_6.9.4-1_amd64.deb ... 2024-09-13T18:49:31.326 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-13T18:49:31.349 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-13T18:49:31.713 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package ceph-test. 2024-09-13T18:49:31.713 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T18:49:31.730 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../096-ceph-test_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:31.733 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../092-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:49:31.770 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking ceph-test (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:31.771 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T18:49:32.049 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package jq. 2024-09-13T18:49:32.066 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../093-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:49:32.107 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T18:49:32.418 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package socat. 2024-09-13T18:49:32.438 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../094-socat_1.7.3.3-2_amd64.deb ... 2024-09-13T18:49:32.476 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-13T18:49:32.814 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T18:49:32.831 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../095-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-13T18:49:32.897 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-13T18:49:33.262 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-test. 2024-09-13T18:49:33.279 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../096-ceph-test_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:33.320 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-test (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:36.084 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T18:49:36.100 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:49:36.140 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking ceph-volume (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:36.537 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-13T18:49:36.555 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../098-cephfs-mirror_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:36.628 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking cephfs-mirror (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:36.976 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-colorama. 2024-09-13T18:49:36.994 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../099-python3-colorama_0.4.3-1build1_all.deb ... 2024-09-13T18:49:37.033 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-09-13T18:49:37.312 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package cephfs-shell. 2024-09-13T18:49:37.330 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../100-cephfs-shell_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:49:37.369 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking cephfs-shell (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:37.516 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T18:49:37.532 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../097-ceph-volume_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:49:37.599 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ceph-volume (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:37.664 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package cephfs-top. 2024-09-13T18:49:37.680 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../101-cephfs-top_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:49:37.722 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking cephfs-top (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:38.019 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-13T18:49:38.022 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-13T18:49:38.038 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../102-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T18:49:38.040 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../098-cephfs-mirror_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:38.078 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T18:49:38.080 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking cephfs-mirror (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:38.425 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-colorama. 2024-09-13T18:49:38.445 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../099-python3-colorama_0.4.3-1build1_all.deb ... 2024-09-13T18:49:38.483 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2024-09-13T18:49:38.581 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T18:49:38.599 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../103-libcephfs-dev_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:38.639 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libcephfs-dev (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:38.778 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package cephfs-shell. 2024-09-13T18:49:38.795 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../100-cephfs-shell_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:49:38.835 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking cephfs-shell (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:39.084 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T18:49:39.100 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../104-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-13T18:49:39.107 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package cephfs-top. 2024-09-13T18:49:39.124 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../101-cephfs-top_18.2.4-860-g6c6b083f-1focal_all.deb ... 2024-09-13T18:49:39.165 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking cephfs-top (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:39.187 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T18:49:39.451 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-13T18:49:39.468 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../102-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T18:49:39.509 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T18:49:39.532 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T18:49:39.550 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../105-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-13T18:49:39.590 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-13T18:49:39.993 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-13T18:49:40.011 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../106-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T18:49:40.052 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T18:49:40.054 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T18:49:40.071 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../103-libcephfs-dev_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:40.112 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libcephfs-dev (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:40.532 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T18:49:40.551 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../104-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-13T18:49:40.609 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T18:49:40.978 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T18:49:40.995 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../105-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-13T18:49:41.036 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-13T18:49:41.465 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-13T18:49:41.481 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../106-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T18:49:41.523 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T18:49:41.692 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-13T18:49:41.705 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../107-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-13T18:49:41.771 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-13T18:49:42.175 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-webencodings. 2024-09-13T18:49:42.192 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../108-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-13T18:49:42.233 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T18:49:42.561 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-html5lib. 2024-09-13T18:49:42.582 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../109-python3-html5lib_1.0.1-2_all.deb ... 2024-09-13T18:49:42.619 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-13T18:49:42.981 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-13T18:49:42.997 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../110-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-13T18:49:43.055 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T18:49:43.243 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-13T18:49:43.264 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../107-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-13T18:49:43.301 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-13T18:49:43.534 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T18:49:43.552 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../111-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T18:49:43.591 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T18:49:43.654 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-webencodings. 2024-09-13T18:49:43.672 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../108-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-13T18:49:43.712 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T18:49:43.965 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T18:49:43.986 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../112-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-13T18:49:44.034 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-html5lib. 2024-09-13T18:49:44.036 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T18:49:44.055 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../109-python3-html5lib_1.0.1-2_all.deb ... 2024-09-13T18:49:44.092 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-13T18:49:44.383 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T18:49:44.403 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../113-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-13T18:49:44.441 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T18:49:44.504 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-13T18:49:44.521 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../110-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-13T18:49:44.561 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T18:49:44.777 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T18:49:44.794 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-13T18:49:44.835 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T18:49:45.015 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T18:49:45.032 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../111-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T18:49:45.081 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T18:49:45.147 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T18:49:45.167 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-13T18:49:45.205 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T18:49:45.426 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T18:49:45.444 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../112-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-13T18:49:45.484 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T18:49:45.822 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T18:49:45.842 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../113-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-13T18:49:45.880 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T18:49:45.909 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package radosgw. 2024-09-13T18:49:45.928 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../116-radosgw_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:45.966 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking radosgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:46.225 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T18:49:46.242 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../114-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-13T18:49:46.282 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T18:49:46.594 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T18:49:46.611 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-13T18:49:46.651 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T18:49:47.322 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package radosgw. 2024-09-13T18:49:47.340 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../116-radosgw_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:47.380 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking radosgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:47.521 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T18:49:47.541 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../117-rbd-fuse_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:47.579 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking rbd-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:47.903 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-13T18:49:48.209 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T18:49:48.368 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T18:49:48.494 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T18:49:48.712 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:48.827 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T18:49:48.847 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../117-rbd-fuse_18.2.4-860-g6c6b083f-1focal_amd64.deb ... 2024-09-13T18:49:48.885 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking rbd-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:49.196 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-13T18:49:49.322 INFO:teuthology.orchestra.run.smithi171.stdout:Adding system user cephadm....done 2024-09-13T18:49:49.485 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T18:49:49.548 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T18:49:49.644 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T18:49:49.707 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T18:49:49.834 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T18:49:49.996 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T18:49:50.061 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:50.289 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T18:49:50.415 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-13T18:49:50.566 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T18:49:50.686 INFO:teuthology.orchestra.run.smithi007.stdout:Adding system user cephadm....done 2024-09-13T18:49:50.884 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T18:49:50.884 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-13T18:49:51.010 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T18:49:51.295 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T18:49:51.436 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T18:49:51.604 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-13T18:49:51.729 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T18:49:51.872 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-09-13T18:49:51.880 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-13T18:49:52.006 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T18:49:52.148 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T18:49:52.333 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-13T18:49:52.727 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T18:49:52.799 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T18:49:53.118 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-ceph-argparse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:53.345 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-13T18:49:53.438 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T18:49:53.569 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-09-13T18:49:53.646 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2024-09-13T18:49:53.864 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-13T18:49:53.973 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T18:49:54.148 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T18:49:54.282 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T18:49:54.441 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T18:49:54.567 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T18:49:54.575 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-ceph-argparse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:54.876 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T18:49:54.935 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-13T18:49:55.002 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2024-09-13T18:49:55.245 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T18:49:55.345 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-13T18:49:55.537 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T18:49:55.671 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T18:49:55.863 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T18:49:55.972 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T18:49:56.098 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T18:49:56.140 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T18:49:56.474 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-13T18:49:56.533 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-13T18:49:56.758 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-13T18:49:56.826 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T18:49:57.043 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T18:49:57.136 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T18:49:57.385 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T18:49:57.385 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T18:49:57.528 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T18:49:57.646 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T18:49:57.846 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T18:49:58.031 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-09-13T18:49:58.222 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-13T18:49:58.340 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T18:49:58.556 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-13T18:49:58.708 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T18:49:58.899 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T18:49:59.050 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T18:49:59.060 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-13T18:49:59.185 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T18:49:59.186 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:49:59.478 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T18:49:59.553 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:49:59.705 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-13T18:49:59.896 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2024-09-13T18:50:00.048 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T18:50:00.189 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T18:50:00.207 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-13T18:50:00.333 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-13T18:50:00.565 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T18:50:00.626 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T18:50:00.928 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:50:00.934 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-13T18:50:01.028 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T18:50:01.082 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:01.154 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T18:50:01.444 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:50:01.532 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T18:50:01.587 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-13T18:50:01.833 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-13T18:50:01.955 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T18:50:02.081 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-13T18:50:02.142 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T18:50:02.240 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-13T18:50:02.268 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T18:50:02.534 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T18:50:02.553 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-13T18:50:02.837 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T18:50:02.877 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:50:03.003 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T18:50:03.124 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T18:50:03.128 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T18:50:03.489 INFO:teuthology.orchestra.run.smithi171.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-13T18:50:03.497 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T18:50:03.758 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-13T18:50:03.832 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-13T18:50:03.885 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T18:50:04.158 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T18:50:04.202 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-13T18:50:04.309 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T18:50:04.561 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T18:50:04.611 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-13T18:50:04.704 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T18:50:04.814 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T18:50:04.919 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T18:50:05.140 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T18:50:05.254 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T18:50:05.416 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-13T18:50:05.635 INFO:teuthology.orchestra.run.smithi007.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-13T18:50:05.923 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-13T18:50:06.100 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T18:50:06.434 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-13T18:50:06.810 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T18:50:06.880 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T18:50:06.945 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T18:50:07.031 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-13T18:50:07.096 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T18:50:07.383 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-13T18:50:07.455 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T18:50:07.676 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-13T18:50:07.767 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-13T18:50:07.960 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:50:08.086 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-13T18:50:08.446 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T18:50:08.798 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-13T18:50:09.199 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T18:50:09.287 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T18:50:09.413 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-13T18:50:09.517 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T18:50:09.790 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-13T18:50:10.133 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-13T18:50:10.442 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T18:50:10.547 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T18:50:10.588 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-13T18:50:10.673 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-13T18:50:10.824 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T18:50:10.950 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-13T18:50:10.970 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T18:50:11.252 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T18:50:11.322 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-13T18:50:11.402 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-13T18:50:11.706 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T18:50:11.729 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T18:50:12.041 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T18:50:12.055 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T18:50:12.390 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T18:50:12.750 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up librados2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:12.992 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-13T18:50:13.122 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T18:50:13.187 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up librgw2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:13.248 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-13T18:50:13.339 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libsqlite3-mod-ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:13.374 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T18:50:13.465 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libcephfs2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:13.558 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-13T18:50:13.591 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-13T18:50:13.860 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T18:50:13.917 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libradosstriper1 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:13.985 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-13T18:50:14.077 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-13T18:50:14.295 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T18:50:14.387 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up librbd1 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:14.513 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up ceph-mgr-modules-core (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:14.596 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T18:50:14.655 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up ceph-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:14.878 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T18:50:14.878 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T18:50:14.914 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T18:50:15.275 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up librados2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:15.374 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libcephfs-dev (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:15.486 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-13T18:50:15.500 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-rados (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:15.638 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up librgw2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:15.651 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up cephfs-top (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:15.763 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libsqlite3-mod-ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:15.777 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T18:50:15.915 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libcephfs2 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:16.024 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-13T18:50:16.317 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libradosstriper1 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:16.443 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-13T18:50:16.723 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-rbd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:16.737 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up librbd1 (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:16.863 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mgr-modules-core (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:16.882 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up rbd-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:17.008 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-rgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:17.030 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:17.167 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up python3-cephfs (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:17.266 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T18:50:17.266 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T18:50:17.293 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up cephfs-shell (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:17.419 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:17.699 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libcephfs-dev (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:17.833 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-rados (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:17.982 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up cephfs-top (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:18.005 INFO:teuthology.orchestra.run.smithi171.stdout:Adding group ceph....done 2024-09-13T18:50:18.111 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T18:50:18.477 INFO:teuthology.orchestra.run.smithi171.stdout:Adding system user ceph....done 2024-09-13T18:50:18.649 INFO:teuthology.orchestra.run.smithi171.stdout:Setting system user ceph properties....done 2024-09-13T18:50:18.667 INFO:teuthology.orchestra.run.smithi171.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T18:50:18.821 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T18:50:19.073 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-rbd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:19.143 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T18:50:19.190 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up rbd-fuse (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:19.317 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-rgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:19.476 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up python3-cephfs (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:19.595 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up cephfs-mirror (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:19.602 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up cephfs-shell (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:19.761 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-common (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:19.813 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T18:50:19.813 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T18:50:20.255 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up ceph-test (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:20.331 INFO:teuthology.orchestra.run.smithi007.stdout:Adding group ceph....done 2024-09-13T18:50:20.414 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up radosgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:20.708 INFO:teuthology.orchestra.run.smithi007.stdout:Adding system user ceph....done 2024-09-13T18:50:20.891 INFO:teuthology.orchestra.run.smithi007.stdout:Setting system user ceph properties....done 2024-09-13T18:50:20.896 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T18:50:20.896 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T18:50:20.916 INFO:teuthology.orchestra.run.smithi007.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T18:50:21.065 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T18:50:21.348 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up ceph-base (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:21.377 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T18:50:21.736 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T18:50:21.821 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up cephfs-mirror (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:22.081 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T18:50:22.081 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T18:50:22.193 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up ceph-mds (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:22.463 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T18:50:22.463 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T18:50:22.523 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-test (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:22.682 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up radosgw (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:22.903 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up ceph-mgr (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:23.164 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T18:50:23.165 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T18:50:23.166 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T18:50:23.167 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T18:50:23.616 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-base (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:23.679 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up ceph-osd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:24.053 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T18:50:24.169 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T18:50:24.169 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T18:50:24.476 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mds (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:24.648 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up ceph-mgr-k8sevents (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:24.720 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T18:50:24.721 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T18:50:24.774 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:24.950 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up ceph-mon (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:25.163 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mgr (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:25.175 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T18:50:25.175 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T18:50:25.382 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T18:50:25.382 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T18:50:25.618 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up ceph-mgr-cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:25.761 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:25.815 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-osd (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:25.912 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up ceph-mgr-dashboard (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:26.063 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up ceph-volume (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:26.321 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T18:50:26.321 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T18:50:26.441 INFO:teuthology.orchestra.run.smithi171.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T18:50:26.618 INFO:teuthology.orchestra.run.smithi171.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T18:50:26.758 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mgr-k8sevents (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:26.902 INFO:teuthology.orchestra.run.smithi171.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T18:50:26.909 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:27.052 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mon (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:27.286 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T18:50:27.286 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T18:50:27.745 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mgr-cephadm (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:27.904 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:28.031 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-mgr-dashboard (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:28.173 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ceph-volume (18.2.4-860-g6c6b083f-1focal) ... 2024-09-13T18:50:28.593 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T18:50:28.745 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T18:50:29.021 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T18:50:29.797 INFO:teuthology.orchestra.run.smithi171.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-13T18:50:30.450 INFO:teuthology.orchestra.run.smithi171.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T18:50:30.452 DEBUG:teuthology.orchestra.run.smithi171:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-13T18:50:30.529 INFO:teuthology.orchestra.run.smithi171.stdout:Reading package lists... 2024-09-13T18:50:30.749 INFO:teuthology.orchestra.run.smithi171.stdout:Building dependency tree... 2024-09-13T18:50:30.749 INFO:teuthology.orchestra.run.smithi171.stdout:Reading state information... 2024-09-13T18:50:30.976 INFO:teuthology.orchestra.run.smithi171.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-09-13T18:50:30.976 INFO:teuthology.orchestra.run.smithi171.stdout:flex is already the newest version (2.6.4-6.2). 2024-09-13T18:50:30.976 INFO:teuthology.orchestra.run.smithi171.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-09-13T18:50:30.976 INFO:teuthology.orchestra.run.smithi171.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-09-13T18:50:30.976 INFO:teuthology.orchestra.run.smithi171.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-09-13T18:50:30.976 INFO:teuthology.orchestra.run.smithi171.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T18:50:30.977 INFO:teuthology.orchestra.run.smithi171.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T18:50:30.977 INFO:teuthology.orchestra.run.smithi171.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T18:50:30.977 INFO:teuthology.orchestra.run.smithi171.stdout:The following additional packages will be installed: 2024-09-13T18:50:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-09-13T18:50:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 2024-09-13T18:50:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: libteamdctl0 libuuid1 modemmanager network-manager-pptp postgresql-12 2024-09-13T18:50:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-client-12 postgresql-client-common ppp pptp-linux usb-modeswitch 2024-09-13T18:50:30.978 INFO:teuthology.orchestra.run.smithi171.stdout: usb-modeswitch-data wpasupplicant 2024-09-13T18:50:30.980 INFO:teuthology.orchestra.run.smithi171.stdout:Suggested packages: 2024-09-13T18:50:30.980 INFO:teuthology.orchestra.run.smithi171.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2024-09-13T18:50:30.980 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2024-09-13T18:50:30.980 INFO:teuthology.orchestra.run.smithi171.stdout: libengine-pkcs11-openssl 2024-09-13T18:50:31.046 INFO:teuthology.orchestra.run.smithi171.stdout:The following NEW packages will be installed: 2024-09-13T18:50:31.047 INFO:teuthology.orchestra.run.smithi171.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-09-13T18:50:31.047 INFO:teuthology.orchestra.run.smithi171.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2024-09-13T18:50:31.047 INFO:teuthology.orchestra.run.smithi171.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-09-13T18:50:31.048 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2024-09-13T18:50:31.048 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2024-09-13T18:50:31.048 INFO:teuthology.orchestra.run.smithi171.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-13T18:50:31.049 INFO:teuthology.orchestra.run.smithi171.stdout:The following packages will be upgraded: 2024-09-13T18:50:31.050 INFO:teuthology.orchestra.run.smithi171.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-09-13T18:50:31.106 INFO:teuthology.orchestra.run.smithi171.stdout:6 upgraded, 31 newly installed, 0 to remove and 289 not upgraded. 2024-09-13T18:50:31.106 INFO:teuthology.orchestra.run.smithi171.stdout:Need to get 25.4 MB of archives. 2024-09-13T18:50:31.106 INFO:teuthology.orchestra.run.smithi171.stdout:After this operation, 80.0 MB of additional disk space will be used. 2024-09-13T18:50:31.106 INFO:teuthology.orchestra.run.smithi171.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-09-13T18:50:31.239 INFO:teuthology.orchestra.run.smithi171.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-09-13T18:50:31.240 INFO:teuthology.orchestra.run.smithi171.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-09-13T18:50:31.240 INFO:teuthology.orchestra.run.smithi171.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-09-13T18:50:31.245 INFO:teuthology.orchestra.run.smithi171.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-09-13T18:50:31.246 INFO:teuthology.orchestra.run.smithi171.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.23 [1,323 kB] 2024-09-13T18:50:31.264 INFO:teuthology.orchestra.run.smithi171.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-09-13T18:50:31.264 INFO:teuthology.orchestra.run.smithi171.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-09-13T18:50:31.266 INFO:teuthology.orchestra.run.smithi171.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-09-13T18:50:31.267 INFO:teuthology.orchestra.run.smithi171.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-09-13T18:50:31.267 INFO:teuthology.orchestra.run.smithi171.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.8 [60.6 kB] 2024-09-13T18:50:31.273 INFO:teuthology.orchestra.run.smithi171.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-09-13T18:50:31.274 INFO:teuthology.orchestra.run.smithi171.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-09-13T18:50:31.275 INFO:teuthology.orchestra.run.smithi171.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-09-13T18:50:31.275 INFO:teuthology.orchestra.run.smithi171.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-09-13T18:50:31.290 INFO:teuthology.orchestra.run.smithi171.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libndp0 amd64 1.7-0ubuntu1.1 [11.1 kB] 2024-09-13T18:50:31.290 INFO:teuthology.orchestra.run.smithi171.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-09-13T18:50:31.296 INFO:teuthology.orchestra.run.smithi171.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.20-0ubuntu0.20.04.1 [117 kB] 2024-09-13T18:50:31.298 INFO:teuthology.orchestra.run.smithi171.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-09-13T18:50:31.305 INFO:teuthology.orchestra.run.smithi171.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-09-13T18:50:31.305 INFO:teuthology.orchestra.run.smithi171.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.23 [1,586 kB] 2024-09-13T18:50:31.319 INFO:teuthology.orchestra.run.smithi171.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-09-13T18:50:31.319 INFO:teuthology.orchestra.run.smithi171.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-09-13T18:50:31.326 INFO:teuthology.orchestra.run.smithi171.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.4 [1,184 kB] 2024-09-13T18:50:31.336 INFO:teuthology.orchestra.run.smithi171.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-09-13T18:50:31.352 INFO:teuthology.orchestra.run.smithi171.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-09-13T18:50:31.354 INFO:teuthology.orchestra.run.smithi171.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-09-13T18:50:31.355 INFO:teuthology.orchestra.run.smithi171.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-09-13T18:50:31.355 INFO:teuthology.orchestra.run.smithi171.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2024-09-13T18:50:31.356 INFO:teuthology.orchestra.run.smithi171.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.20-0ubuntu0.20.04.1 [1,055 kB] 2024-09-13T18:50:31.364 INFO:teuthology.orchestra.run.smithi171.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2024-09-13T18:50:31.366 INFO:teuthology.orchestra.run.smithi171.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.20-0ubuntu0.20.04.1 [13.5 MB] 2024-09-13T18:50:31.475 INFO:teuthology.orchestra.run.smithi171.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2024-09-13T18:50:31.475 INFO:teuthology.orchestra.run.smithi171.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2024-09-13T18:50:31.548 INFO:teuthology.orchestra.run.smithi171.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2024-09-13T18:50:31.548 INFO:teuthology.orchestra.run.smithi171.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-09-13T18:50:31.549 INFO:teuthology.orchestra.run.smithi171.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-09-13T18:50:31.957 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-13T18:50:31.962 INFO:teuthology.orchestra.run.smithi171.stdout:Preconfiguring packages ... 2024-09-13T18:50:32.148 INFO:teuthology.orchestra.run.smithi171.stdout:Fetched 25.4 MB in 0s (51.7 MB/s) 2024-09-13T18:50:32.456 INFO:teuthology.orchestra.run.smithi171.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153099 files and directories currently installed.) 2024-09-13T18:50:32.461 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T18:50:32.548 INFO:teuthology.orchestra.run.smithi007.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T18:50:32.550 DEBUG:teuthology.orchestra.run.smithi007:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-13T18:50:32.628 INFO:teuthology.orchestra.run.smithi007.stdout:Reading package lists... 2024-09-13T18:50:32.649 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T18:50:32.853 INFO:teuthology.orchestra.run.smithi007.stdout:Building dependency tree... 2024-09-13T18:50:32.853 INFO:teuthology.orchestra.run.smithi007.stdout:Reading state information... 2024-09-13T18:50:33.081 INFO:teuthology.orchestra.run.smithi007.stdout:bison is already the newest version (2:3.5.1+dfsg-1). 2024-09-13T18:50:33.081 INFO:teuthology.orchestra.run.smithi007.stdout:flex is already the newest version (2.6.4-6.2). 2024-09-13T18:50:33.081 INFO:teuthology.orchestra.run.smithi007.stdout:iproute2 is already the newest version (5.5.0-1ubuntu1). 2024-09-13T18:50:33.081 INFO:teuthology.orchestra.run.smithi007.stdout:libtool-bin is already the newest version (2.4.6-14). 2024-09-13T18:50:33.081 INFO:teuthology.orchestra.run.smithi007.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2024-09-13T18:50:33.081 INFO:teuthology.orchestra.run.smithi007.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T18:50:33.082 INFO:teuthology.orchestra.run.smithi007.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T18:50:33.082 INFO:teuthology.orchestra.run.smithi007.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T18:50:33.082 INFO:teuthology.orchestra.run.smithi007.stdout:The following additional packages will be installed: 2024-09-13T18:50:33.082 INFO:teuthology.orchestra.run.smithi007.stdout: libbluetooth3 libdw1 libelf1 liblzo2-2 libmbim-glib4 libmbim-proxy 2024-09-13T18:50:33.083 INFO:teuthology.orchestra.run.smithi007.stdout: libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl1.1 2024-09-13T18:50:33.083 INFO:teuthology.orchestra.run.smithi007.stdout: libteamdctl0 libuuid1 modemmanager network-manager-pptp postgresql-12 2024-09-13T18:50:33.083 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-client-12 postgresql-client-common ppp pptp-linux usb-modeswitch 2024-09-13T18:50:33.084 INFO:teuthology.orchestra.run.smithi007.stdout: usb-modeswitch-data wpasupplicant 2024-09-13T18:50:33.084 INFO:teuthology.orchestra.run.smithi007.stdout:Suggested packages: 2024-09-13T18:50:33.084 INFO:teuthology.orchestra.run.smithi007.stdout: indent-doc libssl-doc avahi-autoipd libteam-utils postgresql-doc 2024-09-13T18:50:33.084 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-doc-12 libjson-perl comgt wvdial util-linux-locales wpagui 2024-09-13T18:50:33.084 INFO:teuthology.orchestra.run.smithi007.stdout: libengine-pkcs11-openssl 2024-09-13T18:50:33.149 INFO:teuthology.orchestra.run.smithi007.stdout:The following NEW packages will be installed: 2024-09-13T18:50:33.149 INFO:teuthology.orchestra.run.smithi007.stdout: dump indent libaio-dev libbluetooth3 libelf-dev liblzo2-2 libmbim-glib4 2024-09-13T18:50:33.149 INFO:teuthology.orchestra.run.smithi007.stdout: libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy 2024-09-13T18:50:33.150 INFO:teuthology.orchestra.run.smithi007.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2024-09-13T18:50:33.150 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql postgresql-12 postgresql-client postgresql-client-12 2024-09-13T18:50:33.150 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-client-common postgresql-common postgresql-contrib ppp pptp-linux 2024-09-13T18:50:33.150 INFO:teuthology.orchestra.run.smithi007.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-13T18:50:33.151 INFO:teuthology.orchestra.run.smithi007.stdout:The following packages will be upgraded: 2024-09-13T18:50:33.152 INFO:teuthology.orchestra.run.smithi007.stdout: libdw1 libelf1 libssl1.1 libuuid1 util-linux uuid-dev 2024-09-13T18:50:33.336 INFO:teuthology.orchestra.run.smithi007.stdout:6 upgraded, 31 newly installed, 0 to remove and 289 not upgraded. 2024-09-13T18:50:33.336 INFO:teuthology.orchestra.run.smithi007.stdout:Need to get 25.4 MB of archives. 2024-09-13T18:50:33.336 INFO:teuthology.orchestra.run.smithi007.stdout:After this operation, 80.0 MB of additional disk space will be used. 2024-09-13T18:50:33.336 INFO:teuthology.orchestra.run.smithi007.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 util-linux amd64 2.34-0.1ubuntu9.6 [1,022 kB] 2024-09-13T18:50:33.951 INFO:teuthology.orchestra.run.smithi007.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 uuid-dev amd64 2.34-0.1ubuntu9.6 [33.6 kB] 2024-09-13T18:50:33.952 INFO:teuthology.orchestra.run.smithi007.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libuuid1 amd64 2.34-0.1ubuntu9.6 [20.0 kB] 2024-09-13T18:50:33.954 INFO:teuthology.orchestra.run.smithi007.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libdw1 amd64 0.176-1.1ubuntu0.1 [226 kB] 2024-09-13T18:50:33.958 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-09-13T18:50:33.966 INFO:teuthology.orchestra.run.smithi007.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf1 amd64 0.176-1.1ubuntu0.1 [44.2 kB] 2024-09-13T18:50:33.968 INFO:teuthology.orchestra.run.smithi007.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.23 [1,323 kB] 2024-09-13T18:50:34.066 INFO:teuthology.orchestra.run.smithi007.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2024-09-13T18:50:34.068 INFO:teuthology.orchestra.run.smithi007.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2024-09-13T18:50:34.075 INFO:teuthology.orchestra.run.smithi007.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 indent amd64 2.2.12-1ubuntu0.20.04.1 [86.6 kB] 2024-09-13T18:50:34.080 INFO:teuthology.orchestra.run.smithi007.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2024-09-13T18:50:34.080 INFO:teuthology.orchestra.run.smithi007.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libbluetooth3 amd64 5.53-0ubuntu3.8 [60.6 kB] 2024-09-13T18:50:34.113 INFO:teuthology.orchestra.run.smithi007.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libelf-dev amd64 0.176-1.1ubuntu0.1 [57.1 kB] 2024-09-13T18:50:34.115 INFO:teuthology.orchestra.run.smithi007.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-glib4 amd64 1.26.2-1~ubuntu20.04.1 [123 kB] 2024-09-13T18:50:34.118 INFO:teuthology.orchestra.run.smithi007.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmbim-proxy amd64 1.26.2-1~ubuntu20.04.1 [6,012 B] 2024-09-13T18:50:34.119 INFO:teuthology.orchestra.run.smithi007.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libmm-glib0 amd64 1.18.6-1~ubuntu20.04.1 [203 kB] 2024-09-13T18:50:34.194 INFO:teuthology.orchestra.run.smithi007.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libndp0 amd64 1.7-0ubuntu1.1 [11.1 kB] 2024-09-13T18:50:34.195 INFO:teuthology.orchestra.run.smithi007.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libnm0 amd64 1.22.10-1ubuntu2.4 [370 kB] 2024-09-13T18:50:34.205 INFO:teuthology.orchestra.run.smithi007.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpq5 amd64 12.20-0ubuntu0.20.04.1 [117 kB] 2024-09-13T18:50:34.208 INFO:teuthology.orchestra.run.smithi007.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-glib5 amd64 1.30.4-1~ubuntu20.04.1 [642 kB] 2024-09-13T18:50:34.223 INFO:teuthology.orchestra.run.smithi007.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libqmi-proxy amd64 1.30.4-1~ubuntu20.04.1 [5,908 B] 2024-09-13T18:50:34.223 INFO:teuthology.orchestra.run.smithi007.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.23 [1,586 kB] 2024-09-13T18:50:34.276 INFO:teuthology.orchestra.run.smithi007.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 libteamdctl0 amd64 1.30-1 [11.8 kB] 2024-09-13T18:50:34.277 INFO:teuthology.orchestra.run.smithi007.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 modemmanager amd64 1.18.6-1~ubuntu20.04.1 [895 kB] 2024-09-13T18:50:34.298 INFO:teuthology.orchestra.run.smithi007.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 wpasupplicant amd64 2:2.9-1ubuntu4.4 [1,184 kB] 2024-09-13T18:50:34.327 INFO:teuthology.orchestra.run.smithi007.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 network-manager amd64 1.22.10-1ubuntu2.4 [1,854 kB] 2024-09-13T18:50:34.385 INFO:teuthology.orchestra.run.smithi007.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 ppp amd64 2.4.7-2+4.1ubuntu5.1 [328 kB] 2024-09-13T18:50:34.395 INFO:teuthology.orchestra.run.smithi007.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 pptp-linux amd64 1.10.0-1build1 [42.0 kB] 2024-09-13T18:50:34.396 INFO:teuthology.orchestra.run.smithi007.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 network-manager-pptp amd64 1.2.8-2 [30.1 kB] 2024-09-13T18:50:34.396 INFO:teuthology.orchestra.run.smithi007.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-common all 214ubuntu0.1 [28.2 kB] 2024-09-13T18:50:34.397 INFO:teuthology.orchestra.run.smithi007.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client-12 amd64 12.20-0ubuntu0.20.04.1 [1,055 kB] 2024-09-13T18:50:34.422 INFO:teuthology.orchestra.run.smithi007.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-common all 214ubuntu0.1 [169 kB] 2024-09-13T18:50:34.440 INFO:teuthology.orchestra.run.smithi007.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-12 amd64 12.20-0ubuntu0.20.04.1 [13.5 MB] 2024-09-13T18:50:34.875 INFO:teuthology.orchestra.run.smithi007.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql all 12+214ubuntu0.1 [3,924 B] 2024-09-13T18:50:34.875 INFO:teuthology.orchestra.run.smithi007.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-client all 12+214ubuntu0.1 [3,940 B] 2024-09-13T18:50:34.900 INFO:teuthology.orchestra.run.smithi007.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 postgresql-contrib all 12+214ubuntu0.1 [3,932 B] 2024-09-13T18:50:34.900 INFO:teuthology.orchestra.run.smithi007.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch-data all 20191128-3 [32.3 kB] 2024-09-13T18:50:34.901 INFO:teuthology.orchestra.run.smithi007.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu3 [53.1 kB] 2024-09-13T18:50:35.329 INFO:teuthology.orchestra.run.smithi007.stdout:Preconfiguring packages ... 2024-09-13T18:50:35.418 INFO:teuthology.orchestra.run.smithi171.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153099 files and directories currently installed.) 2024-09-13T18:50:35.423 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T18:50:35.539 INFO:teuthology.orchestra.run.smithi007.stdout:Fetched 25.4 MB in 2s (14.6 MB/s) 2024-09-13T18:50:35.581 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T18:50:35.850 INFO:teuthology.orchestra.run.smithi007.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153099 files and directories currently installed.) 2024-09-13T18:50:35.855 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../util-linux_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T18:50:36.029 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T18:50:36.044 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking util-linux (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T18:50:36.228 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T18:50:36.580 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-09-13T18:50:37.041 INFO:teuthology.orchestra.run.smithi171.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153099 files and directories currently installed.) 2024-09-13T18:50:37.046 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T18:50:37.203 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-09-13T18:50:37.207 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up util-linux (2.34-0.1ubuntu9.6) ... 2024-09-13T18:50:37.684 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T18:50:37.850 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-09-13T18:50:38.349 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-09-13T18:50:38.496 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.23) over (1.1.1f-1ubuntu2.16) ... 2024-09-13T18:50:38.679 INFO:teuthology.orchestra.run.smithi007.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153099 files and directories currently installed.) 2024-09-13T18:50:38.684 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../uuid-dev_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T18:50:38.834 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking uuid-dev:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T18:50:39.041 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-13T18:50:39.057 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-09-13T18:50:39.273 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../libuuid1_2.34-0.1ubuntu9.6_amd64.deb ... 2024-09-13T18:50:39.316 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-09-13T18:50:39.632 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libuuid1:amd64 (2.34-0.1ubuntu9.6) over (2.34-0.1ubuntu9.3) ... 2024-09-13T18:50:40.230 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package dump. 2024-09-13T18:50:40.248 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-09-13T18:50:40.280 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking dump (0.4b46-6) ... 2024-09-13T18:50:40.617 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package indent. 2024-09-13T18:50:40.636 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:50:40.675 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-09-13T18:50:40.919 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libuuid1:amd64 (2.34-0.1ubuntu9.6) ... 2024-09-13T18:50:41.012 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-13T18:50:41.030 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-09-13T18:50:41.095 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-09-13T18:50:41.391 INFO:teuthology.orchestra.run.smithi007.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 153099 files and directories currently installed.) 2024-09-13T18:50:41.396 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../00-libdw1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T18:50:41.457 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-13T18:50:41.476 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.8_amd64.deb ... 2024-09-13T18:50:41.515 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-09-13T18:50:41.567 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libdw1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-09-13T18:50:41.801 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-13T18:50:41.821 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T18:50:41.859 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T18:50:42.039 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../01-libelf1_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T18:50:42.206 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libelf1:amd64 (0.176-1.1ubuntu0.1) over (0.176-1.1build1) ... 2024-09-13T18:50:42.229 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-13T18:50:42.246 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:50:42.287 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T18:50:42.590 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-13T18:50:42.608 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:50:42.673 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T18:50:42.729 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../02-libssl1.1_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-09-13T18:50:42.894 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.23) over (1.1.1f-1ubuntu2.16) ... 2024-09-13T18:50:43.043 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-13T18:50:43.060 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:50:43.101 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T18:50:43.415 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-13T18:50:43.435 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2024-09-13T18:50:43.473 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2024-09-13T18:50:43.487 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-13T18:50:43.504 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1.1_amd64.deb ... 2024-09-13T18:50:43.545 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-09-13T18:50:43.835 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package dump. 2024-09-13T18:50:43.855 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2024-09-13T18:50:43.893 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking dump (0.4b46-6) ... 2024-09-13T18:50:43.898 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-13T18:50:43.916 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-09-13T18:50:43.973 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-09-13T18:50:44.247 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package indent. 2024-09-13T18:50:44.267 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../05-indent_2.2.12-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:50:44.322 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking indent (2.2.12-1ubuntu0.20.04.1) ... 2024-09-13T18:50:44.385 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-13T18:50:44.404 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../14-libpq5_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:50:44.443 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libpq5:amd64 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:50:44.658 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-13T18:50:44.676 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2024-09-13T18:50:44.716 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2024-09-13T18:50:44.837 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-13T18:50:44.854 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../15-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:50:44.920 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T18:50:45.087 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-13T18:50:45.104 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../07-libbluetooth3_5.53-0ubuntu3.8_amd64.deb ... 2024-09-13T18:50:45.145 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-09-13T18:50:45.448 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-13T18:50:45.465 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../08-libelf-dev_0.176-1.1ubuntu0.1_amd64.deb ... 2024-09-13T18:50:45.483 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-13T18:50:45.503 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../16-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:50:45.506 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T18:50:45.561 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T18:50:45.810 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-13T18:50:45.828 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../17-libssl-dev_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-09-13T18:50:45.889 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T18:50:46.253 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-13T18:50:46.274 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../09-libmbim-glib4_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:50:46.406 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-13T18:50:46.422 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../18-libteamdctl0_1.30-1_amd64.deb ... 2024-09-13T18:50:46.464 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-09-13T18:50:46.562 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T18:50:46.884 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package modemmanager. 2024-09-13T18:50:46.900 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../19-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:50:46.916 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-13T18:50:46.933 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../10-libmbim-proxy_1.26.2-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:50:46.952 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T18:50:46.974 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T18:50:47.335 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-13T18:50:47.353 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../11-libmm-glib0_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:50:47.394 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T18:50:47.640 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package wpasupplicant. 2024-09-13T18:50:47.657 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../20-wpasupplicant_2%3a2.9-1ubuntu4.4_amd64.deb ... 2024-09-13T18:50:47.698 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-09-13T18:50:47.788 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-13T18:50:47.808 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../12-libndp0_1.7-0ubuntu1.1_amd64.deb ... 2024-09-13T18:50:47.846 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-09-13T18:50:48.208 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-13T18:50:48.225 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../13-libnm0_1.22.10-1ubuntu2.4_amd64.deb ... 2024-09-13T18:50:48.261 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package network-manager. 2024-09-13T18:50:48.280 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../21-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-09-13T18:50:48.291 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-09-13T18:50:48.343 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-09-13T18:50:48.694 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-13T18:50:48.715 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../14-libpq5_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:50:48.752 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libpq5:amd64 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:50:49.120 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package ppp. 2024-09-13T18:50:49.131 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-13T18:50:49.137 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../22-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-09-13T18:50:49.149 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../15-libqmi-glib5_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:50:49.189 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T18:50:49.407 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-09-13T18:50:49.550 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-13T18:50:49.567 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../16-libqmi-proxy_1.30.4-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:50:49.608 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T18:50:49.776 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package pptp-linux. 2024-09-13T18:50:49.793 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../23-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-09-13T18:50:49.834 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-09-13T18:50:49.911 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-13T18:50:49.929 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../17-libssl-dev_1.1.1f-1ubuntu2.23_amd64.deb ... 2024-09-13T18:50:49.969 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T18:50:50.229 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-13T18:50:50.246 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../24-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-09-13T18:50:50.304 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-09-13T18:50:50.498 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-13T18:50:50.515 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../18-libteamdctl0_1.30-1_amd64.deb ... 2024-09-13T18:50:50.556 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking libteamdctl0:amd64 (1.30-1) ... 2024-09-13T18:50:50.867 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-13T18:50:50.885 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../25-postgresql-client-common_214ubuntu0.1_all.deb ... 2024-09-13T18:50:50.925 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2024-09-13T18:50:50.943 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package modemmanager. 2024-09-13T18:50:50.961 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../19-modemmanager_1.18.6-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T18:50:51.012 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T18:50:51.246 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package postgresql-client-12. 2024-09-13T18:50:51.267 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../26-postgresql-client-12_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:50:51.304 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking postgresql-client-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:50:51.786 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package wpasupplicant. 2024-09-13T18:50:51.807 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../20-wpasupplicant_2%3a2.9-1ubuntu4.4_amd64.deb ... 2024-09-13T18:50:51.984 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package postgresql-common. 2024-09-13T18:50:51.999 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-09-13T18:50:52.001 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../27-postgresql-common_214ubuntu0.1_all.deb ... 2024-09-13T18:50:52.091 INFO:teuthology.orchestra.run.smithi171.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-13T18:50:52.119 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2024-09-13T18:50:52.598 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package network-manager. 2024-09-13T18:50:52.617 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../21-network-manager_1.22.10-1ubuntu2.4_amd64.deb ... 2024-09-13T18:50:52.624 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package postgresql-12. 2024-09-13T18:50:52.641 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../28-postgresql-12_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:50:52.681 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking network-manager (1.22.10-1ubuntu2.4) ... 2024-09-13T18:50:52.714 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking postgresql-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:50:53.456 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package ppp. 2024-09-13T18:50:53.473 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../22-ppp_2.4.7-2+4.1ubuntu5.1_amd64.deb ... 2024-09-13T18:50:53.733 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-09-13T18:50:54.103 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package pptp-linux. 2024-09-13T18:50:54.123 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../23-pptp-linux_1.10.0-1build1_amd64.deb ... 2024-09-13T18:50:54.161 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking pptp-linux (1.10.0-1build1) ... 2024-09-13T18:50:54.410 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package postgresql. 2024-09-13T18:50:54.427 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../29-postgresql_12+214ubuntu0.1_all.deb ... 2024-09-13T18:50:54.468 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2024-09-13T18:50:54.498 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-13T18:50:54.518 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../24-network-manager-pptp_1.2.8-2_amd64.deb ... 2024-09-13T18:50:54.565 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking network-manager-pptp (1.2.8-2) ... 2024-09-13T18:50:54.746 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package postgresql-client. 2024-09-13T18:50:54.764 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../30-postgresql-client_12+214ubuntu0.1_all.deb ... 2024-09-13T18:50:54.795 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2024-09-13T18:50:54.893 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-13T18:50:54.911 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../25-postgresql-client-common_214ubuntu0.1_all.deb ... 2024-09-13T18:50:54.968 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking postgresql-client-common (214ubuntu0.1) ... 2024-09-13T18:50:55.056 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-13T18:50:55.074 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../31-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2024-09-13T18:50:55.114 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2024-09-13T18:50:55.305 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package postgresql-client-12. 2024-09-13T18:50:55.322 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../26-postgresql-client-12_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:50:55.383 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-13T18:50:55.384 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking postgresql-client-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:50:55.401 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../32-usb-modeswitch-data_20191128-3_all.deb ... 2024-09-13T18:50:55.441 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-09-13T18:50:56.019 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package postgresql-common. 2024-09-13T18:50:56.037 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../27-postgresql-common_214ubuntu0.1_all.deb ... 2024-09-13T18:50:56.135 INFO:teuthology.orchestra.run.smithi007.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-13T18:50:56.162 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking postgresql-common (214ubuntu0.1) ... 2024-09-13T18:50:56.361 INFO:teuthology.orchestra.run.smithi171.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-13T18:50:56.380 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing to unpack .../33-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-09-13T18:50:56.430 INFO:teuthology.orchestra.run.smithi171.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-09-13T18:50:56.642 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package postgresql-12. 2024-09-13T18:50:56.660 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../28-postgresql-12_12.20-0ubuntu0.20.04.1_amd64.deb ... 2024-09-13T18:50:56.704 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking postgresql-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:50:56.753 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2024-09-13T18:50:56.992 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T18:50:57.335 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libpq5:amd64 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:50:57.461 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-09-13T18:50:57.604 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-09-13T18:50:57.705 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-09-13T18:50:58.110 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T18:50:58.110 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T18:50:58.591 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up postgresql-client-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:50:58.679 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package postgresql. 2024-09-13T18:50:58.698 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../29-postgresql_12+214ubuntu0.1_all.deb ... 2024-09-13T18:50:58.737 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking postgresql (12+214ubuntu0.1) ... 2024-09-13T18:50:59.006 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package postgresql-client. 2024-09-13T18:50:59.027 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../30-postgresql-client_12+214ubuntu0.1_all.deb ... 2024-09-13T18:50:59.065 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking postgresql-client (12+214ubuntu0.1) ... 2024-09-13T18:50:59.308 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-13T18:50:59.328 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../31-postgresql-contrib_12+214ubuntu0.1_all.deb ... 2024-09-13T18:50:59.366 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking postgresql-contrib (12+214ubuntu0.1) ... 2024-09-13T18:50:59.482 INFO:teuthology.orchestra.run.smithi171.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-13T18:50:59.572 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-09-13T18:50:59.647 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-13T18:50:59.667 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../32-usb-modeswitch-data_20191128-3_all.deb ... 2024-09-13T18:50:59.681 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-09-13T18:50:59.710 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking usb-modeswitch-data (20191128-3) ... 2024-09-13T18:50:59.841 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2024-09-13T18:51:00.114 INFO:teuthology.orchestra.run.smithi007.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-13T18:51:00.132 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing to unpack .../33-usb-modeswitch_2.5.2+repack0-2ubuntu3_amd64.deb ... 2024-09-13T18:51:00.183 INFO:teuthology.orchestra.run.smithi007.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-09-13T18:51:00.505 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up postgresql-client-common (214ubuntu0.1) ... 2024-09-13T18:51:00.712 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T18:51:00.891 INFO:teuthology.orchestra.run.smithi171.stdout:Adding user postgres to group ssl-cert 2024-09-13T18:51:01.005 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libpq5:amd64 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:51:01.143 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:01.143 INFO:teuthology.orchestra.run.smithi171.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-13T18:51:01.156 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2024-09-13T18:51:01.282 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2024-09-13T18:51:01.441 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up wpasupplicant (2:2.9-1ubuntu4.4) ... 2024-09-13T18:51:01.672 INFO:teuthology.orchestra.run.smithi171.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-13T18:51:01.672 INFO:teuthology.orchestra.run.smithi171.stdout:Removing obsolete dictionary files: 2024-09-13T18:51:01.843 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T18:51:01.844 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T18:51:02.007 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-13T18:51:02.311 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up postgresql-client-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:51:02.793 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T18:51:02.919 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-09-13T18:51:03.044 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-09-13T18:51:03.179 INFO:teuthology.orchestra.run.smithi007.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-13T18:51:03.204 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-09-13T18:51:03.267 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up uuid-dev:amd64 (2.34-0.1ubuntu9.6) ... 2024-09-13T18:51:03.393 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libteamdctl0:amd64 (1.30-1) ... 2024-09-13T18:51:03.561 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up postgresql-common (214ubuntu0.1) ... 2024-09-13T18:51:04.209 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-09-13T18:51:04.451 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up dump (0.4b46-6) ... 2024-09-13T18:51:04.548 INFO:teuthology.orchestra.run.smithi171.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-13T18:51:04.609 INFO:teuthology.orchestra.run.smithi007.stdout:Adding user postgres to group ssl-cert 2024-09-13T18:51:04.611 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T18:51:04.762 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-09-13T18:51:04.863 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:04.863 INFO:teuthology.orchestra.run.smithi007.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-13T18:51:04.888 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-09-13T18:51:05.039 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-09-13T18:51:05.112 INFO:teuthology.orchestra.run.smithi007.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-13T18:51:05.112 INFO:teuthology.orchestra.run.smithi007.stdout:Removing obsolete dictionary files: 2024-09-13T18:51:05.216 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-09-13T18:51:05.375 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up postgresql-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:51:05.457 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-13T18:51:05.728 INFO:teuthology.orchestra.run.smithi171.stdout:Creating new PostgreSQL cluster 12/main ... 2024-09-13T18:51:05.740 INFO:teuthology.orchestra.run.smithi171.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2024-09-13T18:51:05.802 INFO:teuthology.orchestra.run.smithi171.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-13T18:51:05.802 INFO:teuthology.orchestra.run.smithi171.stdout:This user must also own the server process. 2024-09-13T18:51:05.802 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:05.802 INFO:teuthology.orchestra.run.smithi171.stdout:The database cluster will be initialized with locale "en_US". 2024-09-13T18:51:05.802 INFO:teuthology.orchestra.run.smithi171.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-13T18:51:05.802 INFO:teuthology.orchestra.run.smithi171.stdout:The default text search configuration will be set to "english". 2024-09-13T18:51:05.802 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:05.803 INFO:teuthology.orchestra.run.smithi171.stdout:Data page checksums are disabled. 2024-09-13T18:51:05.803 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:05.803 INFO:teuthology.orchestra.run.smithi171.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2024-09-13T18:51:05.803 INFO:teuthology.orchestra.run.smithi171.stdout:creating subdirectories ... ok 2024-09-13T18:51:05.803 INFO:teuthology.orchestra.run.smithi171.stdout:selecting dynamic shared memory implementation ... posix 2024-09-13T18:51:05.824 INFO:teuthology.orchestra.run.smithi171.stdout:selecting default max_connections ... 100 2024-09-13T18:51:05.847 INFO:teuthology.orchestra.run.smithi171.stdout:selecting default shared_buffers ... 128MB 2024-09-13T18:51:05.854 INFO:teuthology.orchestra.run.smithi171.stdout:selecting default time zone ... Etc/UTC 2024-09-13T18:51:05.854 INFO:teuthology.orchestra.run.smithi171.stdout:creating configuration files ... ok 2024-09-13T18:51:05.960 INFO:teuthology.orchestra.run.smithi171.stdout:running bootstrap script ... ok 2024-09-13T18:51:06.222 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.23) ... 2024-09-13T18:51:06.364 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up usb-modeswitch-data (20191128-3) ... 2024-09-13T18:51:06.474 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libnm0:amd64 (1.22.10-1ubuntu2.4) ... 2024-09-13T18:51:06.545 INFO:teuthology.orchestra.run.smithi171.stdout:performing post-bootstrap initialization ... ok 2024-09-13T18:51:06.600 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up ppp (2.4.7-2+4.1ubuntu5.1) ... 2024-09-13T18:51:07.409 INFO:teuthology.orchestra.run.smithi171.stdout:syncing data to disk ... ok 2024-09-13T18:51:07.409 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:07.410 INFO:teuthology.orchestra.run.smithi171.stdout:Success. You can now start the database server using: 2024-09-13T18:51:07.410 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:07.410 INFO:teuthology.orchestra.run.smithi171.stdout: pg_ctlcluster 12 main start 2024-09-13T18:51:07.410 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:07.636 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2024-09-13T18:51:07.727 INFO:teuthology.orchestra.run.smithi171.stdout:Ver Cluster Port Status Owner Data directory Log file 2024-09-13T18:51:07.727 INFO:teuthology.orchestra.run.smithi171.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2024-09-13T18:51:07.808 INFO:teuthology.orchestra.run.smithi171.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-13T18:51:07.873 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up dump (0.4b46-6) ... 2024-09-13T18:51:07.970 INFO:teuthology.orchestra.run.smithi007.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-13T18:51:08.033 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libmm-glib0:amd64 (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T18:51:08.192 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libbluetooth3:amd64 (5.53-0ubuntu3.8) ... 2024-09-13T18:51:08.318 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up indent (2.2.12-1ubuntu0.20.04.1) ... 2024-09-13T18:51:08.478 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu3) ... 2024-09-13T18:51:08.654 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libndp0:amd64 (1.7-0ubuntu1.1) ... 2024-09-13T18:51:08.814 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up postgresql-12 (12.20-0ubuntu0.20.04.1) ... 2024-09-13T18:51:09.203 INFO:teuthology.orchestra.run.smithi007.stdout:Creating new PostgreSQL cluster 12/main ... 2024-09-13T18:51:09.223 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5 2024-09-13T18:51:09.277 INFO:teuthology.orchestra.run.smithi007.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-13T18:51:09.277 INFO:teuthology.orchestra.run.smithi007.stdout:This user must also own the server process. 2024-09-13T18:51:09.277 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:09.277 INFO:teuthology.orchestra.run.smithi007.stdout:The database cluster will be initialized with locale "en_US". 2024-09-13T18:51:09.277 INFO:teuthology.orchestra.run.smithi007.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-13T18:51:09.277 INFO:teuthology.orchestra.run.smithi007.stdout:The default text search configuration will be set to "english". 2024-09-13T18:51:09.277 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:09.278 INFO:teuthology.orchestra.run.smithi007.stdout:Data page checksums are disabled. 2024-09-13T18:51:09.278 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:09.278 INFO:teuthology.orchestra.run.smithi007.stdout:fixing permissions on existing directory /var/lib/postgresql/12/main ... ok 2024-09-13T18:51:09.278 INFO:teuthology.orchestra.run.smithi007.stdout:creating subdirectories ... ok 2024-09-13T18:51:09.278 INFO:teuthology.orchestra.run.smithi007.stdout:selecting dynamic shared memory implementation ... posix 2024-09-13T18:51:09.295 INFO:teuthology.orchestra.run.smithi007.stdout:selecting default max_connections ... 100 2024-09-13T18:51:09.321 INFO:teuthology.orchestra.run.smithi007.stdout:selecting default shared_buffers ... 128MB 2024-09-13T18:51:09.338 INFO:teuthology.orchestra.run.smithi007.stdout:selecting default time zone ... Etc/UTC 2024-09-13T18:51:09.338 INFO:teuthology.orchestra.run.smithi007.stdout:creating configuration files ... ok 2024-09-13T18:51:09.451 INFO:teuthology.orchestra.run.smithi007.stdout:running bootstrap script ... ok 2024-09-13T18:51:09.983 INFO:teuthology.orchestra.run.smithi007.stdout:performing post-bootstrap initialization ... ok 2024-09-13T18:51:10.696 INFO:teuthology.orchestra.run.smithi007.stdout:syncing data to disk ... ok 2024-09-13T18:51:10.696 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:10.696 INFO:teuthology.orchestra.run.smithi007.stdout:Success. You can now start the database server using: 2024-09-13T18:51:10.696 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:10.696 INFO:teuthology.orchestra.run.smithi007.stdout: pg_ctlcluster 12 main start 2024-09-13T18:51:10.696 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:10.907 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T18:51:11.004 INFO:teuthology.orchestra.run.smithi007.stdout:Ver Cluster Port Status Owner Data directory Log file 2024-09-13T18:51:11.004 INFO:teuthology.orchestra.run.smithi007.stdout:12 main 5432 down postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log 2024-09-13T18:51:11.032 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T18:51:11.090 INFO:teuthology.orchestra.run.smithi007.stdout:update-alternatives: using /usr/share/postgresql/12/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-13T18:51:11.200 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-09-13T18:51:11.387 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T18:51:11.519 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2024-09-13T18:51:11.645 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-09-13T18:51:12.377 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:12.378 INFO:teuthology.orchestra.run.smithi171.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-13T18:51:12.378 INFO:teuthology.orchestra.run.smithi171.stdout:which means they are currently configured by ifupdown: 2024-09-13T18:51:12.378 INFO:teuthology.orchestra.run.smithi171.stdout:- enp3s0f1 2024-09-13T18:51:12.378 INFO:teuthology.orchestra.run.smithi171.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-13T18:51:12.378 INFO:teuthology.orchestra.run.smithi171.stdout:remove their configuration from /etc/network/interfaces. 2024-09-13T18:51:12.378 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:12.545 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-13T18:51:12.872 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-13T18:51:13.407 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-13T18:51:13.898 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T18:51:14.066 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-09-13T18:51:14.129 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libelf1:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T18:51:14.209 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T18:51:14.255 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libmbim-glib4:amd64 (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T18:51:14.368 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2024-09-13T18:51:14.381 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up pptp-linux (1.10.0-1build1) ... 2024-09-13T18:51:14.494 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2024-09-13T18:51:14.582 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libdw1:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T18:51:14.653 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T18:51:14.708 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up postgresql-client (12+214ubuntu0.1) ... 2024-09-13T18:51:14.779 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T18:51:14.885 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up network-manager (1.22.10-1ubuntu2.4) ... 2024-09-13T18:51:14.939 INFO:teuthology.orchestra.run.smithi171.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T18:51:15.209 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T18:51:15.210 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T18:51:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:15.590 INFO:teuthology.orchestra.run.smithi007.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-13T18:51:15.590 INFO:teuthology.orchestra.run.smithi007.stdout:which means they are currently configured by ifupdown: 2024-09-13T18:51:15.590 INFO:teuthology.orchestra.run.smithi007.stdout:- ens1f0 2024-09-13T18:51:15.591 INFO:teuthology.orchestra.run.smithi007.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-13T18:51:15.591 INFO:teuthology.orchestra.run.smithi007.stdout:remove their configuration from /etc/network/interfaces. 2024-09-13T18:51:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:15.717 INFO:teuthology.orchestra.run.smithi171.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-09-13T18:51:15.757 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-13T18:51:15.826 INFO:teuthology.orchestra.run.smithi171.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-13T18:51:15.985 INFO:teuthology.orchestra.run.smithi171.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T18:51:16.092 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-13T18:51:16.137 INFO:teuthology.orchestra.run.smithi171.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T18:51:16.446 INFO:teuthology.orchestra.run.smithi171.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T18:51:16.626 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-13T18:51:17.104 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libmbim-proxy (1.26.2-1~ubuntu20.04.1) ... 2024-09-13T18:51:17.297 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up network-manager-pptp (1.2.8-2) ... 2024-09-13T18:51:17.440 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libelf-dev:amd64 (0.176-1.1ubuntu0.1) ... 2024-09-13T18:51:17.566 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up postgresql-contrib (12+214ubuntu0.1) ... 2024-09-13T18:51:17.734 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up postgresql (12+214ubuntu0.1) ... 2024-09-13T18:51:17.860 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libqmi-glib5:amd64 (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T18:51:18.011 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up libqmi-proxy (1.30.4-1~ubuntu20.04.1) ... 2024-09-13T18:51:18.162 INFO:teuthology.orchestra.run.smithi007.stdout:Setting up modemmanager (1.18.6-1~ubuntu20.04.1) ... 2024-09-13T18:51:18.442 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T18:51:18.442 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T18:51:18.940 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for mime-support (3.64ubuntu1) ... 2024-09-13T18:51:19.083 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-13T18:51:19.209 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T18:51:19.369 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T18:51:19.670 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T18:51:23.826 INFO:teuthology.orchestra.run.smithi171.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-09-13T18:51:23.951 INFO:teuthology.orchestra.run.smithi171.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-09-13T18:51:24.824 INFO:teuthology.orchestra.run.smithi171.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T18:51:24.830 DEBUG:teuthology.parallel:result is None 2024-09-13T18:51:26.991 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for dbus (1.12.16-2ubuntu2.2) ... 2024-09-13T18:51:27.099 INFO:teuthology.orchestra.run.smithi007.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2024-09-13T18:51:28.011 INFO:teuthology.orchestra.run.smithi007.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T18:51:28.016 DEBUG:teuthology.parallel:result is None 2024-09-13T18:51:28.017 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:51:28.166 DEBUG:teuthology.orchestra.run.smithi007:> dpkg-query -W -f '${Version}' ceph 2024-09-13T18:51:28.190 INFO:teuthology.orchestra.run.smithi007.stdout:18.2.4-860-g6c6b083f-1focal 2024-09-13T18:51:28.191 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-860-g6c6b083f-1focal 2024-09-13T18:51:28.191 INFO:teuthology.task.install:The correct ceph version 18.2.4-860-g6c6b083f-1focal is installed. 2024-09-13T18:51:28.193 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T18:51:28.341 DEBUG:teuthology.orchestra.run.smithi171:> dpkg-query -W -f '${Version}' ceph 2024-09-13T18:51:28.369 INFO:teuthology.orchestra.run.smithi171.stdout:18.2.4-860-g6c6b083f-1focal 2024-09-13T18:51:28.370 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-860-g6c6b083f-1focal 2024-09-13T18:51:28.370 INFO:teuthology.task.install:The correct ceph version 18.2.4-860-g6c6b083f-1focal is installed. 2024-09-13T18:51:28.373 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T18:51:28.373 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:28.373 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T18:51:28.392 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:51:28.392 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T18:51:28.406 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T18:51:28.406 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:28.407 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T18:51:28.447 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T18:51:28.457 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:51:28.457 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T18:51:28.473 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T18:51:28.483 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T18:51:28.483 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:28.484 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T18:51:28.498 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T18:51:28.556 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:51:28.556 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T18:51:28.573 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T18:51:28.582 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T18:51:28.582 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:28.582 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T18:51:28.599 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T18:51:28.609 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:51:28.609 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T18:51:28.623 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T18:51:28.679 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T18:51:28.767 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T18:51:28.767 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /var/log/ceph 2024-09-13T18:51:28.770 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 777 /var/log/ceph 2024-09-13T18:51:28.785 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T18:51:28.785 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T18:51:28.787 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T18:51:28.843 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T18:51:28.844 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T18:51:28.846 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T18:51:28.866 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-13T18:51:28.866 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-13T18:51:28.867 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-13T18:51:28.867 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:28.867 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-13T18:51:28.874 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-13T18:51:28.956 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:51:28.956 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-13T18:51:28.963 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-13T18:51:29.006 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T18:51:29.006 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T18:51:29.007 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7904006', 'branch': 'wip-rishabh-testing-20240910.100730-reef', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/openfiletable}', 'email': None, 'first_in_suite': False, 'job_id': '7904006', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240910.100730-reef'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240910.100730-reef', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6439, 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'sleep_before_teardown': 0, 'subset': '964/1024', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240910.100730-reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi007.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi171.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_openfiletable']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-13_14:42:16', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T18:51:29.007 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T18:51:29.009 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T18:51:29.015 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T18:51:29.058 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T18:51:29.075 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:29.075 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T18:51:29.080 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T18:51:29.080 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-09-13T18:51:29.128 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T18:51:29.128 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:51:29.128 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 359 Links: 1 2024-09-13T18:51:29.128 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:51:29.128 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-13 18:46:54.124139469 +0000 2024-09-13T18:51:29.128 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-13 18:46:53.900144638 +0000 2024-09-13T18:51:29.128 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-13 18:46:53.900144638 +0000 2024-09-13T18:51:29.128 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2024-09-13T18:51:29.128 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T18:51:29.140 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-13T18:51:29.140 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-13T18:51:29.140 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000315509 s, 1.6 MB/s 2024-09-13T18:51:29.141 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T18:51:29.151 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-09-13T18:51:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T18:51:29.199 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:51:29.199 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 365 Links: 1 2024-09-13T18:51:29.200 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:51:29.200 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-13 18:46:54.528130151 +0000 2024-09-13T18:51:29.200 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-13 18:46:54.316135041 +0000 2024-09-13T18:51:29.200 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-13 18:46:54.316135041 +0000 2024-09-13T18:51:29.200 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2024-09-13T18:51:29.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T18:51:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-13T18:51:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-13T18:51:29.215 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000305774 s, 1.7 MB/s 2024-09-13T18:51:29.216 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T18:51:29.224 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-09-13T18:51:29.299 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T18:51:29.299 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:51:29.299 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 372 Links: 1 2024-09-13T18:51:29.299 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:51:29.299 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-13 18:46:54.996119355 +0000 2024-09-13T18:51:29.299 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-13 18:46:54.772124522 +0000 2024-09-13T18:51:29.299 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-13 18:46:54.772124522 +0000 2024-09-13T18:51:29.299 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2024-09-13T18:51:29.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T18:51:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-13T18:51:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-13T18:51:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.00029924 s, 1.7 MB/s 2024-09-13T18:51:29.313 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T18:51:29.323 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-09-13T18:51:29.371 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T18:51:29.371 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:51:29.371 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 376 Links: 1 2024-09-13T18:51:29.371 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:51:29.372 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-13 18:46:55.400110037 +0000 2024-09-13T18:51:29.372 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-13 18:46:55.196114742 +0000 2024-09-13T18:51:29.372 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-13 18:46:55.196114742 +0000 2024-09-13T18:51:29.372 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2024-09-13T18:51:29.372 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T18:51:29.385 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-13T18:51:29.385 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-13T18:51:29.385 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000306913 s, 1.7 MB/s 2024-09-13T18:51:29.386 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T18:51:29.394 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T18:51:29.394 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:51:29.394 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T18:51:29.401 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T18:51:29.401 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_1 2024-09-13T18:51:29.408 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T18:51:29.409 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:51:29.409 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 6h/6d Inode: 361 Links: 1 2024-09-13T18:51:29.409 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:51:29.409 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-13 18:46:54.944608003 +0000 2024-09-13T18:51:29.409 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-13 18:46:54.728612980 +0000 2024-09-13T18:51:29.409 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-13 18:46:54.728612980 +0000 2024-09-13T18:51:29.409 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: - 2024-09-13T18:51:29.409 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T18:51:29.422 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-13T18:51:29.422 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-13T18:51:29.422 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000432284 s, 1.2 MB/s 2024-09-13T18:51:29.424 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T18:51:29.477 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_2 2024-09-13T18:51:29.484 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T18:51:29.484 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:51:29.484 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 6h/6d Inode: 365 Links: 1 2024-09-13T18:51:29.484 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:51:29.484 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-13 18:46:55.388597767 +0000 2024-09-13T18:51:29.484 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-13 18:46:55.180602562 +0000 2024-09-13T18:51:29.484 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-13 18:46:55.180602562 +0000 2024-09-13T18:51:29.485 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: - 2024-09-13T18:51:29.485 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T18:51:29.498 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-13T18:51:29.498 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-13T18:51:29.498 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000418174 s, 1.2 MB/s 2024-09-13T18:51:29.499 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T18:51:29.506 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_3 2024-09-13T18:51:29.559 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T18:51:29.559 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:51:29.559 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 6h/6d Inode: 373 Links: 1 2024-09-13T18:51:29.559 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:51:29.559 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-13 18:46:55.800588271 +0000 2024-09-13T18:51:29.560 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-13 18:46:55.604592790 +0000 2024-09-13T18:51:29.560 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-13 18:46:55.604592790 +0000 2024-09-13T18:51:29.560 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: - 2024-09-13T18:51:29.560 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T18:51:29.573 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-13T18:51:29.573 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-13T18:51:29.573 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000408904 s, 1.3 MB/s 2024-09-13T18:51:29.574 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T18:51:29.583 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_4 2024-09-13T18:51:29.593 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T18:51:29.593 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T18:51:29.593 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 6h/6d Inode: 379 Links: 1 2024-09-13T18:51:29.593 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T18:51:29.593 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-13 18:46:56.232578314 +0000 2024-09-13T18:51:29.593 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-13 18:46:56.040582739 +0000 2024-09-13T18:51:29.593 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-13 18:46:56.040582739 +0000 2024-09-13T18:51:29.593 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: - 2024-09-13T18:51:29.593 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T18:51:29.650 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-13T18:51:29.651 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-13T18:51:29.651 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000367275 s, 1.4 MB/s 2024-09-13T18:51:29.651 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T18:51:29.658 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T18:51:29.659 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi171.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T18:51:29.659 INFO:tasks.ceph:Generating config... 2024-09-13T18:51:29.660 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-13T18:51:29.660 INFO:tasks.ceph:[client] debug client = 20 2024-09-13T18:51:29.660 INFO:tasks.ceph:[client] debug ms = 1 2024-09-13T18:51:29.660 INFO:tasks.ceph:[client] pid file = 2024-09-13T18:51:29.660 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-13T18:51:29.660 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-13T18:51:29.661 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-13T18:51:29.661 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-13T18:51:29.661 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-13T18:51:29.661 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-13T18:51:29.661 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-13T18:51:29.661 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-13T18:51:29.661 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-13T18:51:29.661 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-13T18:51:29.661 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-13T18:51:29.661 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-13T18:51:29.661 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-13T18:51:29.661 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T18:51:29.661 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T18:51:29.662 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T18:51:29.662 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T18:51:29.662 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T18:51:29.662 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-13T18:51:29.662 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-13T18:51:29.662 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-13T18:51:29.662 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-13T18:51:29.662 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-13T18:51:29.662 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T18:51:29.662 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T18:51:29.662 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-13T18:51:29.662 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-13T18:51:29.662 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T18:51:29.663 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T18:51:29.663 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-13T18:51:29.663 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T18:51:29.663 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T18:51:29.663 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T18:51:29.663 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T18:51:29.663 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T18:51:29.663 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-13T18:51:29.663 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T18:51:29.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T18:51:29.706 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T18:51:29.722 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T18:51:29.763 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T18:51:29.776 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.7'), ('mon.b', '172.21.15.171'), ('mon.c', '[v2:172.21.15.171:3301,v1:172.21.15.171:6790]')] 2024-09-13T18:51:29.776 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.7,172.21.15.171,[v2:172.21.15.171:3301,v1:172.21.15.171:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-13T18:51:29.777 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:29.777 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T18:51:29.827 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.7 --add b 172.21.15.171 --addv c '[v2:172.21.15.171:3301,v1:172.21.15.171:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:51:29.871 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:51:29.871 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: generated fsid 33406d92-f401-438a-876e-a56de3962d8e 2024-09-13T18:51:29.871 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = pacific 2024-09-13T18:51:29.871 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 0 2024-09-13T18:51:29.871 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 33406d92-f401-438a-876e-a56de3962d8e 2024-09-13T18:51:29.871 INFO:teuthology.orchestra.run.smithi007.stdout:last_changed 2024-09-13T18:51:29.869501+0000 2024-09-13T18:51:29.871 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-13T18:51:29.869501+0000 2024-09-13T18:51:29.871 INFO:teuthology.orchestra.run.smithi007.stdout:min_mon_release 16 (pacific) 2024-09-13T18:51:29.871 INFO:teuthology.orchestra.run.smithi007.stdout:election_strategy: 1 2024-09-13T18:51:29.872 INFO:teuthology.orchestra.run.smithi007.stdout:0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-09-13T18:51:29.872 INFO:teuthology.orchestra.run.smithi007.stdout:1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.b 2024-09-13T18:51:29.872 INFO:teuthology.orchestra.run.smithi007.stdout:2: [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] mon.c 2024-09-13T18:51:29.872 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T18:51:29.873 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T18:51:29.878 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 33406d92-f401-438a-876e-a56de3962d8e... 2024-09-13T18:51:29.879 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T18:51:29.881 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T18:51:29.911 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-09-13T18:51:29.911 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-09-13T18:51:29.911 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T18:51:29.911 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-09-13T18:51:29.911 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.911 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-09-13T18:51:29.911 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-09-13T18:51:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-09-13T18:51:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-09-13T18:51:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-09-13T18:51:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-09-13T18:51:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T18:51:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-09-13T18:51:29.912 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T18:51:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T18:51:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-09-13T18:51:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-09-13T18:51:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-09-13T18:51:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-09-13T18:51:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-09-13T18:51:29.913 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-09-13T18:51:29.914 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-09-13T18:51:29.914 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-09-13T18:51:29.914 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T18:51:29.914 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T18:51:29.914 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-09-13T18:51:29.914 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.914 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T18:51:29.914 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.914 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-09-13T18:51:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-09-13T18:51:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-09-13T18:51:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-09-13T18:51:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-09-13T18:51:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.171,[v2:172.21.15.171:3301,v1:172.21.15.171:6790]" 2024-09-13T18:51:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 33406d92-f401-438a-876e-a56de3962d8e 2024-09-13T18:51:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.915 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-09-13T18:51:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-09-13T18:51:29.915 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-09-13T18:51:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-09-13T18:51:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-09-13T18:51:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-09-13T18:51:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-09-13T18:51:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-09-13T18:51:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-09-13T18:51:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-09-13T18:51:29.916 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-09-13T18:51:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-09-13T18:51:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-09-13T18:51:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-09-13T18:51:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-09-13T18:51:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-09-13T18:51:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-09-13T18:51:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-09-13T18:51:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-09-13T18:51:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: bdev async discard = True 2024-09-13T18:51:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: bdev enable discard = True 2024-09-13T18:51:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore allocator = bitmap 2024-09-13T18:51:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-09-13T18:51:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-09-13T18:51:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 1/20 2024-09-13T18:51:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 1/20 2024-09-13T18:51:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:51:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-09-13T18:51:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 4/10 2024-09-13T18:51:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T18:51:29.918 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-09-13T18:51:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-09-13T18:51:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2024-09-13T18:51:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2024-09-13T18:51:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-09-13T18:51:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.919 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-09-13T18:51:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:51:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-09-13T18:51:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-13T18:51:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-13T18:51:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:51:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:51:29.919 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-09-13T18:51:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-09-13T18:51:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.920 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-09-13T18:51:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:51:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-13T18:51:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-09-13T18:51:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-13T18:51:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-09-13T18:51:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-09-13T18:51:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:51:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-09-13T18:51:29.920 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-09-13T18:51:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:51:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T18:51:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T18:51:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T18:51:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T18:51:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T18:51:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T18:51:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.921 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-09-13T18:51:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-09-13T18:51:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-09-13T18:51:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: mon op complaint time = 120 2024-09-13T18:51:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on pool no app = False 2024-09-13T18:51:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:51:29.922 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-09-13T18:51:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-09-13T18:51:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-09-13T18:51:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-09-13T18:51:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T18:51:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T18:51:29.922 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 600 2024-09-13T18:51:29.923 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-09-13T18:51:29.923 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:51:29.923 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = "" 2024-09-13T18:51:29.923 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-09-13T18:51:29.923 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-09-13T18:51:29.923 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-09-13T18:51:29.923 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-09-13T18:51:29.923 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-09-13T18:51:29.923 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-09-13T18:51:29.923 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-09-13T18:51:29.923 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-09-13T18:51:29.923 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-09-13T18:51:29.923 INFO:teuthology.orchestra.run.smithi007.stdout:[mds] 2024-09-13T18:51:29.923 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds = 20 2024-09-13T18:51:29.923 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds balancer = 20 2024-09-13T18:51:29.924 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:51:29.924 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug frag = True 2024-09-13T18:51:29.924 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug scatterstat = True 2024-09-13T18:51:29.924 INFO:teuthology.orchestra.run.smithi007.stdout: mds op complaint time = 180 2024-09-13T18:51:29.924 INFO:teuthology.orchestra.run.smithi007.stdout: mds verify scatter = True 2024-09-13T18:51:29.924 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-09-13T18:51:29.924 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-09-13T18:51:29.924 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-09-13T18:51:29.926 INFO:teuthology.orchestra.run.smithi171.stdout:[global] 2024-09-13T18:51:29.926 INFO:teuthology.orchestra.run.smithi171.stdout: chdir = "" 2024-09-13T18:51:29.926 INFO:teuthology.orchestra.run.smithi171.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T18:51:29.926 INFO:teuthology.orchestra.run.smithi171.stdout: auth supported = cephx 2024-09-13T18:51:29.926 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.926 INFO:teuthology.orchestra.run.smithi171.stdout: filestore xattr use omap = true 2024-09-13T18:51:29.926 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.926 INFO:teuthology.orchestra.run.smithi171.stdout: mon clock drift allowed = 1.000 2024-09-13T18:51:29.926 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.926 INFO:teuthology.orchestra.run.smithi171.stdout: osd crush chooseleaf type = 0 2024-09-13T18:51:29.927 INFO:teuthology.orchestra.run.smithi171.stdout: auth debug = true 2024-09-13T18:51:29.927 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.927 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on old message = true 2024-09-13T18:51:29.927 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on bug = true 2024-09-13T18:51:29.927 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.927 INFO:teuthology.orchestra.run.smithi171.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T18:51:29.927 INFO:teuthology.orchestra.run.smithi171.stdout: mon pg warn max object skew = 0 2024-09-13T18:51:29.927 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.927 INFO:teuthology.orchestra.run.smithi171.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T18:51:29.927 INFO:teuthology.orchestra.run.smithi171.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T18:51:29.927 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.927 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default size = 2 2024-09-13T18:51:29.927 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow primary affinity = true 2024-09-13T18:51:29.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow pg remap = true 2024-09-13T18:51:29.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on legacy crush tunables = false 2024-09-13T18:51:29.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on crush straw calc version zero = false 2024-09-13T18:51:29.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on no sortbitwise = false 2024-09-13T18:51:29.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on osd down out interval zero = false 2024-09-13T18:51:29.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on too few osds = false 2024-09-13T18:51:29.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T18:51:29.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T18:51:29.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon_allow_pool_size_one = true 2024-09-13T18:51:29.928 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.928 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T18:51:29.928 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.929 INFO:teuthology.orchestra.run.smithi171.stdout: osd default data pool replay window = 5 2024-09-13T18:51:29.929 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.929 INFO:teuthology.orchestra.run.smithi171.stdout: mon allow pool delete = true 2024-09-13T18:51:29.929 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.929 INFO:teuthology.orchestra.run.smithi171.stdout: mon cluster log file level = debug 2024-09-13T18:51:29.929 INFO:teuthology.orchestra.run.smithi171.stdout: debug asserts on shutdown = true 2024-09-13T18:51:29.930 INFO:teuthology.orchestra.run.smithi171.stdout: mon health detail to clog = false 2024-09-13T18:51:29.930 INFO:teuthology.orchestra.run.smithi171.stdout: mon host = "172.21.15.7,172.21.15.171,[v2:172.21.15.171:3301,v1:172.21.15.171:6790]" 2024-09-13T18:51:29.930 INFO:teuthology.orchestra.run.smithi171.stdout: fsid = 33406d92-f401-438a-876e-a56de3962d8e 2024-09-13T18:51:29.930 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.930 INFO:teuthology.orchestra.run.smithi171.stdout:[osd] 2024-09-13T18:51:29.930 INFO:teuthology.orchestra.run.smithi171.stdout: osd journal size = 100 2024-09-13T18:51:29.930 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.930 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub load threshold = 5.0 2024-09-13T18:51:29.930 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub max interval = 600 2024-09-13T18:51:29.930 INFO:teuthology.orchestra.run.smithi171.stdout: osd mclock profile = high_recovery_ops 2024-09-13T18:51:29.930 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.930 INFO:teuthology.orchestra.run.smithi171.stdout: osd recover clone overlap = true 2024-09-13T18:51:29.931 INFO:teuthology.orchestra.run.smithi171.stdout: osd recovery max chunk = 1048576 2024-09-13T18:51:29.931 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.931 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug shutdown = true 2024-09-13T18:51:29.931 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug op order = true 2024-09-13T18:51:29.931 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug verify stray on activate = true 2024-09-13T18:51:29.931 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.932 INFO:teuthology.orchestra.run.smithi171.stdout: osd open classes on start = true 2024-09-13T18:51:29.932 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug pg log writeout = true 2024-09-13T18:51:29.932 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.932 INFO:teuthology.orchestra.run.smithi171.stdout: osd deep scrub update digest min age = 30 2024-09-13T18:51:29.932 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.933 INFO:teuthology.orchestra.run.smithi171.stdout: osd map max advance = 10 2024-09-13T18:51:29.933 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.933 INFO:teuthology.orchestra.run.smithi171.stdout: journal zero on create = true 2024-09-13T18:51:29.933 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.933 INFO:teuthology.orchestra.run.smithi171.stdout: filestore ondisk finisher threads = 3 2024-09-13T18:51:29.934 INFO:teuthology.orchestra.run.smithi171.stdout: filestore apply finisher threads = 3 2024-09-13T18:51:29.934 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.934 INFO:teuthology.orchestra.run.smithi171.stdout: bdev debug aio = true 2024-09-13T18:51:29.934 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug misdirected ops = true 2024-09-13T18:51:29.934 INFO:teuthology.orchestra.run.smithi171.stdout: bdev async discard = True 2024-09-13T18:51:29.935 INFO:teuthology.orchestra.run.smithi171.stdout: bdev enable discard = True 2024-09-13T18:51:29.935 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore allocator = bitmap 2024-09-13T18:51:29.935 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore block size = 96636764160 2024-09-13T18:51:29.935 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore fsck on mount = True 2024-09-13T18:51:29.935 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluefs = 1/20 2024-09-13T18:51:29.935 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluestore = 1/20 2024-09-13T18:51:29.935 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:51:29.935 INFO:teuthology.orchestra.run.smithi171.stdout: debug osd = 20 2024-09-13T18:51:29.935 INFO:teuthology.orchestra.run.smithi171.stdout: debug rocksdb = 4/10 2024-09-13T18:51:29.935 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T18:51:29.935 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd full ratio = 0.9 2024-09-13T18:51:29.935 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd nearfull ratio = 0.8 2024-09-13T18:51:29.936 INFO:teuthology.orchestra.run.smithi171.stdout: osd failsafe full ratio = 0.95 2024-09-13T18:51:29.936 INFO:teuthology.orchestra.run.smithi171.stdout: osd objectstore = bluestore 2024-09-13T18:51:29.936 INFO:teuthology.orchestra.run.smithi171.stdout: osd op complaint time = 180 2024-09-13T18:51:29.936 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.936 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr] 2024-09-13T18:51:29.936 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:51:29.936 INFO:teuthology.orchestra.run.smithi171.stdout: debug mgr = 20 2024-09-13T18:51:29.936 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-09-13T18:51:29.937 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-09-13T18:51:29.937 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:51:29.937 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:51:29.937 INFO:teuthology.orchestra.run.smithi171.stdout: mgr/telemetry/nag = false 2024-09-13T18:51:29.937 INFO:teuthology.orchestra.run.smithi171.stdout: debug client = 20 2024-09-13T18:51:29.937 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.937 INFO:teuthology.orchestra.run.smithi171.stdout:[mon] 2024-09-13T18:51:29.937 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:51:29.937 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-09-13T18:51:29.937 INFO:teuthology.orchestra.run.smithi171.stdout: debug paxos = 20 2024-09-13T18:51:29.937 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-09-13T18:51:29.937 INFO:teuthology.orchestra.run.smithi171.stdout: mon data avail warn = 5 2024-09-13T18:51:29.938 INFO:teuthology.orchestra.run.smithi171.stdout: mon mgr mkfs grace = 240 2024-09-13T18:51:29.938 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:51:29.938 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd reporter subtree level = osd 2024-09-13T18:51:29.938 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd prime pg temp = true 2024-09-13T18:51:29.938 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:51:29.938 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.938 INFO:teuthology.orchestra.run.smithi171.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T18:51:29.939 INFO:teuthology.orchestra.run.smithi171.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T18:51:29.939 INFO:teuthology.orchestra.run.smithi171.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T18:51:29.939 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.939 INFO:teuthology.orchestra.run.smithi171.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T18:51:29.939 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T18:51:29.939 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T18:51:29.939 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.940 INFO:teuthology.orchestra.run.smithi171.stdout: # 1m isn't quite enough 2024-09-13T18:51:29.940 INFO:teuthology.orchestra.run.smithi171.stdout: mon_down_mkfs_grace = 2m 2024-09-13T18:51:29.940 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.940 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_filestore_osds = false 2024-09-13T18:51:29.940 INFO:teuthology.orchestra.run.smithi171.stdout: mon op complaint time = 120 2024-09-13T18:51:29.940 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on pool no app = False 2024-09-13T18:51:29.940 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:51:29.940 INFO:teuthology.orchestra.run.smithi171.stdout:[client] 2024-09-13T18:51:29.940 INFO:teuthology.orchestra.run.smithi171.stdout: rgw cache enabled = true 2024-09-13T18:51:29.940 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable ops log = true 2024-09-13T18:51:29.940 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable usage log = true 2024-09-13T18:51:29.940 INFO:teuthology.orchestra.run.smithi171.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T18:51:29.940 INFO:teuthology.orchestra.run.smithi171.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T18:51:29.941 INFO:teuthology.orchestra.run.smithi171.stdout: client mount timeout = 600 2024-09-13T18:51:29.941 INFO:teuthology.orchestra.run.smithi171.stdout: debug client = 20 2024-09-13T18:51:29.941 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:51:29.941 INFO:teuthology.orchestra.run.smithi171.stdout: pid file = "" 2024-09-13T18:51:29.941 INFO:teuthology.orchestra.run.smithi171.stdout: rados mon op timeout = 900 2024-09-13T18:51:29.941 INFO:teuthology.orchestra.run.smithi171.stdout: rados osd op timeout = 900 2024-09-13T18:51:29.941 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.a] 2024-09-13T18:51:29.941 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.b] 2024-09-13T18:51:29.941 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.c] 2024-09-13T18:51:29.941 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.a] 2024-09-13T18:51:29.942 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.c] 2024-09-13T18:51:29.942 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.b] 2024-09-13T18:51:29.942 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.d] 2024-09-13T18:51:29.942 INFO:teuthology.orchestra.run.smithi171.stdout:[mds] 2024-09-13T18:51:29.942 INFO:teuthology.orchestra.run.smithi171.stdout: debug mds = 20 2024-09-13T18:51:29.942 INFO:teuthology.orchestra.run.smithi171.stdout: debug mds balancer = 20 2024-09-13T18:51:29.942 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:51:29.942 INFO:teuthology.orchestra.run.smithi171.stdout: mds debug frag = True 2024-09-13T18:51:29.942 INFO:teuthology.orchestra.run.smithi171.stdout: mds debug scatterstat = True 2024-09-13T18:51:29.942 INFO:teuthology.orchestra.run.smithi171.stdout: mds op complaint time = 180 2024-09-13T18:51:29.942 INFO:teuthology.orchestra.run.smithi171.stdout: mds verify scatter = True 2024-09-13T18:51:29.942 INFO:teuthology.orchestra.run.smithi171.stdout: osd op complaint time = 180 2024-09-13T18:51:29.942 INFO:teuthology.orchestra.run.smithi171.stdout: rados mon op timeout = 900 2024-09-13T18:51:29.942 INFO:teuthology.orchestra.run.smithi171.stdout: rados osd op timeout = 900 2024-09-13T18:51:29.943 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T18:51:29.943 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T18:51:30.025 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T18:51:30.026 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:30.026 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T18:51:30.033 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:30.033 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T18:51:30.040 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi007.front.sepia.ceph.com 2024-09-13T18:51:30.040 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:30.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T18:51:30.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T18:51:30.107 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:30.107 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:51:30.112 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T18:51:30.112 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:51:30.112 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T18:51:30.112 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T18:51:30.139 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:51:30.140 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:51:30.144 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T18:51:30.144 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T18:51:30.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T18:51:30.193 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T18:51:30.197 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T18:51:30.248 INFO:teuthology.orchestra.run.smithi171.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T18:51:30.277 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T18:51:30.278 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-13T18:51:30.331 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-13T18:51:30.334 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-13T18:51:30.350 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-13T18:51:30.399 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-13T18:51:30.402 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-13T18:51:30.415 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-13T18:51:30.463 INFO:teuthology.orchestra.run.smithi171.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-13T18:51:30.466 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-13T18:51:30.482 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-13T18:51:30.526 INFO:teuthology.orchestra.run.smithi171.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-13T18:51:30.529 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-13T18:51:30.539 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T18:51:30.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T18:51:30.582 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T18:51:30.597 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-13T18:51:30.685 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-13T18:51:30.699 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-13T18:51:30.740 INFO:teuthology.orchestra.run.smithi171.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-13T18:51:30.753 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-13T18:51:30.835 INFO:teuthology.orchestra.run.smithi171.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-13T18:51:30.849 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T18:51:30.849 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi171.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T18:51:30.849 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T18:51:30.867 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T18:51:30.867 INFO:tasks.ceph:role: osd.0 2024-09-13T18:51:30.867 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-13T18:51:30.867 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T18:51:30.953 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:51:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:51:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:51:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 2024-09-13T18:51:30.953 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:51:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:51:30.954 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:51:30.954 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T18:51:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:51:30.954 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:51:30.955 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-13T18:51:30.956 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T18:51:30.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T18:51:30.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:51:30.987 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T18:51:30.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T18:51:31.045 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T18:51:31.045 INFO:tasks.ceph:role: osd.1 2024-09-13T18:51:31.045 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-13T18:51:31.045 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T18:51:31.086 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:51:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:51:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:51:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 2024-09-13T18:51:31.086 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:51:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:51:31.086 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:51:31.086 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T18:51:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:51:31.087 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:51:31.088 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-13T18:51:31.088 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T18:51:31.109 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T18:51:31.164 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T18:51:31.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:51:31.166 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T18:51:31.176 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T18:51:31.176 INFO:tasks.ceph:role: osd.2 2024-09-13T18:51:31.176 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-13T18:51:31.177 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T18:51:31.216 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:51:31.216 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:51:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:51:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 2024-09-13T18:51:31.217 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:51:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:51:31.217 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:51:31.217 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T18:51:31.217 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:51:31.217 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:51:31.218 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-13T18:51:31.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T18:51:31.282 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T18:51:31.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:51:31.292 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T18:51:31.292 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T18:51:31.303 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T18:51:31.303 INFO:tasks.ceph:role: osd.3 2024-09-13T18:51:31.303 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-13T18:51:31.303 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T18:51:31.345 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:51:31.345 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:51:31.345 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:51:31.345 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 2024-09-13T18:51:31.345 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:51:31.346 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:51:31.346 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:51:31.346 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T18:51:31.346 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:51:31.346 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:51:31.347 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-13T18:51:31.347 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-13T18:51:31.409 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T18:51:31.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:51:31.423 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T18:51:31.424 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:51:31.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:31.495+0000 7f69c8b1a880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T18:51:31.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:31.495+0000 7f69c8b1a880 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T18:51:31.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:31.495+0000 7f69c8b1a880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T18:51:31.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:31.495+0000 7f69c8b1a880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T18:51:31.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:31.495+0000 7f69c8b1a880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T18:51:31.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:31.495+0000 7f69c8b1a880 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T18:51:34.488 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T18:51:34.505 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:51:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:34.543+0000 7f64507b6880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T18:51:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:34.543+0000 7f64507b6880 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T18:51:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:34.543+0000 7f64507b6880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T18:51:34.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:34.543+0000 7f64507b6880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T18:51:34.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:34.543+0000 7f64507b6880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T18:51:34.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:34.543+0000 7f64507b6880 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T18:51:37.560 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T18:51:37.575 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:51:37.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:37.659+0000 7ff447999880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T18:51:37.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:37.659+0000 7ff447999880 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T18:51:37.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:37.659+0000 7ff447999880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T18:51:37.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:37.659+0000 7ff447999880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T18:51:37.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:37.659+0000 7ff447999880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T18:51:37.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:37.659+0000 7ff447999880 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T18:51:40.668 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T18:51:40.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:51:40.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:40.723+0000 7f6f383c1880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T18:51:40.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:40.723+0000 7f6f383c1880 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T18:51:40.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:40.723+0000 7f6f383c1880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T18:51:40.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:40.723+0000 7f6f383c1880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T18:51:40.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:40.723+0000 7f6f383c1880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T18:51:40.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:51:40.727+0000 7f6f383c1880 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T18:51:43.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T18:51:43.770 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-13T18:51:43.787 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T18:51:43.787 INFO:tasks.ceph:role: osd.4 2024-09-13T18:51:43.787 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T18:51:43.788 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T18:51:44.309 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:51:44.310 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:51:44.310 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:51:44.310 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 2024-09-13T18:51:44.310 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:51:44.310 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:51:44.310 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:51:44.310 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T18:51:44.310 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:51:44.310 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:51:44.312 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-09-13T18:51:44.312 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-13T18:51:44.407 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-13T18:51:44.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:51:44.418 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T18:51:44.419 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-13T18:51:44.429 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T18:51:44.429 INFO:tasks.ceph:role: osd.5 2024-09-13T18:51:44.429 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T18:51:44.429 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T18:51:44.976 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:51:44.976 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:51:44.976 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:51:44.976 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 2024-09-13T18:51:44.976 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:51:44.977 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:51:44.977 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:51:44.977 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T18:51:44.977 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:51:44.977 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:51:44.978 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-09-13T18:51:44.978 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-13T18:51:45.017 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-13T18:51:45.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:51:45.025 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T18:51:45.025 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-13T18:51:45.035 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T18:51:45.035 INFO:tasks.ceph:role: osd.6 2024-09-13T18:51:45.036 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T18:51:45.036 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T18:51:45.496 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:51:45.496 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:51:45.496 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:51:45.496 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 2024-09-13T18:51:45.496 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:51:45.497 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:51:45.497 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:51:45.497 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T18:51:45.498 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:51:45.498 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:51:45.499 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-09-13T18:51:45.499 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-13T18:51:45.559 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-13T18:51:45.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:51:45.571 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T18:51:45.571 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-13T18:51:45.581 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T18:51:45.581 INFO:tasks.ceph:role: osd.7 2024-09-13T18:51:45.582 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T18:51:45.582 DEBUG:teuthology.orchestra.run.smithi171:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T18:51:46.087 INFO:teuthology.orchestra.run.smithi171.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T18:51:46.087 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T18:51:46.087 INFO:teuthology.orchestra.run.smithi171.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T18:51:46.087 INFO:teuthology.orchestra.run.smithi171.stdout: = reflink=1 2024-09-13T18:51:46.087 INFO:teuthology.orchestra.run.smithi171.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T18:51:46.087 INFO:teuthology.orchestra.run.smithi171.stdout: = sunit=0 swidth=0 blks 2024-09-13T18:51:46.088 INFO:teuthology.orchestra.run.smithi171.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T18:51:46.088 INFO:teuthology.orchestra.run.smithi171.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T18:51:46.088 INFO:teuthology.orchestra.run.smithi171.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T18:51:46.088 INFO:teuthology.orchestra.run.smithi171.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T18:51:46.089 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi171.front.sepia.ceph.com -o noatime 2024-09-13T18:51:46.089 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-13T18:51:46.140 DEBUG:teuthology.orchestra.run.smithi171:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-13T18:51:46.153 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T18:51:46.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:51:46.154 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:51:46.291 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:46.286+0000 7feea7f22880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-13T18:51:46.291 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:46.286+0000 7feea7f22880 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-13T18:51:46.291 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:46.286+0000 7feea7f22880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T18:51:46.291 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:46.286+0000 7feea7f22880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T18:51:46.292 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:46.286+0000 7feea7f22880 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T18:51:46.292 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:46.290+0000 7feea7f22880 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-13T18:51:49.312 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-13T18:51:49.326 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:51:49.373 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:49.370+0000 7f32e43c3880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-13T18:51:49.373 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:49.370+0000 7f32e43c3880 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-13T18:51:49.373 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:49.370+0000 7f32e43c3880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T18:51:49.373 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:49.370+0000 7f32e43c3880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T18:51:49.373 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:49.370+0000 7f32e43c3880 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T18:51:49.374 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:49.370+0000 7f32e43c3880 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-13T18:51:52.382 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-13T18:51:52.396 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:51:52.438 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:52.434+0000 7fa390334880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-13T18:51:52.438 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:52.434+0000 7fa390334880 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-13T18:51:52.438 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:52.434+0000 7fa390334880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T18:51:52.438 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:52.434+0000 7fa390334880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T18:51:52.438 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:52.434+0000 7fa390334880 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T18:51:52.438 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:52.434+0000 7fa390334880 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-13T18:51:55.467 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-13T18:51:55.525 DEBUG:teuthology.orchestra.run.smithi171:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:51:55.569 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:55.566+0000 7f5e18ce1880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-13T18:51:55.570 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:55.566+0000 7f5e18ce1880 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-13T18:51:55.570 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:55.566+0000 7f5e18ce1880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T18:51:55.570 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:55.566+0000 7f5e18ce1880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T18:51:55.570 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:55.566+0000 7f5e18ce1880 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T18:51:55.571 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:51:55.566+0000 7f5e18ce1880 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-13T18:51:58.579 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-13T18:51:58.595 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T18:51:58.595 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:58.595 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-13T18:51:58.612 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:58.612 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-13T18:51:58.666 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:58.667 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-13T18:51:58.678 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:58.679 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T18:51:58.694 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:58.694 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T18:51:58.707 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:58.707 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T18:51:58.721 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:58.721 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T18:51:58.733 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:51:58.734 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T18:51:58.749 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:51:58.750 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-13T18:51:58.808 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:51:58.808 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-13T18:51:58.819 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:51:58.819 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-13T18:51:58.829 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:51:58.829 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-13T18:51:58.885 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:51:58.885 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-13T18:51:58.901 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:51:58.901 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-13T18:51:58.915 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:58.916 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T18:51:58.922 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:51:58.923 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-13T18:51:58.971 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:51:58.971 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-13T18:51:58.978 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:51:58.978 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-13T18:51:58.985 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T18:51:58.985 DEBUG:teuthology.orchestra.run.smithi007:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T18:51:58.987 DEBUG:teuthology.orchestra.run.smithi171:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T18:51:58.998 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr.y] 2024-09-13T18:51:58.998 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCyieRmJQ94CxAAIusXMD6x4ey4OLJO9t7n+w== 2024-09-13T18:51:58.998 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.a] 2024-09-13T18:51:58.998 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCyieRmluqnExAAsUgfqXyo3d0MsBU4pTUFKA== 2024-09-13T18:51:58.998 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.c] 2024-09-13T18:51:58.998 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCyieRmovm+FxAAACx3SjaGO+6TwKcii8ZfcQ== 2024-09-13T18:51:58.998 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.0] 2024-09-13T18:51:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCzieRmoweYHRAAJq4W0iwb4bNJtNoojDmYvA== 2024-09-13T18:51:58.999 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.1] 2024-09-13T18:51:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQC2ieRmMlp6IBAA6UPBa8XtvToSEXOH7jRp2Q== 2024-09-13T18:51:58.999 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.2] 2024-09-13T18:51:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQC5ieRm8PRwJxAAJ6u6O2CY9AB8ADXGUq3tHw== 2024-09-13T18:51:58.999 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.3] 2024-09-13T18:51:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQC8ieRmpelUKxAAZkfxHmxH4AQs2+ZmYEMI5g== 2024-09-13T18:51:58.999 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr.x] 2024-09-13T18:51:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCyieRm67fCDhAAm9sMp6tlCuJ3vwp57kczzA== 2024-09-13T18:51:58.999 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.b] 2024-09-13T18:51:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCyieRmse2YGxAA1xevwPXbqkiFfH1c51CyUQ== 2024-09-13T18:51:58.999 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.d] 2024-09-13T18:51:58.999 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCyieRms9RQHxAAWQjRb5IFrsEkHgk1fFl1gA== 2024-09-13T18:51:59.000 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.4] 2024-09-13T18:51:59.000 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQDCieRmt6JLERAAOSeTXcLflQO4UStq8/r4Kg== 2024-09-13T18:51:59.000 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.5] 2024-09-13T18:51:59.000 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQDFieRmnrQzFhAAbpEXq+QqxOuUEon2UfwRww== 2024-09-13T18:51:59.000 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.6] 2024-09-13T18:51:59.000 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQDIieRmbacMGhAAXeApS17wKjLK0KlAvOFMCw== 2024-09-13T18:51:59.000 INFO:teuthology.orchestra.run.smithi171.stdout:[osd.7] 2024-09-13T18:51:59.000 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQDLieRmcxLrIRAAnXXU0UeSpdKkMzVuT2U9tA== 2024-09-13T18:51:59.000 INFO:teuthology.orchestra.run.smithi171.stdout:[client.0] 2024-09-13T18:51:59.001 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCyieRmcmOmIhAAoiL5c3tp8skQ5VcKF/6zVw== 2024-09-13T18:51:59.001 INFO:teuthology.orchestra.run.smithi171.stdout:[client.2] 2024-09-13T18:51:59.001 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCyieRmvonEKBAAefHhRIV+mux4L2yvJieukQ== 2024-09-13T18:51:59.001 INFO:teuthology.orchestra.run.smithi171.stdout:[client.1] 2024-09-13T18:51:59.001 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCyieRmcNsbLBAAj36vmzv7emQ+ueYkP/2HtA== 2024-09-13T18:51:59.001 INFO:teuthology.orchestra.run.smithi171.stdout:[client.3] 2024-09-13T18:51:59.001 INFO:teuthology.orchestra.run.smithi171.stdout: key = AQCyieRmh8G8MRAAeqibcXujnGSy9zs4XPf2hA== 2024-09-13T18:51:59.002 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.y] 2024-09-13T18:51:59.003 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCyieRmJQ94CxAAIusXMD6x4ey4OLJO9t7n+w== 2024-09-13T18:51:59.003 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-09-13T18:51:59.003 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCyieRmluqnExAAsUgfqXyo3d0MsBU4pTUFKA== 2024-09-13T18:51:59.003 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-09-13T18:51:59.003 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCyieRmovm+FxAAACx3SjaGO+6TwKcii8ZfcQ== 2024-09-13T18:51:59.003 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.0] 2024-09-13T18:51:59.003 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCzieRmoweYHRAAJq4W0iwb4bNJtNoojDmYvA== 2024-09-13T18:51:59.003 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.1] 2024-09-13T18:51:59.003 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQC2ieRmMlp6IBAA6UPBa8XtvToSEXOH7jRp2Q== 2024-09-13T18:51:59.003 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.2] 2024-09-13T18:51:59.003 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQC5ieRm8PRwJxAAJ6u6O2CY9AB8ADXGUq3tHw== 2024-09-13T18:51:59.003 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.3] 2024-09-13T18:51:59.003 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQC8ieRmpelUKxAAZkfxHmxH4AQs2+ZmYEMI5g== 2024-09-13T18:51:59.003 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.x] 2024-09-13T18:51:59.004 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCyieRm67fCDhAAm9sMp6tlCuJ3vwp57kczzA== 2024-09-13T18:51:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-09-13T18:51:59.004 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCyieRmse2YGxAA1xevwPXbqkiFfH1c51CyUQ== 2024-09-13T18:51:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-09-13T18:51:59.004 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCyieRms9RQHxAAWQjRb5IFrsEkHgk1fFl1gA== 2024-09-13T18:51:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.4] 2024-09-13T18:51:59.004 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDCieRmt6JLERAAOSeTXcLflQO4UStq8/r4Kg== 2024-09-13T18:51:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.5] 2024-09-13T18:51:59.004 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDFieRmnrQzFhAAbpEXq+QqxOuUEon2UfwRww== 2024-09-13T18:51:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.6] 2024-09-13T18:51:59.004 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDIieRmbacMGhAAXeApS17wKjLK0KlAvOFMCw== 2024-09-13T18:51:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.7] 2024-09-13T18:51:59.004 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDLieRmcxLrIRAAnXXU0UeSpdKkMzVuT2U9tA== 2024-09-13T18:51:59.004 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-09-13T18:51:59.005 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCyieRmcmOmIhAAoiL5c3tp8skQ5VcKF/6zVw== 2024-09-13T18:51:59.005 INFO:teuthology.orchestra.run.smithi007.stdout:[client.2] 2024-09-13T18:51:59.005 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCyieRmvonEKBAAefHhRIV+mux4L2yvJieukQ== 2024-09-13T18:51:59.005 INFO:teuthology.orchestra.run.smithi007.stdout:[client.1] 2024-09-13T18:51:59.005 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCyieRmcNsbLBAAj36vmzv7emQ+ueYkP/2HtA== 2024-09-13T18:51:59.005 INFO:teuthology.orchestra.run.smithi007.stdout:[client.3] 2024-09-13T18:51:59.005 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCyieRmh8G8MRAAeqibcXujnGSy9zs4XPf2hA== 2024-09-13T18:51:59.005 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T18:51:59.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:51:59.047 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T18:51:59.048 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:51:59.097 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:51:59.100 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:51:59.152 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:51:59.154 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:51:59.208 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:51:59.242 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:51:59.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:51:59.321 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:51:59.391 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:51:59.393 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:51:59.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:51:59.447 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:51:59.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T18:51:59.499 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T18:51:59.547 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:51:59.549 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:51:59.600 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:51:59.602 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T18:51:59.653 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:51:59.656 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:51:59.706 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:51:59.746 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:51:59.798 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:51:59.800 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:51:59.845 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:51:59.848 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T18:51:59.896 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T18:51:59.898 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T18:51:59.949 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T18:51:59.968 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T18:52:00.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T18:52:00.036 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T18:52:00.086 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T18:52:00.088 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T18:52:00.139 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T18:52:00.140 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T18:52:00.151 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T18:52:00.247 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T18:52:00.264 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T18:52:00.280 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T18:52:00.378 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T18:52:00.395 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T18:52:00.405 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T18:52:00.486 DEBUG:teuthology.orchestra.run.smithi171:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T18:52:00.502 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:52:00.504 DEBUG:teuthology.orchestra.run.smithi171:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T18:52:00.509 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T18:52:00.510 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T18:52:00.510 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T18:52:00.512 INFO:tasks.ceph.mon.a:Started 2024-09-13T18:52:00.512 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T18:52:00.512 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T18:52:00.514 INFO:tasks.ceph.mon.b:Started 2024-09-13T18:52:00.514 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T18:52:00.515 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T18:52:00.517 INFO:tasks.ceph.mon.c:Started 2024-09-13T18:52:00.517 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T18:52:00.517 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T18:52:00.517 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T18:52:00.519 INFO:tasks.ceph.mgr.y:Started 2024-09-13T18:52:00.519 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T18:52:00.520 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T18:52:00.522 INFO:tasks.ceph.mgr.x:Started 2024-09-13T18:52:00.522 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T18:52:00.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T18:52:00.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.622+0000 7efff4618700 1 Processor -- start 2024-09-13T18:52:00.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.622+0000 7efff4618700 1 -- start start 2024-09-13T18:52:00.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.626+0000 7efff4618700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effec158ea0 0x7effec06b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:00.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.626+0000 7efff4618700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effec071430 con 0x7effec159280 2024-09-13T18:52:00.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.626+0000 7efff4618700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effec071680 con 0x7effec06eb30 2024-09-13T18:52:00.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.626+0000 7efff23b4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effec158ea0 0x7effec06b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:00.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.626+0000 7efff4618700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effec071960 con 0x7effec158ea0 2024-09-13T18:52:00.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.626+0000 7efff1bb3700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7effec159280 0x7effec06e420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51476/0 (socket says 172.21.15.7:51476) 2024-09-13T18:52:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.626+0000 7efff1bb3700 1 -- 172.21.15.7:0/2065154316 learned_addr learned my addr 172.21.15.7:0/2065154316 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.626+0000 7efff23b4700 1 -- 172.21.15.7:0/2065154316 >> v2:172.21.15.7:3300/0 conn(0x7effec158ea0 msgr2=0x7effec06b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T18:52:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.626+0000 7efff23b4700 1 -- 172.21.15.7:0/2065154316 >> v2:172.21.15.7:3300/0 conn(0x7effec158ea0 msgr2=0x7effec06b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T18:52:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.626+0000 7efff23b4700 1 --2- 172.21.15.7:0/2065154316 >> v2:172.21.15.7:3300/0 conn(0x7effec158ea0 0x7effec06b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T18:52:00.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.626+0000 7efff23b4700 1 --2- 172.21.15.7:0/2065154316 >> v2:172.21.15.7:3300/0 conn(0x7effec158ea0 0x7effec06b850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T18:52:00.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.662+0000 7efff13b2700 1 -- 172.21.15.7:0/2065154316 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100810460 0 0) 0x7effec071680 con 0x7effec06eb30 2024-09-13T18:52:00.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff13b2700 1 -- 172.21.15.7:0/2065154316 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effd80035d0 con 0x7effec06eb30 2024-09-13T18:52:00.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff13b2700 1 -- 172.21.15.7:0/2065154316 <== mon.1 v1:172.21.15.171:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2349495694 0 0) 0x7effd80035d0 con 0x7effec06eb30 2024-09-13T18:52:00.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff13b2700 1 -- 172.21.15.7:0/2065154316 >> v2:172.21.15.7:3300/0 conn(0x7effec158ea0 msgr2=0x7effec06b850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:52:00.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff13b2700 1 --2- 172.21.15.7:0/2065154316 >> v2:172.21.15.7:3300/0 conn(0x7effec158ea0 0x7effec06b850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:00.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff13b2700 1 -- 172.21.15.7:0/2065154316 >> v1:172.21.15.7:6789/0 conn(0x7effec159280 legacy=0x7effec06e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:00.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff13b2700 1 -- 172.21.15.7:0/2065154316 --> v1:172.21.15.171:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effec071c40 con 0x7effec06eb30 2024-09-13T18:52:00.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff13b2700 1 -- 172.21.15.7:0/2065154316 <== mon.1 v1:172.21.15.171:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7effe40019f0 con 0x7effec06eb30 2024-09-13T18:52:00.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff13b2700 1 -- 172.21.15.7:0/2065154316 >> v1:172.21.15.171:6789/0 conn(0x7effec06eb30 legacy=0x7effec0b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:00.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff13b2700 1 --2- 172.21.15.7:0/2065154316 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effd8003a20 0x7effd8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:00.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff13b2700 1 --2- 172.21.15.7:0/2065154316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effd8004870 0x7effd8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:00.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff13b2700 1 --2- 172.21.15.7:0/2065154316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effd8005230 0x7effd800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:00.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff13b2700 1 -- 172.21.15.7:0/2065154316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec071960 con 0x7effd8005230 2024-09-13T18:52:00.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff13b2700 1 -- 172.21.15.7:0/2065154316 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd800c180 con 0x7effd8004870 2024-09-13T18:52:00.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff13b2700 1 -- 172.21.15.7:0/2065154316 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd800c340 con 0x7effd8003a20 2024-09-13T18:52:00.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff23b4700 1 --2- 172.21.15.7:0/2065154316 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effd8003a20 0x7effd8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:00.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff1bb3700 1 --2- 172.21.15.7:0/2065154316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effd8004870 0x7effd8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:00.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff2bb5700 1 --2- 172.21.15.7:0/2065154316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effd8005230 0x7effd800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:00.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff2bb5700 1 -- 172.21.15.7:0/2065154316 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effd8003a20 msgr2=0x7effd8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:00.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff2bb5700 1 --2- 172.21.15.7:0/2065154316 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effd8003a20 0x7effd8003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:00.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff2bb5700 1 -- 172.21.15.7:0/2065154316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effd8004870 msgr2=0x7effd8004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:00.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff2bb5700 1 --2- 172.21.15.7:0/2065154316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effd8004870 0x7effd8004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:00.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff2bb5700 1 -- 172.21.15.7:0/2065154316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd80035d0 con 0x7effd8005230 2024-09-13T18:52:00.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff1bb3700 1 --2- 172.21.15.7:0/2065154316 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effd8004870 0x7effd8004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:00.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff2bb5700 1 -- 172.21.15.7:0/2065154316 reap_dead start 2024-09-13T18:52:00.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.666+0000 7efff2bb5700 1 --2- 172.21.15.7:0/2065154316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effd8005230 0x7effd800ba30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7effe400a860 tx=0x7effd80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9427eb24c0e04b36 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:00.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.670+0000 7efff13b2700 1 -- 172.21.15.7:0/2065154316 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe400a320 con 0x7effd8005230 2024-09-13T18:52:00.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.670+0000 7efff13b2700 1 -- 172.21.15.7:0/2065154316 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe4012660 con 0x7effd8005230 2024-09-13T18:52:00.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.670+0000 7efff13b2700 1 -- 172.21.15.7:0/2065154316 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe4012a10 con 0x7effd8005230 2024-09-13T18:52:00.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.670+0000 7efff4618700 1 -- 172.21.15.7:0/2065154316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effd8005230 msgr2=0x7effd800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:00.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.670+0000 7efff4618700 1 --2- 172.21.15.7:0/2065154316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effd8005230 0x7effd800ba30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7effe400a860 tx=0x7effd80058a0 comp rx=0 tx=0).stop 2024-09-13T18:52:00.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.670+0000 7efff4618700 1 -- 172.21.15.7:0/2065154316 shutdown_connections 2024-09-13T18:52:00.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.670+0000 7efff4618700 1 --2- 172.21.15.7:0/2065154316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effd8005230 0x7effd800ba30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:00.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.670+0000 7efff4618700 1 -- 172.21.15.7:0/2065154316 >> 172.21.15.7:0/2065154316 conn(0x7effec06d900 msgr2=0x7effec06dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:00.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.674+0000 7efff4618700 1 -- 172.21.15.7:0/2065154316 shutdown_connections 2024-09-13T18:52:00.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.674+0000 7efff4618700 1 -- 172.21.15.7:0/2065154316 wait complete. 2024-09-13T18:52:00.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.674+0000 7efff4618700 1 Processor -- start 2024-09-13T18:52:00.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.674+0000 7efff4618700 1 -- start start 2024-09-13T18:52:00.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.674+0000 7efff4618700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effd8005230 0x7effec14f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:00.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.674+0000 7efff4618700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effec143920 0x7effec143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:00.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.674+0000 7efff4618700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effec145950 0x7effec145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:00.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.674+0000 7efff4618700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec1443e0 con 0x7effec145950 2024-09-13T18:52:00.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.674+0000 7efff4618700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec1446c0 con 0x7effd8005230 2024-09-13T18:52:00.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.674+0000 7efff4618700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effec1449a0 con 0x7effec143920 2024-09-13T18:52:00.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.674+0000 7efff2bb5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effec145950 0x7effec145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:00.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.674+0000 7efff2bb5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effec145950 0x7effec145de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50218/0 (socket says 172.21.15.7:50218) 2024-09-13T18:52:00.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.674+0000 7efff2bb5700 1 -- 172.21.15.7:0/2505761726 learned_addr learned my addr 172.21.15.7:0/2505761726 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:00.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.674+0000 7efff1bb3700 1 --2- 172.21.15.7:0/2505761726 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effec143920 0x7effec143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:00.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.678+0000 7efff23b4700 1 --2- 172.21.15.7:0/2505761726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effd8005230 0x7effec14f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:00.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.678+0000 7efff23b4700 1 -- 172.21.15.7:0/2505761726 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effec143920 msgr2=0x7effec143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:00.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.678+0000 7efff23b4700 1 --2- 172.21.15.7:0/2505761726 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effec143920 0x7effec143da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:00.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.678+0000 7efff23b4700 1 -- 172.21.15.7:0/2505761726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effec145950 msgr2=0x7effec145de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:00.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.678+0000 7efff23b4700 1 --2- 172.21.15.7:0/2505761726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effec145950 0x7effec145de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:00.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.678+0000 7efff23b4700 1 -- 172.21.15.7:0/2505761726 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effec144c80 con 0x7effd8005230 2024-09-13T18:52:00.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.678+0000 7efff1bb3700 1 --2- 172.21.15.7:0/2505761726 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effec143920 0x7effec143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:00.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.678+0000 7efff23b4700 1 --2- 172.21.15.7:0/2505761726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effd8005230 0x7effec14f020 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7effe800a010 tx=0x7effe8011840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:00.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.678+0000 7effe2ffd700 1 -- 172.21.15.7:0/2505761726 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe800c070 con 0x7effd8005230 2024-09-13T18:52:00.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.678+0000 7effe2ffd700 1 -- 172.21.15.7:0/2505761726 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe8015870 con 0x7effd8005230 2024-09-13T18:52:00.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.678+0000 7efff4618700 1 -- 172.21.15.7:0/2505761726 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effec144f60 con 0x7effd8005230 2024-09-13T18:52:00.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.678+0000 7effe2ffd700 1 -- 172.21.15.7:0/2505761726 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8015c00 con 0x7effd8005230 2024-09-13T18:52:00.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.678+0000 7efff4618700 1 -- 172.21.15.7:0/2505761726 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effec14f560 con 0x7effd8005230 2024-09-13T18:52:00.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.678+0000 7effe2ffd700 1 -- 172.21.15.7:0/2505761726 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7effe801e450 con 0x7effd8005230 2024-09-13T18:52:00.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.678+0000 7effe2ffd700 1 -- 172.21.15.7:0/2505761726 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7effe8014720 con 0x7effd8005230 2024-09-13T18:52:00.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.678+0000 7efff4618700 1 -- 172.21.15.7:0/2505761726 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effec071680 con 0x7effd8005230 2024-09-13T18:52:00.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.682+0000 7effe2ffd700 1 -- 172.21.15.7:0/2505761726 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7effe8023060 con 0x7effd8005230 2024-09-13T18:52:00.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.770+0000 7efff4618700 1 -- 172.21.15.7:0/2505761726 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7effec1450f0 con 0x7effd8005230 2024-09-13T18:52:00.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.774+0000 7effe2ffd700 1 -- 172.21.15.7:0/2505761726 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7effe8019070 con 0x7effd8005230 2024-09-13T18:52:00.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.778+0000 7efff4618700 1 -- 172.21.15.7:0/2505761726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effd8005230 msgr2=0x7effec14f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:00.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.778+0000 7efff4618700 1 --2- 172.21.15.7:0/2505761726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effd8005230 0x7effec14f020 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7effe800a010 tx=0x7effe8011840 comp rx=0 tx=0).stop 2024-09-13T18:52:00.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.778+0000 7efff4618700 1 -- 172.21.15.7:0/2505761726 shutdown_connections 2024-09-13T18:52:00.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.778+0000 7efff4618700 1 --2- 172.21.15.7:0/2505761726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effec145950 0x7effec145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:00.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.778+0000 7efff4618700 1 --2- 172.21.15.7:0/2505761726 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effec143920 0x7effec143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:00.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.778+0000 7efff4618700 1 --2- 172.21.15.7:0/2505761726 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effd8005230 0x7effec14f020 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:00.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.778+0000 7efff4618700 1 -- 172.21.15.7:0/2505761726 >> 172.21.15.7:0/2505761726 conn(0x7effec06d900 msgr2=0x7effec0b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:00.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.778+0000 7efff4618700 1 -- 172.21.15.7:0/2505761726 shutdown_connections 2024-09-13T18:52:00.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:00.778+0000 7efff4618700 1 -- 172.21.15.7:0/2505761726 wait complete. 2024-09-13T18:52:00.783 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-09-13T18:52:00.794 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T18:52:00.794 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T18:52:00.795 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:52:00.795 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T18:52:00.808 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:52:00.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T18:52:00.817 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:52:00.817 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T18:52:00.827 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:52:00.827 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T18:52:00.839 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:52:00.840 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-13T18:52:00.855 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:52:00.855 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-13T18:52:00.887 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:00.882+0000 7fed851d82c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T18:52:00.911 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:00.906+0000 7f197de502c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T18:52:00.912 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:52:00.913 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-13T18:52:00.923 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:52:00.923 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-13T18:52:00.933 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new 9c224cb7-67e0-431d-84ca-c42b02ffa864 0 2024-09-13T18:52:01.092 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4302585700 1 Processor -- start 2024-09-13T18:52:01.092 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4302585700 1 -- start start 2024-09-13T18:52:01.092 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4302585700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42fc15cda0 0x7f42fc06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4302585700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42fc0b8430 con 0x7f42fc0ceb70 2024-09-13T18:52:01.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4302585700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42fc0cf350 con 0x7f42fc06a520 2024-09-13T18:52:01.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4302585700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42fc06be40 con 0x7f42fc15cda0 2024-09-13T18:52:01.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4301d84700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f42fc06a520 0x7f42fc06a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.171:46362/0 (socket says 172.21.15.171:46362) 2024-09-13T18:52:01.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4301d84700 1 -- 172.21.15.171:0/4227878439 learned_addr learned my addr 172.21.15.171:0/4227878439 (peer_addr_for_me v1:172.21.15.171:0/0) 2024-09-13T18:52:01.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4301583700 1 --2- 172.21.15.171:0/4227878439 >> v2:172.21.15.7:3300/0 conn(0x7f42fc15cda0 0x7f42fc06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 -- 172.21.15.171:0/4227878439 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 203663077 0 0) 0x7f42fc0cf350 con 0x7f42fc06a520 2024-09-13T18:52:01.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 -- 172.21.15.171:0/4227878439 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42e80035d0 con 0x7f42fc06a520 2024-09-13T18:52:01.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 -- 172.21.15.171:0/4227878439 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3219846214 0 0) 0x7f42fc0b8430 con 0x7f42fc0ceb70 2024-09-13T18:52:01.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 -- 172.21.15.171:0/4227878439 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42fc0cf350 con 0x7f42fc0ceb70 2024-09-13T18:52:01.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 -- 172.21.15.171:0/4227878439 <== mon.1 v1:172.21.15.171:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3143883609 0 0) 0x7f42e80035d0 con 0x7f42fc06a520 2024-09-13T18:52:01.093 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 -- 172.21.15.171:0/4227878439 >> v2:172.21.15.7:3300/0 conn(0x7f42fc15cda0 msgr2=0x7f42fc06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:01.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 --2- 172.21.15.171:0/4227878439 >> v2:172.21.15.7:3300/0 conn(0x7f42fc15cda0 0x7f42fc06b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 -- 172.21.15.171:0/4227878439 >> v1:172.21.15.7:6789/0 conn(0x7f42fc0ceb70 legacy=0x7f42fc0b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 -- 172.21.15.171:0/4227878439 --> v1:172.21.15.171:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42fc0d0880 con 0x7f42fc06a520 2024-09-13T18:52:01.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 -- 172.21.15.171:0/4227878439 <== mon.1 v1:172.21.15.171:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7f42f40019f0 con 0x7f42fc06a520 2024-09-13T18:52:01.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 -- 172.21.15.171:0/4227878439 >> v1:172.21.15.171:6789/0 conn(0x7f42fc06a520 legacy=0x7f42fc06a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 --2- 172.21.15.171:0/4227878439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e8003bb0 0x7f42e8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 --2- 172.21.15.171:0/4227878439 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f42e8004870 0x7f42e8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 --2- 172.21.15.171:0/4227878439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42e8005230 0x7f42e800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 -- 172.21.15.171:0/4227878439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e800c080 con 0x7f42e8003bb0 2024-09-13T18:52:01.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 -- 172.21.15.171:0/4227878439 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e800c1f0 con 0x7f42e8005230 2024-09-13T18:52:01.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 -- 172.21.15.171:0/4227878439 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e800c3b0 con 0x7f42e8004870 2024-09-13T18:52:01.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4301d84700 1 --2- 172.21.15.171:0/4227878439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42e8005230 0x7f42e800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.094 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4300d82700 1 --2- 172.21.15.171:0/4227878439 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f42e8004870 0x7f42e8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4301583700 1 --2- 172.21.15.171:0/4227878439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e8003bb0 0x7f42e8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4301d84700 1 -- 172.21.15.171:0/4227878439 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f42e8004870 msgr2=0x7f42e8004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:01.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4301d84700 1 --2- 172.21.15.171:0/4227878439 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f42e8004870 0x7f42e8004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4301d84700 1 -- 172.21.15.171:0/4227878439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e8003bb0 msgr2=0x7f42e8003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:01.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4301d84700 1 --2- 172.21.15.171:0/4227878439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e8003bb0 0x7f42e8003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4301d84700 1 -- 172.21.15.171:0/4227878439 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e80035d0 con 0x7f42e8005230 2024-09-13T18:52:01.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4301d84700 1 -- 172.21.15.171:0/4227878439 reap_dead start 2024-09-13T18:52:01.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4301d84700 1 --2- 172.21.15.171:0/4227878439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42e8005230 0x7f42e800ba30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f42f400a7a0 tx=0x7f42e80058a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=170259c4da9f638 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:01.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 -- 172.21.15.171:0/4227878439 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f40072d0 con 0x7f42e8005230 2024-09-13T18:52:01.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 -- 172.21.15.171:0/4227878439 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f4007450 con 0x7f42e8005230 2024-09-13T18:52:01.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f42f3fff700 1 -- 172.21.15.171:0/4227878439 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f4007800 con 0x7f42e8005230 2024-09-13T18:52:01.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4302585700 1 -- 172.21.15.171:0/4227878439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42e8005230 msgr2=0x7f42e800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.095 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4302585700 1 --2- 172.21.15.171:0/4227878439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42e8005230 0x7f42e800ba30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f42f400a7a0 tx=0x7f42e80058a0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.096 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4302585700 1 -- 172.21.15.171:0/4227878439 shutdown_connections 2024-09-13T18:52:01.096 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4302585700 1 --2- 172.21.15.171:0/4227878439 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42e8005230 0x7f42e800ba30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.096 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.086+0000 7f4302585700 1 -- 172.21.15.171:0/4227878439 >> 172.21.15.171:0/4227878439 conn(0x7f42fc0d2cb0 msgr2=0x7f42fc0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:01.096 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4302585700 1 -- 172.21.15.171:0/4227878439 shutdown_connections 2024-09-13T18:52:01.096 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4302585700 1 -- 172.21.15.171:0/4227878439 wait complete. 2024-09-13T18:52:01.098 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4302585700 1 Processor -- start 2024-09-13T18:52:01.098 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4302585700 1 -- start start 2024-09-13T18:52:01.098 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4302585700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f42e8005230 0x7f42fc150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.098 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4302585700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42fc1f5d20 0x7f42fc1511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.098 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4302585700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42fc156bd0 0x7f42fc157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.098 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4302585700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc157950 con 0x7f42fc156bd0 2024-09-13T18:52:01.098 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4302585700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc0c7800 con 0x7f42fc1f5d20 2024-09-13T18:52:01.099 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4302585700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc0c7a50 con 0x7f42e8005230 2024-09-13T18:52:01.099 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4301583700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f42e8005230 0x7f42fc150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.099 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4301583700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f42e8005230 0x7f42fc150c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.171:36330/0 (socket says 172.21.15.171:36330) 2024-09-13T18:52:01.099 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4300d82700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42fc1f5d20 0x7f42fc1511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.099 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4301583700 1 -- 172.21.15.171:0/1649158581 learned_addr learned my addr 172.21.15.171:0/1649158581 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-13T18:52:01.099 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4301d84700 1 --2- 172.21.15.171:0/1649158581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42fc156bd0 0x7f42fc157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.099 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4301583700 1 -- 172.21.15.171:0/1649158581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42fc1f5d20 msgr2=0x7f42fc1511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.099 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4301583700 1 --2- 172.21.15.171:0/1649158581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42fc1f5d20 0x7f42fc1511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.099 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4301583700 1 -- 172.21.15.171:0/1649158581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42fc156bd0 msgr2=0x7f42fc157050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.099 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4301583700 1 --2- 172.21.15.171:0/1649158581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42fc156bd0 0x7f42fc157050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.099 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4301583700 1 -- 172.21.15.171:0/1649158581 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42fc0c7d30 con 0x7f42e8005230 2024-09-13T18:52:01.099 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4301d84700 1 --2- 172.21.15.171:0/1649158581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42fc156bd0 0x7f42fc157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:01.099 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f4301583700 1 --2- 172.21.15.171:0/1649158581 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f42e8005230 0x7f42fc150c70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42f80119e0 tx=0x7f42f8011da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:01.100 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.090+0000 7f42f1ffb700 1 -- 172.21.15.171:0/1649158581 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f800f940 con 0x7f42e8005230 2024-09-13T18:52:01.100 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.094+0000 7f42f1ffb700 1 -- 172.21.15.171:0/1649158581 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f800faa0 con 0x7f42e8005230 2024-09-13T18:52:01.100 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.094+0000 7f4302585700 1 -- 172.21.15.171:0/1649158581 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42fc0c8010 con 0x7f42e8005230 2024-09-13T18:52:01.100 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.094+0000 7f42f1ffb700 1 -- 172.21.15.171:0/1649158581 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f800fe30 con 0x7f42e8005230 2024-09-13T18:52:01.100 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.094+0000 7f4302585700 1 -- 172.21.15.171:0/1649158581 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42fc0c8810 con 0x7f42e8005230 2024-09-13T18:52:01.100 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.094+0000 7f42f1ffb700 1 -- 172.21.15.171:0/1649158581 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f42f8014740 con 0x7f42e8005230 2024-09-13T18:52:01.101 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.094+0000 7f42f1ffb700 1 -- 172.21.15.171:0/1649158581 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f42f801da40 con 0x7f42e8005230 2024-09-13T18:52:01.101 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.094+0000 7f4302585700 1 -- 172.21.15.171:0/1649158581 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42fc06b900 con 0x7f42e8005230 2024-09-13T18:52:01.102 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.098+0000 7f42f1ffb700 1 -- 172.21.15.171:0/1649158581 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f42f801dc00 con 0x7f42e8005230 2024-09-13T18:52:01.179 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.174+0000 7f4302585700 1 -- 172.21.15.171:0/1649158581 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9c224cb7-67e0-431d-84ca-c42b02ffa864", "id": 0} v 0) v1 -- 0x7f42fc14bd90 con 0x7f42e8005230 2024-09-13T18:52:01.186 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.182+0000 7f42f1ffb700 1 -- 172.21.15.171:0/1649158581 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9c224cb7-67e0-431d-84ca-c42b02ffa864", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f42f801dda0 con 0x7f42e8005230 2024-09-13T18:52:01.186 INFO:teuthology.orchestra.run.smithi171.stdout:0 2024-09-13T18:52:01.188 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.182+0000 7f4302585700 1 -- 172.21.15.171:0/1649158581 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f42e8005230 msgr2=0x7f42fc150c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.188 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.182+0000 7f4302585700 1 --2- 172.21.15.171:0/1649158581 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f42e8005230 0x7f42fc150c70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f42f80119e0 tx=0x7f42f8011da0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.188 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.182+0000 7f4302585700 1 -- 172.21.15.171:0/1649158581 shutdown_connections 2024-09-13T18:52:01.188 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.182+0000 7f4302585700 1 --2- 172.21.15.171:0/1649158581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42fc156bd0 0x7f42fc157050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.188 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.182+0000 7f4302585700 1 --2- 172.21.15.171:0/1649158581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f42fc1f5d20 0x7f42fc1511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.188 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.182+0000 7f4302585700 1 --2- 172.21.15.171:0/1649158581 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f42e8005230 0x7f42fc150c70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.188 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.182+0000 7f4302585700 1 -- 172.21.15.171:0/1649158581 >> 172.21.15.171:0/1649158581 conn(0x7f42fc0d2cb0 msgr2=0x7f42fc0d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:01.188 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.182+0000 7f4302585700 1 -- 172.21.15.171:0/1649158581 shutdown_connections 2024-09-13T18:52:01.189 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.182+0000 7f4302585700 1 -- 172.21.15.171:0/1649158581 wait complete. 2024-09-13T18:52:01.201 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new d638fbca-bf68-46c2-8b13-1e701b58fdc2 1 2024-09-13T18:52:01.316 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.310+0000 7fcbddea8700 1 Processor -- start 2024-09-13T18:52:01.316 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.310+0000 7fcbddea8700 1 -- start start 2024-09-13T18:52:01.318 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbddea8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcbd80b4170 0x7fcbd80cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.318 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbddea8700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fcbd80b4550 0x7fcbd80b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.318 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbddea8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd80ac960 con 0x7fcbd8071a90 2024-09-13T18:52:01.318 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbddea8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd8072630 con 0x7fcbd80b4170 2024-09-13T18:52:01.318 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbddea8700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd815cf50 con 0x7fcbd80b4550 2024-09-13T18:52:01.318 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbd7fff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fcbd80b4550 0x7fcbd80b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.318 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbd7fff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fcbd80b4550 0x7fcbd80b4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:55372/0 (socket says 172.21.15.171:55372) 2024-09-13T18:52:01.318 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbd7fff700 1 -- 172.21.15.171:0/769938818 learned_addr learned my addr 172.21.15.171:0/769938818 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-13T18:52:01.318 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbdcea6700 1 --2- 172.21.15.171:0/769938818 >> v2:172.21.15.7:3300/0 conn(0x7fcbd80b4170 0x7fcbd80cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbd7fff700 1 -- 172.21.15.171:0/769938818 >> v2:172.21.15.7:3300/0 conn(0x7fcbd80b4170 msgr2=0x7fcbd80cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:01.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbd7fff700 1 --2- 172.21.15.171:0/769938818 >> v2:172.21.15.7:3300/0 conn(0x7fcbd80b4170 0x7fcbd80cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbd7fff700 1 -- 172.21.15.171:0/769938818 >> v1:172.21.15.7:6789/0 conn(0x7fcbd8071a90 legacy=0x7fcbd8071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbd7fff700 1 -- 172.21.15.171:0/769938818 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd80acc00 con 0x7fcbd80b4550 2024-09-13T18:52:01.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbd7fff700 1 --2- 172.21.15.171:0/769938818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbd80b4550 0x7fcbd80b4990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8072a00 tx=0x7fcbcc009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2016e47c7d8a240e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:01.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbd77fe700 1 -- 172.21.15.171:0/769938818 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc00b8a0 con 0x7fcbd80b4550 2024-09-13T18:52:01.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbd77fe700 1 -- 172.21.15.171:0/769938818 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbcc00ba00 con 0x7fcbd80b4550 2024-09-13T18:52:01.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbd77fe700 1 -- 172.21.15.171:0/769938818 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc00bd90 con 0x7fcbd80b4550 2024-09-13T18:52:01.320 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbddea8700 1 -- 172.21.15.171:0/769938818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbd80b4550 msgr2=0x7fcbd80b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.320 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbddea8700 1 --2- 172.21.15.171:0/769938818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbd80b4550 0x7fcbd80b4990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8072a00 tx=0x7fcbcc009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.320 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbddea8700 1 -- 172.21.15.171:0/769938818 shutdown_connections 2024-09-13T18:52:01.320 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbddea8700 1 --2- 172.21.15.171:0/769938818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbd80b4550 0x7fcbd80b4990 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.320 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbddea8700 1 --2- 172.21.15.171:0/769938818 >> v2:172.21.15.7:3300/0 conn(0x7fcbd80b4170 0x7fcbd80cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.320 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbddea8700 1 -- 172.21.15.171:0/769938818 >> 172.21.15.171:0/769938818 conn(0x7fcbd80cfad0 msgr2=0x7fcbd80cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:01.320 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbddea8700 1 -- 172.21.15.171:0/769938818 shutdown_connections 2024-09-13T18:52:01.320 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.314+0000 7fcbddea8700 1 -- 172.21.15.171:0/769938818 wait complete. 2024-09-13T18:52:01.321 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbddea8700 1 Processor -- start 2024-09-13T18:52:01.321 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbddea8700 1 -- start start 2024-09-13T18:52:01.323 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbddea8700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcbd8071a90 0x7fcbd81e5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.323 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbddea8700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbd80b4170 0x7fcbd815bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.324 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbddea8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd80b4550 0x7fcbd815c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.324 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbddea8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd806d950 con 0x7fcbd80b4550 2024-09-13T18:52:01.324 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbddea8700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd806cd00 con 0x7fcbd80b4170 2024-09-13T18:52:01.324 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbddea8700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd806cfb0 con 0x7fcbd8071a90 2024-09-13T18:52:01.324 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbdcea6700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcbd8071a90 0x7fcbd81e5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.324 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbdcea6700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcbd8071a90 0x7fcbd81e5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.171:36334/0 (socket says 172.21.15.171:36334) 2024-09-13T18:52:01.324 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbdcea6700 1 -- 172.21.15.171:0/3961903640 learned_addr learned my addr 172.21.15.171:0/3961903640 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-13T18:52:01.324 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbd7fff700 1 --2- 172.21.15.171:0/3961903640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbd80b4170 0x7fcbd815bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.324 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbdd6a7700 1 --2- 172.21.15.171:0/3961903640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd80b4550 0x7fcbd815c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.324 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbdcea6700 1 -- 172.21.15.171:0/3961903640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbd80b4170 msgr2=0x7fcbd815bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.324 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbdcea6700 1 --2- 172.21.15.171:0/3961903640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbd80b4170 0x7fcbd815bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.324 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbdcea6700 1 -- 172.21.15.171:0/3961903640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd80b4550 msgr2=0x7fcbd815c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.324 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbdcea6700 1 --2- 172.21.15.171:0/3961903640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd80b4550 0x7fcbd815c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.325 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbdcea6700 1 -- 172.21.15.171:0/3961903640 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc009570 con 0x7fcbd8071a90 2024-09-13T18:52:01.325 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbdcea6700 1 --2- 172.21.15.171:0/3961903640 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcbd8071a90 0x7fcbd81e5ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0007e60 tx=0x7fcbd000e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:01.325 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbd57fa700 1 -- 172.21.15.171:0/3961903640 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd000ed30 con 0x7fcbd8071a90 2024-09-13T18:52:01.325 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbd57fa700 1 -- 172.21.15.171:0/3961903640 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd000ee90 con 0x7fcbd8071a90 2024-09-13T18:52:01.325 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbddea8700 1 -- 172.21.15.171:0/3961903640 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd806d290 con 0x7fcbd8071a90 2024-09-13T18:52:01.325 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbd57fa700 1 -- 172.21.15.171:0/3961903640 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd0014620 con 0x7fcbd8071a90 2024-09-13T18:52:01.325 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbddea8700 1 -- 172.21.15.171:0/3961903640 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd81e6220 con 0x7fcbd8071a90 2024-09-13T18:52:01.325 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbd57fa700 1 -- 172.21.15.171:0/3961903640 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fcbd0014a80 con 0x7fcbd8071a90 2024-09-13T18:52:01.325 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.318+0000 7fcbd57fa700 1 -- 172.21.15.171:0/3961903640 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fcbd001ec80 con 0x7fcbd8071a90 2024-09-13T18:52:01.325 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.322+0000 7fcbddea8700 1 -- 172.21.15.171:0/3961903640 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd806da90 con 0x7fcbd8071a90 2024-09-13T18:52:01.326 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.322+0000 7fcbd57fa700 1 -- 172.21.15.171:0/3961903640 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fcbd001e460 con 0x7fcbd8071a90 2024-09-13T18:52:01.405 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.402+0000 7fcbddea8700 1 -- 172.21.15.171:0/3961903640 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d638fbca-bf68-46c2-8b13-1e701b58fdc2", "id": 1} v 0) v1 -- 0x7fcbd80cc940 con 0x7fcbd8071a90 2024-09-13T18:52:01.412 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.406+0000 7fcbd57fa700 1 -- 172.21.15.171:0/3961903640 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d638fbca-bf68-46c2-8b13-1e701b58fdc2", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcbd001e600 con 0x7fcbd8071a90 2024-09-13T18:52:01.412 INFO:teuthology.orchestra.run.smithi171.stdout:1 2024-09-13T18:52:01.414 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.410+0000 7fcbddea8700 1 -- 172.21.15.171:0/3961903640 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcbd8071a90 msgr2=0x7fcbd81e5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.414 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.410+0000 7fcbddea8700 1 --2- 172.21.15.171:0/3961903640 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcbd8071a90 0x7fcbd81e5ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0007e60 tx=0x7fcbd000e3c0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.414 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.410+0000 7fcbddea8700 1 -- 172.21.15.171:0/3961903640 shutdown_connections 2024-09-13T18:52:01.414 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.410+0000 7fcbddea8700 1 --2- 172.21.15.171:0/3961903640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbd80b4550 0x7fcbd815c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.414 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.410+0000 7fcbddea8700 1 --2- 172.21.15.171:0/3961903640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcbd80b4170 0x7fcbd815bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.414 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.410+0000 7fcbddea8700 1 --2- 172.21.15.171:0/3961903640 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcbd8071a90 0x7fcbd81e5ce0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.414 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.410+0000 7fcbddea8700 1 -- 172.21.15.171:0/3961903640 >> 172.21.15.171:0/3961903640 conn(0x7fcbd80cfad0 msgr2=0x7fcbd80cfeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:01.414 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.410+0000 7fcbddea8700 1 -- 172.21.15.171:0/3961903640 shutdown_connections 2024-09-13T18:52:01.415 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.410+0000 7fcbddea8700 1 -- 172.21.15.171:0/3961903640 wait complete. 2024-09-13T18:52:01.426 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new 36f14e49-0413-40df-9769-f1129a36003e 2 2024-09-13T18:52:01.543 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.538+0000 7f0f12040700 1 Processor -- start 2024-09-13T18:52:01.544 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.538+0000 7f0f12040700 1 -- start start 2024-09-13T18:52:01.545 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f12040700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f0c15a800 0x7f0f0c15abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.546 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f12040700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f0c0b3290 0x7f0f0c15c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.546 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f12040700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f0c15b2e0 con 0x7f0f0c06f8c0 2024-09-13T18:52:01.546 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f12040700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0c15bf60 con 0x7f0f0c15a800 2024-09-13T18:52:01.546 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f12040700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c15b5c0 con 0x7f0f0c0b3290 2024-09-13T18:52:01.546 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f1083d700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f0f0c06f8c0 0x7f0f0c0b2b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.171:46372/0 (socket says 172.21.15.171:46372) 2024-09-13T18:52:01.546 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f1083d700 1 -- 172.21.15.171:0/1005573736 learned_addr learned my addr 172.21.15.171:0/1005573736 (peer_addr_for_me v1:172.21.15.171:0/0) 2024-09-13T18:52:01.546 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f1183f700 1 --2- 172.21.15.171:0/1005573736 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f0c0b3290 0x7f0f0c15c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.546 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f1103e700 1 --2- 172.21.15.171:0/1005573736 >> v2:172.21.15.7:3300/0 conn(0x7f0f0c15a800 0x7f0f0c15abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.546 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f03fff700 1 -- 172.21.15.171:0/1005573736 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 417448585 0 0) 0x7f0f0c15b2e0 con 0x7f0f0c06f8c0 2024-09-13T18:52:01.546 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f03fff700 1 -- 172.21.15.171:0/1005573736 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ef80035d0 con 0x7f0f0c06f8c0 2024-09-13T18:52:01.546 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f1183f700 1 -- 172.21.15.171:0/1005573736 >> v2:172.21.15.7:3300/0 conn(0x7f0f0c15a800 msgr2=0x7f0f0c15abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:01.546 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f1183f700 1 --2- 172.21.15.171:0/1005573736 >> v2:172.21.15.7:3300/0 conn(0x7f0f0c15a800 0x7f0f0c15abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.547 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f1183f700 1 -- 172.21.15.171:0/1005573736 >> v1:172.21.15.171:6789/0 conn(0x7f0f0c06f8c0 legacy=0x7f0f0c0b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.547 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f1183f700 1 -- 172.21.15.171:0/1005573736 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c15b8a0 con 0x7f0f0c0b3290 2024-09-13T18:52:01.547 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f1183f700 1 --2- 172.21.15.171:0/1005573736 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f0c0b3290 0x7f0f0c15c380 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009d90 tx=0x7f0f0400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46654674ac5549fb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:01.547 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f03fff700 1 -- 172.21.15.171:0/1005573736 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0400e070 con 0x7f0f0c0b3290 2024-09-13T18:52:01.547 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f03fff700 1 -- 172.21.15.171:0/1005573736 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0400bb20 con 0x7f0f0c0b3290 2024-09-13T18:52:01.547 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f03fff700 1 -- 172.21.15.171:0/1005573736 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04012430 con 0x7f0f0c0b3290 2024-09-13T18:52:01.548 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f12040700 1 -- 172.21.15.171:0/1005573736 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f0c0b3290 msgr2=0x7f0f0c15c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.548 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f12040700 1 --2- 172.21.15.171:0/1005573736 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f0c0b3290 0x7f0f0c15c380 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009d90 tx=0x7f0f0400b330 comp rx=0 tx=0).stop 2024-09-13T18:52:01.548 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f12040700 1 -- 172.21.15.171:0/1005573736 shutdown_connections 2024-09-13T18:52:01.548 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f12040700 1 --2- 172.21.15.171:0/1005573736 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f0c0b3290 0x7f0f0c15c380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.549 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f12040700 1 --2- 172.21.15.171:0/1005573736 >> v2:172.21.15.7:3300/0 conn(0x7f0f0c15a800 0x7f0f0c15abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.549 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f12040700 1 -- 172.21.15.171:0/1005573736 >> 172.21.15.171:0/1005573736 conn(0x7f0f0c071ce0 msgr2=0x7f0f0c0720c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:01.549 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f12040700 1 -- 172.21.15.171:0/1005573736 shutdown_connections 2024-09-13T18:52:01.549 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f12040700 1 -- 172.21.15.171:0/1005573736 wait complete. 2024-09-13T18:52:01.549 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f12040700 1 Processor -- start 2024-09-13T18:52:01.549 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.542+0000 7f0f12040700 1 -- start start 2024-09-13T18:52:01.551 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f12040700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f0c06f8c0 0x7f0f0c144610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.552 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f12040700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f0c0b3290 0x7f0f0c1f0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.552 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f12040700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f0c15a800 0x7f0f0c14ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.552 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f12040700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c14fb10 con 0x7f0f0c15a800 2024-09-13T18:52:01.552 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f12040700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c1f09a0 con 0x7f0f0c0b3290 2024-09-13T18:52:01.552 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f12040700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c156190 con 0x7f0f0c06f8c0 2024-09-13T18:52:01.552 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f1103e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f0c06f8c0 0x7f0f0c144610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.552 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f1103e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f0c06f8c0 0x7f0f0c144610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.171:36352/0 (socket says 172.21.15.171:36352) 2024-09-13T18:52:01.552 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f1103e700 1 -- 172.21.15.171:0/2679010238 learned_addr learned my addr 172.21.15.171:0/2679010238 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-13T18:52:01.552 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f1083d700 1 --2- 172.21.15.171:0/2679010238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f0c0b3290 0x7f0f0c1f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.552 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f1183f700 1 --2- 172.21.15.171:0/2679010238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f0c15a800 0x7f0f0c14ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.552 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f1103e700 1 -- 172.21.15.171:0/2679010238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f0c0b3290 msgr2=0x7f0f0c1f0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.552 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f1103e700 1 --2- 172.21.15.171:0/2679010238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f0c0b3290 0x7f0f0c1f0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.552 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f1103e700 1 -- 172.21.15.171:0/2679010238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f0c15a800 msgr2=0x7f0f0c14ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.553 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f1103e700 1 --2- 172.21.15.171:0/2679010238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f0c15a800 0x7f0f0c14ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.553 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f1103e700 1 -- 172.21.15.171:0/2679010238 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f04009a70 con 0x7f0f0c06f8c0 2024-09-13T18:52:01.553 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f1183f700 1 --2- 172.21.15.171:0/2679010238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f0c15a800 0x7f0f0c14ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:01.553 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f1103e700 1 --2- 172.21.15.171:0/2679010238 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f0c06f8c0 0x7f0f0c144610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800e840 tx=0x7f0f0800eb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:01.553 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f01ffb700 1 -- 172.21.15.171:0/2679010238 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f080095a0 con 0x7f0f0c06f8c0 2024-09-13T18:52:01.553 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f01ffb700 1 -- 172.21.15.171:0/2679010238 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f08013680 con 0x7f0f0c06f8c0 2024-09-13T18:52:01.553 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f12040700 1 -- 172.21.15.171:0/2679010238 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0c1f0bd0 con 0x7f0f0c06f8c0 2024-09-13T18:52:01.553 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f01ffb700 1 -- 172.21.15.171:0/2679010238 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f080139f0 con 0x7f0f0c06f8c0 2024-09-13T18:52:01.553 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f12040700 1 -- 172.21.15.171:0/2679010238 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0c1f1190 con 0x7f0f0c06f8c0 2024-09-13T18:52:01.553 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f01ffb700 1 -- 172.21.15.171:0/2679010238 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0f08005a40 con 0x7f0f0c06f8c0 2024-09-13T18:52:01.553 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f01ffb700 1 -- 172.21.15.171:0/2679010238 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f0f08006700 con 0x7f0f0c06f8c0 2024-09-13T18:52:01.553 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.546+0000 7f0f12040700 1 -- 172.21.15.171:0/2679010238 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f0c14fd80 con 0x7f0f0c06f8c0 2024-09-13T18:52:01.555 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.550+0000 7f0f01ffb700 1 -- 172.21.15.171:0/2679010238 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f0f08021060 con 0x7f0f0c06f8c0 2024-09-13T18:52:01.632 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.630+0000 7f0f12040700 1 -- 172.21.15.171:0/2679010238 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd new", "uuid": "36f14e49-0413-40df-9769-f1129a36003e", "id": 2} v 0) v1 -- 0x7f0f0c1f1380 con 0x7f0f0c06f8c0 2024-09-13T18:52:01.637 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:01.634+0000 7fed851d82c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T18:52:01.638 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.634+0000 7f0f01ffb700 1 -- 172.21.15.171:0/2679010238 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "36f14e49-0413-40df-9769-f1129a36003e", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0f08017070 con 0x7f0f0c06f8c0 2024-09-13T18:52:01.638 INFO:teuthology.orchestra.run.smithi171.stdout:2 2024-09-13T18:52:01.640 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.634+0000 7f0f12040700 1 -- 172.21.15.171:0/2679010238 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f0c06f8c0 msgr2=0x7f0f0c144610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.640 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.634+0000 7f0f12040700 1 --2- 172.21.15.171:0/2679010238 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f0c06f8c0 0x7f0f0c144610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800e840 tx=0x7f0f0800eb50 comp rx=0 tx=0).stop 2024-09-13T18:52:01.640 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.634+0000 7f0f12040700 1 -- 172.21.15.171:0/2679010238 shutdown_connections 2024-09-13T18:52:01.640 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.634+0000 7f0f12040700 1 --2- 172.21.15.171:0/2679010238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f0c15a800 0x7f0f0c14ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.640 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.634+0000 7f0f12040700 1 --2- 172.21.15.171:0/2679010238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f0c0b3290 0x7f0f0c1f0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.640 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.634+0000 7f0f12040700 1 --2- 172.21.15.171:0/2679010238 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f0c06f8c0 0x7f0f0c144610 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.640 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.634+0000 7f0f12040700 1 -- 172.21.15.171:0/2679010238 >> 172.21.15.171:0/2679010238 conn(0x7f0f0c071ce0 msgr2=0x7f0f0c15e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:01.641 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.634+0000 7f0f12040700 1 -- 172.21.15.171:0/2679010238 shutdown_connections 2024-09-13T18:52:01.641 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.634+0000 7f0f12040700 1 -- 172.21.15.171:0/2679010238 wait complete. 2024-09-13T18:52:01.650 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:01.646+0000 7f197de502c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T18:52:01.655 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new f0bcc77b-d80a-48f9-8596-a825c5f35b00 3 2024-09-13T18:52:01.698 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:01.694+0000 7fed851d82c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T18:52:01.710 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:01.706+0000 7f197de502c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T18:52:01.762 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.758+0000 7f0fb6474700 1 Processor -- start 2024-09-13T18:52:01.762 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.758+0000 7f0fb6474700 1 -- start start 2024-09-13T18:52:01.764 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.758+0000 7f0fb6474700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0fb006eb30 0x7f0fb006ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.764 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.758+0000 7f0fb6474700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb0072360 con 0x7f0fb0158e80 2024-09-13T18:52:01.764 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.758+0000 7f0fb6474700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb0071ac0 con 0x7f0fb0071430 2024-09-13T18:52:01.764 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.758+0000 7f0fb6474700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fb00718a0 con 0x7f0fb006eb30 2024-09-13T18:52:01.764 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.758+0000 7f0faf7fe700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f0fb0071430 0x7f0fb006e420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.171:46388/0 (socket says 172.21.15.171:46388) 2024-09-13T18:52:01.764 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.758+0000 7f0faf7fe700 1 -- 172.21.15.171:0/2631865894 learned_addr learned my addr 172.21.15.171:0/2631865894 (peer_addr_for_me v1:172.21.15.171:0/0) 2024-09-13T18:52:01.764 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.758+0000 7f0fb4a11700 1 --2- 172.21.15.171:0/2631865894 >> v2:172.21.15.7:3300/0 conn(0x7f0fb006eb30 0x7f0fb006ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.764 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.758+0000 7f0faeffd700 1 -- 172.21.15.171:0/2631865894 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3234662986 0 0) 0x7f0fb0071ac0 con 0x7f0fb0071430 2024-09-13T18:52:01.765 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.758+0000 7f0faeffd700 1 -- 172.21.15.171:0/2631865894 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f980035d0 con 0x7f0fb0071430 2024-09-13T18:52:01.765 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.762+0000 7f0faeffd700 1 -- 172.21.15.171:0/2631865894 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485390374 0 0) 0x7f0fb0072360 con 0x7f0fb0158e80 2024-09-13T18:52:01.765 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.762+0000 7f0faeffd700 1 -- 172.21.15.171:0/2631865894 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fb0071ac0 con 0x7f0fb0158e80 2024-09-13T18:52:01.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.762+0000 7f0faeffd700 1 -- 172.21.15.171:0/2631865894 <== mon.1 v1:172.21.15.171:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3486803762 0 0) 0x7f0f980035d0 con 0x7f0fb0071430 2024-09-13T18:52:01.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.762+0000 7f0fb4a11700 1 -- 172.21.15.171:0/2631865894 >> v1:172.21.15.171:6789/0 conn(0x7f0fb0071430 legacy=0x7f0fb006e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.762+0000 7f0fb4a11700 1 -- 172.21.15.171:0/2631865894 >> v1:172.21.15.7:6789/0 conn(0x7f0fb0158e80 legacy=0x7f0fb0159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.766 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.762+0000 7f0fb4a11700 1 -- 172.21.15.171:0/2631865894 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb0071da0 con 0x7f0fb006eb30 2024-09-13T18:52:01.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.762+0000 7f0fb4a11700 1 --2- 172.21.15.171:0/2631865894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fb006eb30 0x7f0fb006ad30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009d90 tx=0x7f0fa800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460eb2254569bc27 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:01.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.762+0000 7f0faeffd700 1 -- 172.21.15.171:0/2631865894 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa800e070 con 0x7f0fb006eb30 2024-09-13T18:52:01.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.762+0000 7f0faeffd700 1 -- 172.21.15.171:0/2631865894 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa8003a40 con 0x7f0fb006eb30 2024-09-13T18:52:01.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.762+0000 7f0faeffd700 1 -- 172.21.15.171:0/2631865894 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4001670 con 0x7f0fb006eb30 2024-09-13T18:52:01.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.762+0000 7f0fb6474700 1 -- 172.21.15.171:0/2631865894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fb006eb30 msgr2=0x7f0fb006ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.762+0000 7f0fb6474700 1 --2- 172.21.15.171:0/2631865894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fb006eb30 0x7f0fb006ad30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009d90 tx=0x7f0fa800b330 comp rx=0 tx=0).stop 2024-09-13T18:52:01.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.762+0000 7f0fb6474700 1 -- 172.21.15.171:0/2631865894 shutdown_connections 2024-09-13T18:52:01.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.762+0000 7f0fb6474700 1 --2- 172.21.15.171:0/2631865894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fb006eb30 0x7f0fb006ad30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.762+0000 7f0fb6474700 1 -- 172.21.15.171:0/2631865894 >> 172.21.15.171:0/2631865894 conn(0x7f0fb006d900 msgr2=0x7f0fb006dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:01.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.762+0000 7f0fb6474700 1 -- 172.21.15.171:0/2631865894 shutdown_connections 2024-09-13T18:52:01.767 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.762+0000 7f0fb6474700 1 -- 172.21.15.171:0/2631865894 wait complete. 2024-09-13T18:52:01.768 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0fb6474700 1 Processor -- start 2024-09-13T18:52:01.769 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0fb6474700 1 -- start start 2024-09-13T18:52:01.771 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0fb6474700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0fb006eb30 0x7f0fb014d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.771 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0fb6474700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0fb0071430 0x7f0fb014a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.772 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0fb6474700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fb0158e80 0x7f0fb0153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.772 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0fb6474700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb014d6e0 con 0x7f0fb0158e80 2024-09-13T18:52:01.772 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0fb6474700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb014b220 con 0x7f0fb0071430 2024-09-13T18:52:01.772 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0fb6474700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb015b540 con 0x7f0fb006eb30 2024-09-13T18:52:01.772 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0faffff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0fb006eb30 0x7f0fb014d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.772 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0faf7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0fb0071430 0x7f0fb014a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.772 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0faffff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0fb006eb30 0x7f0fb014d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.171:36368/0 (socket says 172.21.15.171:36368) 2024-09-13T18:52:01.772 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0faffff700 1 -- 172.21.15.171:0/2689939730 learned_addr learned my addr 172.21.15.171:0/2689939730 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-13T18:52:01.772 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0fb4a11700 1 --2- 172.21.15.171:0/2689939730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fb0158e80 0x7f0fb0153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.772 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0faffff700 1 -- 172.21.15.171:0/2689939730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0fb0071430 msgr2=0x7f0fb014a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.772 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0faffff700 1 --2- 172.21.15.171:0/2689939730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0fb0071430 0x7f0fb014a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.772 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0faffff700 1 -- 172.21.15.171:0/2689939730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fb0158e80 msgr2=0x7f0fb0153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.773 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0faffff700 1 --2- 172.21.15.171:0/2689939730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fb0158e80 0x7f0fb0153140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.773 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0faffff700 1 -- 172.21.15.171:0/2689939730 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa8009a70 con 0x7f0fb006eb30 2024-09-13T18:52:01.773 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0faf7fe700 1 --2- 172.21.15.171:0/2689939730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0fb0071430 0x7f0fb014a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:01.773 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0faffff700 1 --2- 172.21.15.171:0/2689939730 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0fb006eb30 0x7f0fb014d1a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000c490 tx=0x7f0fa000c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:01.773 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0facff9700 1 -- 172.21.15.171:0/2689939730 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa00052d0 con 0x7f0fb006eb30 2024-09-13T18:52:01.773 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0facff9700 1 -- 172.21.15.171:0/2689939730 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa0005450 con 0x7f0fb006eb30 2024-09-13T18:52:01.773 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0fb6474700 1 -- 172.21.15.171:0/2689939730 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb014de90 con 0x7f0fb006eb30 2024-09-13T18:52:01.773 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0facff9700 1 -- 172.21.15.171:0/2689939730 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa00038e0 con 0x7f0fb006eb30 2024-09-13T18:52:01.773 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0fb6474700 1 -- 172.21.15.171:0/2689939730 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb014dbb0 con 0x7f0fb006eb30 2024-09-13T18:52:01.773 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0facff9700 1 -- 172.21.15.171:0/2689939730 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0fa0003d40 con 0x7f0fb006eb30 2024-09-13T18:52:01.773 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.766+0000 7f0facff9700 1 -- 172.21.15.171:0/2689939730 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f0fa000ce40 con 0x7f0fb006eb30 2024-09-13T18:52:01.774 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.770+0000 7f0fb6474700 1 -- 172.21.15.171:0/2689939730 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb006a820 con 0x7f0fb006eb30 2024-09-13T18:52:01.776 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.774+0000 7f0facff9700 1 -- 172.21.15.171:0/2689939730 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f0fa0017070 con 0x7f0fb006eb30 2024-09-13T18:52:01.853 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.850+0000 7f0fb6474700 1 -- 172.21.15.171:0/2689939730 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f0bcc77b-d80a-48f9-8596-a825c5f35b00", "id": 3} v 0) v1 -- 0x7f0fb01554f0 con 0x7f0fb006eb30 2024-09-13T18:52:01.859 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.854+0000 7f0facff9700 1 -- 172.21.15.171:0/2689939730 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f0bcc77b-d80a-48f9-8596-a825c5f35b00", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0fa0011d60 con 0x7f0fb006eb30 2024-09-13T18:52:01.859 INFO:teuthology.orchestra.run.smithi171.stdout:3 2024-09-13T18:52:01.861 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.858+0000 7f0fb6474700 1 -- 172.21.15.171:0/2689939730 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0fb006eb30 msgr2=0x7f0fb014d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.861 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.858+0000 7f0fb6474700 1 --2- 172.21.15.171:0/2689939730 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0fb006eb30 0x7f0fb014d1a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000c490 tx=0x7f0fa000c7a0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.861 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.858+0000 7f0fb6474700 1 -- 172.21.15.171:0/2689939730 shutdown_connections 2024-09-13T18:52:01.861 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.858+0000 7f0fb6474700 1 --2- 172.21.15.171:0/2689939730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fb0158e80 0x7f0fb0153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.861 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.858+0000 7f0fb6474700 1 --2- 172.21.15.171:0/2689939730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0fb0071430 0x7f0fb014a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.861 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.858+0000 7f0fb6474700 1 --2- 172.21.15.171:0/2689939730 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0fb006eb30 0x7f0fb014d1a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.861 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.858+0000 7f0fb6474700 1 -- 172.21.15.171:0/2689939730 >> 172.21.15.171:0/2689939730 conn(0x7f0fb006d900 msgr2=0x7f0fb0156a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:01.862 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.858+0000 7f0fb6474700 1 -- 172.21.15.171:0/2689939730 shutdown_connections 2024-09-13T18:52:01.862 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.858+0000 7f0fb6474700 1 -- 172.21.15.171:0/2689939730 wait complete. 2024-09-13T18:52:01.875 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:01.870+0000 7fed851d82c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:52:01.876 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new 5cc4ed05-2413-4e36-83eb-119e303fed77 4 2024-09-13T18:52:01.886 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:01.882+0000 7f197de502c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:52:01.990 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.986+0000 7fdd6f199700 1 Processor -- start 2024-09-13T18:52:01.990 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.986+0000 7fdd6f199700 1 -- start start 2024-09-13T18:52:01.993 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.986+0000 7fdd6f199700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fdd68159080 0x7fdd680caa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.993 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.986+0000 7fdd6f199700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdd680ce2d0 0x7fdd680c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.993 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.986+0000 7fdd6f199700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6815d800 con 0x7fdd680cb0d0 2024-09-13T18:52:01.993 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.986+0000 7fdd6f199700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6815cc10 con 0x7fdd680ce2d0 2024-09-13T18:52:01.993 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.986+0000 7fdd6f199700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6815ced0 con 0x7fdd68159080 2024-09-13T18:52:01.993 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.986+0000 7fdd6e197700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fdd68159080 0x7fdd680caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.993 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.986+0000 7fdd6e197700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fdd68159080 0x7fdd680caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:55402/0 (socket says 172.21.15.171:55402) 2024-09-13T18:52:01.993 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.986+0000 7fdd6e197700 1 -- 172.21.15.171:0/959032903 learned_addr learned my addr 172.21.15.171:0/959032903 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-13T18:52:01.993 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.986+0000 7fdd6e998700 1 --2- 172.21.15.171:0/959032903 >> v2:172.21.15.7:3300/0 conn(0x7fdd680ce2d0 0x7fdd680c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.993 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6e197700 1 -- 172.21.15.171:0/959032903 >> v2:172.21.15.7:3300/0 conn(0x7fdd680ce2d0 msgr2=0x7fdd680c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:01.994 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6e197700 1 --2- 172.21.15.171:0/959032903 >> v2:172.21.15.7:3300/0 conn(0x7fdd680ce2d0 0x7fdd680c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.994 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6e197700 1 -- 172.21.15.171:0/959032903 >> v1:172.21.15.171:6789/0 conn(0x7fdd680cb0d0 legacy=0x7fdd680cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.994 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6e197700 1 -- 172.21.15.171:0/959032903 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6815d1b0 con 0x7fdd68159080 2024-09-13T18:52:01.994 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6e998700 1 --2- 172.21.15.171:0/959032903 >> v2:172.21.15.7:3300/0 conn(0x7fdd680ce2d0 0x7fdd680c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:01.994 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6e197700 1 --2- 172.21.15.171:0/959032903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd68159080 0x7fdd680caa00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdd68067160 tx=0x7fdd64009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6a86c2fdc0e4c5e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:01.994 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6d195700 1 -- 172.21.15.171:0/959032903 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6400b8a0 con 0x7fdd68159080 2024-09-13T18:52:01.994 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6d195700 1 -- 172.21.15.171:0/959032903 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6400ba00 con 0x7fdd68159080 2024-09-13T18:52:01.994 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6d195700 1 -- 172.21.15.171:0/959032903 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6400bd90 con 0x7fdd68159080 2024-09-13T18:52:01.995 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6f199700 1 -- 172.21.15.171:0/959032903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd68159080 msgr2=0x7fdd680caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.995 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6f199700 1 --2- 172.21.15.171:0/959032903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd68159080 0x7fdd680caa00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdd68067160 tx=0x7fdd64009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.995 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6f199700 1 -- 172.21.15.171:0/959032903 shutdown_connections 2024-09-13T18:52:01.995 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6f199700 1 --2- 172.21.15.171:0/959032903 >> v2:172.21.15.7:3300/0 conn(0x7fdd680ce2d0 0x7fdd680c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.995 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6f199700 1 --2- 172.21.15.171:0/959032903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd68159080 0x7fdd680caa00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.996 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6f199700 1 -- 172.21.15.171:0/959032903 >> 172.21.15.171:0/959032903 conn(0x7fdd680cba60 msgr2=0x7fdd680cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:01.996 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6f199700 1 -- 172.21.15.171:0/959032903 shutdown_connections 2024-09-13T18:52:01.996 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6f199700 1 -- 172.21.15.171:0/959032903 wait complete. 2024-09-13T18:52:01.996 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6f199700 1 Processor -- start 2024-09-13T18:52:01.996 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6f199700 1 -- start start 2024-09-13T18:52:01.998 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6f199700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd680cb0d0 0x7fdd68150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.998 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6f199700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdd680ce2d0 0x7fdd68157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.998 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6f199700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd68159080 0x7fdd68167360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:01.998 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6f199700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd681509a0 con 0x7fdd680cb0d0 2024-09-13T18:52:01.998 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6f199700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd680c4030 con 0x7fdd68159080 2024-09-13T18:52:01.998 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6f199700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6814fa80 con 0x7fdd680ce2d0 2024-09-13T18:52:01.998 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6d996700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdd680ce2d0 0x7fdd68157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.999 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6d996700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdd680ce2d0 0x7fdd68157e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.171:36372/0 (socket says 172.21.15.171:36372) 2024-09-13T18:52:01.999 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6d996700 1 -- 172.21.15.171:0/2862323437 learned_addr learned my addr 172.21.15.171:0/2862323437 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-13T18:52:01.999 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6e998700 1 --2- 172.21.15.171:0/2862323437 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd68159080 0x7fdd68167360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.999 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.990+0000 7fdd6e197700 1 --2- 172.21.15.171:0/2862323437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd680cb0d0 0x7fdd68150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:01.999 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.994+0000 7fdd6d996700 1 -- 172.21.15.171:0/2862323437 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd68159080 msgr2=0x7fdd68167360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.999 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.994+0000 7fdd6d996700 1 --2- 172.21.15.171:0/2862323437 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd68159080 0x7fdd68167360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.999 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.994+0000 7fdd6d996700 1 -- 172.21.15.171:0/2862323437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd680cb0d0 msgr2=0x7fdd68150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:01.999 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.994+0000 7fdd6d996700 1 --2- 172.21.15.171:0/2862323437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd680cb0d0 0x7fdd68150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:01.999 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.994+0000 7fdd6d996700 1 -- 172.21.15.171:0/2862323437 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd64009570 con 0x7fdd680ce2d0 2024-09-13T18:52:01.999 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.994+0000 7fdd6e998700 1 --2- 172.21.15.171:0/2862323437 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd68159080 0x7fdd68167360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:01.999 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.994+0000 7fdd6d996700 1 --2- 172.21.15.171:0/2862323437 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdd680ce2d0 0x7fdd68157e10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800c500 tx=0x7fdd5800c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:01.999 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.994+0000 7fdd5effd700 1 -- 172.21.15.171:0/2862323437 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd580052d0 con 0x7fdd680ce2d0 2024-09-13T18:52:01.999 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.994+0000 7fdd5effd700 1 -- 172.21.15.171:0/2862323437 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd58005430 con 0x7fdd680ce2d0 2024-09-13T18:52:01.999 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.994+0000 7fdd6f199700 1 -- 172.21.15.171:0/2862323437 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6814fd20 con 0x7fdd680ce2d0 2024-09-13T18:52:02.000 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.994+0000 7fdd5effd700 1 -- 172.21.15.171:0/2862323437 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd580055a0 con 0x7fdd680ce2d0 2024-09-13T18:52:02.000 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.994+0000 7fdd6f199700 1 -- 172.21.15.171:0/2862323437 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd681502e0 con 0x7fdd680ce2d0 2024-09-13T18:52:02.000 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.994+0000 7fdd5effd700 1 -- 172.21.15.171:0/2862323437 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fdd58003be0 con 0x7fdd680ce2d0 2024-09-13T18:52:02.000 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.994+0000 7fdd5effd700 1 -- 172.21.15.171:0/2862323437 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fdd58011d10 con 0x7fdd680ce2d0 2024-09-13T18:52:02.000 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.994+0000 7fdd6f199700 1 -- 172.21.15.171:0/2862323437 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd68151300 con 0x7fdd680ce2d0 2024-09-13T18:52:02.001 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:01.998+0000 7fdd5effd700 1 -- 172.21.15.171:0/2862323437 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fdd58017070 con 0x7fdd680ce2d0 2024-09-13T18:52:02.077 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.074+0000 7fdd6f199700 1 -- 172.21.15.171:0/2862323437 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5cc4ed05-2413-4e36-83eb-119e303fed77", "id": 4} v 0) v1 -- 0x7fdd68150ae0 con 0x7fdd680ce2d0 2024-09-13T18:52:02.082 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.078+0000 7fdd5effd700 1 -- 172.21.15.171:0/2862323437 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5cc4ed05-2413-4e36-83eb-119e303fed77", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdd68150ae0 con 0x7fdd680ce2d0 2024-09-13T18:52:02.083 INFO:teuthology.orchestra.run.smithi171.stdout:4 2024-09-13T18:52:02.084 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.078+0000 7fdd6f199700 1 -- 172.21.15.171:0/2862323437 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdd680ce2d0 msgr2=0x7fdd68157e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.084 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.078+0000 7fdd6f199700 1 --2- 172.21.15.171:0/2862323437 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdd680ce2d0 0x7fdd68157e10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800c500 tx=0x7fdd5800c810 comp rx=0 tx=0).stop 2024-09-13T18:52:02.084 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.078+0000 7fdd6f199700 1 -- 172.21.15.171:0/2862323437 shutdown_connections 2024-09-13T18:52:02.084 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.078+0000 7fdd6f199700 1 --2- 172.21.15.171:0/2862323437 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd68159080 0x7fdd68167360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.084 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.078+0000 7fdd6f199700 1 --2- 172.21.15.171:0/2862323437 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdd680ce2d0 0x7fdd68157e10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.084 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.078+0000 7fdd6f199700 1 --2- 172.21.15.171:0/2862323437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd680cb0d0 0x7fdd68150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.084 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.078+0000 7fdd6f199700 1 -- 172.21.15.171:0/2862323437 >> 172.21.15.171:0/2862323437 conn(0x7fdd680cba60 msgr2=0x7fdd680b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:02.084 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.078+0000 7fdd6f199700 1 -- 172.21.15.171:0/2862323437 shutdown_connections 2024-09-13T18:52:02.085 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.078+0000 7fdd6f199700 1 -- 172.21.15.171:0/2862323437 wait complete. 2024-09-13T18:52:02.096 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new d31123fe-975a-47bd-8b4e-f74f55e62135 5 2024-09-13T18:52:02.110 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:02.106+0000 7f197de502c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T18:52:02.112 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:02.106+0000 7fed851d82c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T18:52:02.163 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:02.158+0000 7f197de502c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T18:52:02.168 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:02.162+0000 7fed851d82c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T18:52:02.212 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.206+0000 7ff00bf01700 1 Processor -- start 2024-09-13T18:52:02.212 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.206+0000 7ff00bf01700 1 -- start start 2024-09-13T18:52:02.214 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00bf01700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff0040cd6d0 0x7ff0040cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.214 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00bf01700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff00406dbf0 0x7ff0040b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.214 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00bf01700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0040cdff0 con 0x7ff00406d760 2024-09-13T18:52:02.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00bf01700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff00415be50 con 0x7ff0040cd6d0 2024-09-13T18:52:02.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00bf01700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff00406b4d0 con 0x7ff00406dbf0 2024-09-13T18:52:02.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00949c700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7ff00406d760 0x7ff0040b2b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.171:46404/0 (socket says 172.21.15.171:46404) 2024-09-13T18:52:02.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00a49e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff00406dbf0 0x7ff0040b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00949c700 1 -- 172.21.15.171:0/4088775634 learned_addr learned my addr 172.21.15.171:0/4088775634 (peer_addr_for_me v1:172.21.15.171:0/0) 2024-09-13T18:52:02.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff009c9d700 1 --2- 172.21.15.171:0/4088775634 >> v2:172.21.15.7:3300/0 conn(0x7ff0040cd6d0 0x7ff0040cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff008c9b700 1 -- 172.21.15.171:0/4088775634 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 565292438 0 0) 0x7ff0040cdff0 con 0x7ff00406d760 2024-09-13T18:52:02.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff008c9b700 1 -- 172.21.15.171:0/4088775634 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feff00035d0 con 0x7ff00406d760 2024-09-13T18:52:02.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00a49e700 1 -- 172.21.15.171:0/4088775634 >> v2:172.21.15.7:3300/0 conn(0x7ff0040cd6d0 msgr2=0x7ff0040cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:02.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00a49e700 1 --2- 172.21.15.171:0/4088775634 >> v2:172.21.15.7:3300/0 conn(0x7ff0040cd6d0 0x7ff0040cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00a49e700 1 -- 172.21.15.171:0/4088775634 >> v1:172.21.15.171:6789/0 conn(0x7ff00406d760 legacy=0x7ff0040b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00a49e700 1 -- 172.21.15.171:0/4088775634 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00415b1d0 con 0x7ff00406dbf0 2024-09-13T18:52:02.215 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00a49e700 1 --2- 172.21.15.171:0/4088775634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff00406dbf0 0x7ff0040b3290 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feffc009d90 tx=0x7feffc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4524c537cdbf35 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:02.216 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff008c9b700 1 -- 172.21.15.171:0/4088775634 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc00e070 con 0x7ff00406dbf0 2024-09-13T18:52:02.216 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff008c9b700 1 -- 172.21.15.171:0/4088775634 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feffc00bb20 con 0x7ff00406dbf0 2024-09-13T18:52:02.216 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff008c9b700 1 -- 172.21.15.171:0/4088775634 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc012430 con 0x7ff00406dbf0 2024-09-13T18:52:02.217 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00bf01700 1 -- 172.21.15.171:0/4088775634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff00406dbf0 msgr2=0x7ff0040b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.217 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00bf01700 1 --2- 172.21.15.171:0/4088775634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff00406dbf0 0x7ff0040b3290 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feffc009d90 tx=0x7feffc00b330 comp rx=0 tx=0).stop 2024-09-13T18:52:02.217 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00bf01700 1 -- 172.21.15.171:0/4088775634 shutdown_connections 2024-09-13T18:52:02.217 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00bf01700 1 --2- 172.21.15.171:0/4088775634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff00406dbf0 0x7ff0040b3290 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.217 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00bf01700 1 --2- 172.21.15.171:0/4088775634 >> v2:172.21.15.7:3300/0 conn(0x7ff0040cd6d0 0x7ff0040cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.217 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00bf01700 1 -- 172.21.15.171:0/4088775634 >> 172.21.15.171:0/4088775634 conn(0x7ff0040af620 msgr2=0x7ff0040afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:02.217 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00bf01700 1 -- 172.21.15.171:0/4088775634 shutdown_connections 2024-09-13T18:52:02.217 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.210+0000 7ff00bf01700 1 -- 172.21.15.171:0/4088775634 wait complete. 2024-09-13T18:52:02.218 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff00bf01700 1 Processor -- start 2024-09-13T18:52:02.218 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff00bf01700 1 -- start start 2024-09-13T18:52:02.221 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff00bf01700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff00406d760 0x7ff004153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.222 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff00bf01700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff00406dbf0 0x7ff0040acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.222 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff00bf01700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0040cd6d0 0x7ff0040b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.222 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff00bf01700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0040ab970 con 0x7ff0040cd6d0 2024-09-13T18:52:02.222 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff00bf01700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0040ad730 con 0x7ff00406dbf0 2024-09-13T18:52:02.222 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff00bf01700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0040ada10 con 0x7ff00406d760 2024-09-13T18:52:02.222 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff009c9d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff00406d760 0x7ff004153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.222 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff009c9d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff00406d760 0x7ff004153c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.171:36388/0 (socket says 172.21.15.171:36388) 2024-09-13T18:52:02.222 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff009c9d700 1 -- 172.21.15.171:0/4150789890 learned_addr learned my addr 172.21.15.171:0/4150789890 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-13T18:52:02.222 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff00949c700 1 --2- 172.21.15.171:0/4150789890 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff00406dbf0 0x7ff0040acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.222 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff00a49e700 1 --2- 172.21.15.171:0/4150789890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0040cd6d0 0x7ff0040b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.222 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff009c9d700 1 -- 172.21.15.171:0/4150789890 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff00406dbf0 msgr2=0x7ff0040acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.222 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff009c9d700 1 --2- 172.21.15.171:0/4150789890 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff00406dbf0 0x7ff0040acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.223 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff009c9d700 1 -- 172.21.15.171:0/4150789890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0040cd6d0 msgr2=0x7ff0040b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.223 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff009c9d700 1 --2- 172.21.15.171:0/4150789890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0040cd6d0 0x7ff0040b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.223 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff009c9d700 1 -- 172.21.15.171:0/4150789890 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feffc009a70 con 0x7ff00406d760 2024-09-13T18:52:02.223 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff00a49e700 1 --2- 172.21.15.171:0/4150789890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0040cd6d0 0x7ff0040b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:02.223 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7ff009c9d700 1 --2- 172.21.15.171:0/4150789890 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff00406d760 0x7ff004153c20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff00000e790 tx=0x7ff00000eaa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:02.223 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.214+0000 7feffa7fc700 1 -- 172.21.15.171:0/4150789890 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff000013610 con 0x7ff00406d760 2024-09-13T18:52:02.223 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.218+0000 7feffa7fc700 1 -- 172.21.15.171:0/4150789890 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff000013770 con 0x7ff00406d760 2024-09-13T18:52:02.223 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.218+0000 7ff00bf01700 1 -- 172.21.15.171:0/4150789890 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0040adcf0 con 0x7ff00406d760 2024-09-13T18:52:02.223 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.218+0000 7feffa7fc700 1 -- 172.21.15.171:0/4150789890 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff000013ae0 con 0x7ff00406d760 2024-09-13T18:52:02.223 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.218+0000 7ff00bf01700 1 -- 172.21.15.171:0/4150789890 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0040ae2b0 con 0x7ff00406d760 2024-09-13T18:52:02.223 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.218+0000 7feffa7fc700 1 -- 172.21.15.171:0/4150789890 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff00001c450 con 0x7ff00406d760 2024-09-13T18:52:02.224 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.218+0000 7feffa7fc700 1 -- 172.21.15.171:0/4150789890 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7ff0000129e0 con 0x7ff00406d760 2024-09-13T18:52:02.224 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.218+0000 7ff00bf01700 1 -- 172.21.15.171:0/4150789890 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0040a8ba0 con 0x7ff00406d760 2024-09-13T18:52:02.224 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.222+0000 7feffa7fc700 1 -- 172.21.15.171:0/4150789890 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7ff0040a8ba0 con 0x7ff00406d760 2024-09-13T18:52:02.298 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.294+0000 7ff00bf01700 1 -- 172.21.15.171:0/4150789890 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d31123fe-975a-47bd-8b4e-f74f55e62135", "id": 5} v 0) v1 -- 0x7ff0040717d0 con 0x7ff00406d760 2024-09-13T18:52:02.304 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.298+0000 7feffa7fc700 1 -- 172.21.15.171:0/4150789890 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d31123fe-975a-47bd-8b4e-f74f55e62135", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff00002a0b0 con 0x7ff00406d760 2024-09-13T18:52:02.304 INFO:teuthology.orchestra.run.smithi171.stdout:5 2024-09-13T18:52:02.306 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.302+0000 7ff00bf01700 1 -- 172.21.15.171:0/4150789890 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff00406d760 msgr2=0x7ff004153c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.306 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.302+0000 7ff00bf01700 1 --2- 172.21.15.171:0/4150789890 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff00406d760 0x7ff004153c20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff00000e790 tx=0x7ff00000eaa0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.307 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.302+0000 7ff00bf01700 1 -- 172.21.15.171:0/4150789890 shutdown_connections 2024-09-13T18:52:02.307 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.302+0000 7ff00bf01700 1 --2- 172.21.15.171:0/4150789890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0040cd6d0 0x7ff0040b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.307 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.302+0000 7ff00bf01700 1 --2- 172.21.15.171:0/4150789890 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff00406dbf0 0x7ff0040acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.307 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.302+0000 7ff00bf01700 1 --2- 172.21.15.171:0/4150789890 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff00406d760 0x7ff004153c20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.307 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.302+0000 7ff00bf01700 1 -- 172.21.15.171:0/4150789890 >> 172.21.15.171:0/4150789890 conn(0x7ff0040af620 msgr2=0x7ff00406b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:02.307 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.302+0000 7ff00bf01700 1 -- 172.21.15.171:0/4150789890 shutdown_connections 2024-09-13T18:52:02.307 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.302+0000 7ff00bf01700 1 -- 172.21.15.171:0/4150789890 wait complete. 2024-09-13T18:52:02.318 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new d7b9748c-4637-4802-a1b8-9d99e2abd698 6 2024-09-13T18:52:02.353 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:02.350+0000 7f197de502c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T18:52:02.382 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:02.378+0000 7fed851d82c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T18:52:02.418 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:02.414+0000 7f197de502c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T18:52:02.420 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.414+0000 7f3d30ab5700 1 Processor -- start 2024-09-13T18:52:02.420 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d30ab5700 1 -- start start 2024-09-13T18:52:02.422 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d30ab5700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f3d2c15aab0 0x7f3d2c15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.422 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d30ab5700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3d2c071330 0x7f3d2c0cba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.422 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d30ab5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d2c0cd350 con 0x7f3d2c06e010 2024-09-13T18:52:02.422 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d30ab5700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d2c0cbf90 con 0x7f3d2c15aab0 2024-09-13T18:52:02.422 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d30ab5700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c0717f0 con 0x7f3d2c071330 2024-09-13T18:52:02.422 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d2affd700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3d2c071330 0x7f3d2c0cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.422 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d2b7fe700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f3d2c15aab0 0x7f3d2c15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.422 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d2b7fe700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f3d2c15aab0 0x7f3d2c15ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:55426/0 (socket says 172.21.15.171:55426) 2024-09-13T18:52:02.422 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d2affd700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3d2c071330 0x7f3d2c0cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.171:36402/0 (socket says 172.21.15.171:36402) 2024-09-13T18:52:02.423 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d2b7fe700 1 -- 172.21.15.171:0/557372892 learned_addr learned my addr 172.21.15.171:0/557372892 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-13T18:52:02.424 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d2b7fe700 1 -- 172.21.15.171:0/557372892 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3d2c071330 msgr2=0x7f3d2c0cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:02.424 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d2b7fe700 1 --2- 172.21.15.171:0/557372892 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3d2c071330 0x7f3d2c0cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.424 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d2b7fe700 1 -- 172.21.15.171:0/557372892 >> v1:172.21.15.7:6789/0 conn(0x7f3d2c06e010 legacy=0x7f3d2c06adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.424 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d2b7fe700 1 -- 172.21.15.171:0/557372892 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c071ad0 con 0x7f3d2c15aab0 2024-09-13T18:52:02.424 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d2affd700 1 --2- 172.21.15.171:0/557372892 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3d2c071330 0x7f3d2c0cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:02.424 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d2b7fe700 1 --2- 172.21.15.171:0/557372892 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d2c15aab0 0x7f3d2c15ae90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009910 tx=0x7f3d1c009c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c62c2a4d626cfd05 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:02.425 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d2a7fc700 1 -- 172.21.15.171:0/557372892 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c011830 con 0x7f3d2c15aab0 2024-09-13T18:52:02.425 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d2a7fc700 1 -- 172.21.15.171:0/557372892 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1c011990 con 0x7f3d2c15aab0 2024-09-13T18:52:02.425 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d2a7fc700 1 -- 172.21.15.171:0/557372892 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c011d40 con 0x7f3d2c15aab0 2024-09-13T18:52:02.425 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d30ab5700 1 -- 172.21.15.171:0/557372892 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d2c15aab0 msgr2=0x7f3d2c15ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.425 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d30ab5700 1 --2- 172.21.15.171:0/557372892 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d2c15aab0 0x7f3d2c15ae90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009910 tx=0x7f3d1c009c20 comp rx=0 tx=0).stop 2024-09-13T18:52:02.425 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d30ab5700 1 -- 172.21.15.171:0/557372892 shutdown_connections 2024-09-13T18:52:02.425 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d30ab5700 1 --2- 172.21.15.171:0/557372892 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3d2c071330 0x7f3d2c0cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.425 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d30ab5700 1 --2- 172.21.15.171:0/557372892 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d2c15aab0 0x7f3d2c15ae90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.425 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d30ab5700 1 -- 172.21.15.171:0/557372892 >> 172.21.15.171:0/557372892 conn(0x7f3d2c0cf670 msgr2=0x7f3d2c06d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:02.425 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d30ab5700 1 -- 172.21.15.171:0/557372892 shutdown_connections 2024-09-13T18:52:02.425 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.418+0000 7f3d30ab5700 1 -- 172.21.15.171:0/557372892 wait complete. 2024-09-13T18:52:02.426 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d30ab5700 1 Processor -- start 2024-09-13T18:52:02.426 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d30ab5700 1 -- start start 2024-09-13T18:52:02.428 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d30ab5700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3d2c06e010 0x7f3d2c13ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.428 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d30ab5700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d2c071330 0x7f3d2c157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.428 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d30ab5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d2c15aab0 0x7f3d2c155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.429 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d30ab5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c13f0f0 con 0x7f3d2c15aab0 2024-09-13T18:52:02.429 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d30ab5700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c152660 con 0x7f3d2c071330 2024-09-13T18:52:02.429 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d30ab5700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c152940 con 0x7f3d2c06e010 2024-09-13T18:52:02.429 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d2b7fe700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3d2c06e010 0x7f3d2c13ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.429 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d2b7fe700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3d2c06e010 0x7f3d2c13ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.171:36410/0 (socket says 172.21.15.171:36410) 2024-09-13T18:52:02.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d2b7fe700 1 -- 172.21.15.171:0/181175082 learned_addr learned my addr 172.21.15.171:0/181175082 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-13T18:52:02.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d2affd700 1 --2- 172.21.15.171:0/181175082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d2c071330 0x7f3d2c157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d2bfff700 1 --2- 172.21.15.171:0/181175082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d2c15aab0 0x7f3d2c155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d2b7fe700 1 -- 172.21.15.171:0/181175082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d2c071330 msgr2=0x7f3d2c157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d2b7fe700 1 --2- 172.21.15.171:0/181175082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d2c071330 0x7f3d2c157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d2b7fe700 1 -- 172.21.15.171:0/181175082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d2c15aab0 msgr2=0x7f3d2c155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.430 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d2b7fe700 1 --2- 172.21.15.171:0/181175082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d2c15aab0 0x7f3d2c155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.431 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d2b7fe700 1 -- 172.21.15.171:0/181175082 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d200094b0 con 0x7f3d2c06e010 2024-09-13T18:52:02.431 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d2b7fe700 1 --2- 172.21.15.171:0/181175082 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3d2c06e010 0x7f3d2c13ebb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c06d6c0 tx=0x7f3d1c004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:02.431 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d13fff700 1 -- 172.21.15.171:0/181175082 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c004510 con 0x7f3d2c06e010 2024-09-13T18:52:02.431 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d13fff700 1 -- 172.21.15.171:0/181175082 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1c01e070 con 0x7f3d2c06e010 2024-09-13T18:52:02.431 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d13fff700 1 -- 172.21.15.171:0/181175082 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c003900 con 0x7f3d2c06e010 2024-09-13T18:52:02.431 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d30ab5700 1 -- 172.21.15.171:0/181175082 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1c009570 con 0x7f3d2c06e010 2024-09-13T18:52:02.432 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.422+0000 7f3d30ab5700 1 -- 172.21.15.171:0/181175082 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d2c152f00 con 0x7f3d2c06e010 2024-09-13T18:52:02.432 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.426+0000 7f3d13fff700 1 -- 172.21.15.171:0/181175082 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f3d1c01a420 con 0x7f3d2c06e010 2024-09-13T18:52:02.432 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.426+0000 7f3d13fff700 1 -- 172.21.15.171:0/181175082 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f3d1c003db0 con 0x7f3d2c06e010 2024-09-13T18:52:02.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.426+0000 7f3d30ab5700 1 -- 172.21.15.171:0/181175082 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2c06e8a0 con 0x7f3d2c06e010 2024-09-13T18:52:02.433 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.430+0000 7f3d13fff700 1 -- 172.21.15.171:0/181175082 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f3d1c020070 con 0x7f3d2c06e010 2024-09-13T18:52:02.451 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:02.446+0000 7fed851d82c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T18:52:02.482 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:02.478+0000 7f197de502c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T18:52:02.508 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.502+0000 7f3d30ab5700 1 -- 172.21.15.171:0/181175082 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d7b9748c-4637-4802-a1b8-9d99e2abd698", "id": 6} v 0) v1 -- 0x7f3d2c06bba0 con 0x7f3d2c06e010 2024-09-13T18:52:02.514 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.510+0000 7f3d13fff700 1 -- 172.21.15.171:0/181175082 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d7b9748c-4637-4802-a1b8-9d99e2abd698", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3d1c015020 con 0x7f3d2c06e010 2024-09-13T18:52:02.515 INFO:teuthology.orchestra.run.smithi171.stdout:6 2024-09-13T18:52:02.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.510+0000 7f3d30ab5700 1 -- 172.21.15.171:0/181175082 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3d2c06e010 msgr2=0x7f3d2c13ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.510+0000 7f3d30ab5700 1 --2- 172.21.15.171:0/181175082 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3d2c06e010 0x7f3d2c13ebb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c06d6c0 tx=0x7f3d1c004770 comp rx=0 tx=0).stop 2024-09-13T18:52:02.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.510+0000 7f3d30ab5700 1 -- 172.21.15.171:0/181175082 shutdown_connections 2024-09-13T18:52:02.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.510+0000 7f3d30ab5700 1 --2- 172.21.15.171:0/181175082 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d2c15aab0 0x7f3d2c155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.510+0000 7f3d30ab5700 1 --2- 172.21.15.171:0/181175082 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d2c071330 0x7f3d2c157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.510+0000 7f3d30ab5700 1 --2- 172.21.15.171:0/181175082 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3d2c06e010 0x7f3d2c13ebb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.510+0000 7f3d30ab5700 1 -- 172.21.15.171:0/181175082 >> 172.21.15.171:0/181175082 conn(0x7f3d2c0cf670 msgr2=0x7f3d2c157df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:02.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.510+0000 7f3d30ab5700 1 -- 172.21.15.171:0/181175082 shutdown_connections 2024-09-13T18:52:02.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.510+0000 7f3d30ab5700 1 -- 172.21.15.171:0/181175082 wait complete. 2024-09-13T18:52:02.523 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:02.518+0000 7fed851d82c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T18:52:02.527 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --cluster ceph osd new e3d83d03-0918-404a-a548-b985f8e18f60 7 2024-09-13T18:52:02.635 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.630+0000 7f80ae724700 1 Processor -- start 2024-09-13T18:52:02.636 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.630+0000 7f80ae724700 1 -- start start 2024-09-13T18:52:02.640 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80ae724700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f80a8156c10 0x7f80a806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.640 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80ae724700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f80a806e720 0x7f80a80b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.640 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80ae724700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a8071fe0 con 0x7f80a806be40 2024-09-13T18:52:02.640 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80ae724700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a8156ff0 con 0x7f80a8156c10 2024-09-13T18:52:02.640 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80ae724700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a80713d0 con 0x7f80a806e720 2024-09-13T18:52:02.641 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80adf23700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f80a806e720 0x7f80a80b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.641 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80adf23700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f80a806e720 0x7f80a80b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:55440/0 (socket says 172.21.15.171:55440) 2024-09-13T18:52:02.641 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80adf23700 1 -- 172.21.15.171:0/172217904 learned_addr learned my addr 172.21.15.171:0/172217904 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-13T18:52:02.641 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80ad722700 1 --2- 172.21.15.171:0/172217904 >> v2:172.21.15.7:3300/0 conn(0x7f80a8156c10 0x7f80a806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.642 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80adf23700 1 -- 172.21.15.171:0/172217904 >> v2:172.21.15.7:3300/0 conn(0x7f80a8156c10 msgr2=0x7f80a806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:02.642 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80adf23700 1 --2- 172.21.15.171:0/172217904 >> v2:172.21.15.7:3300/0 conn(0x7f80a8156c10 0x7f80a806b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.642 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80adf23700 1 -- 172.21.15.171:0/172217904 >> v1:172.21.15.7:6789/0 conn(0x7f80a806be40 legacy=0x7f80a806e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:02.642 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80adf23700 1 -- 172.21.15.171:0/172217904 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a80716b0 con 0x7f80a806e720 2024-09-13T18:52:02.642 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80adf23700 1 --2- 172.21.15.171:0/172217904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a806e720 0x7f80a80b2680 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f80a80b0d40 tx=0x7f80a000d430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=530fb8c68e1a58ad server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:02.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f809ffff700 1 -- 172.21.15.171:0/172217904 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0010070 con 0x7f80a806e720 2024-09-13T18:52:02.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f809ffff700 1 -- 172.21.15.171:0/172217904 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a000dd50 con 0x7f80a806e720 2024-09-13T18:52:02.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f809ffff700 1 -- 172.21.15.171:0/172217904 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a000b310 con 0x7f80a806e720 2024-09-13T18:52:02.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80ae724700 1 -- 172.21.15.171:0/172217904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a806e720 msgr2=0x7f80a80b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80ae724700 1 --2- 172.21.15.171:0/172217904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a806e720 0x7f80a80b2680 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f80a80b0d40 tx=0x7f80a000d430 comp rx=0 tx=0).stop 2024-09-13T18:52:02.643 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80ae724700 1 -- 172.21.15.171:0/172217904 shutdown_connections 2024-09-13T18:52:02.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80ae724700 1 --2- 172.21.15.171:0/172217904 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a806e720 0x7f80a80b2680 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80ae724700 1 --2- 172.21.15.171:0/172217904 >> v2:172.21.15.7:3300/0 conn(0x7f80a8156c10 0x7f80a806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80ae724700 1 -- 172.21.15.171:0/172217904 >> 172.21.15.171:0/172217904 conn(0x7f80a80cf670 msgr2=0x7f80a806d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:02.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80ae724700 1 -- 172.21.15.171:0/172217904 shutdown_connections 2024-09-13T18:52:02.644 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.634+0000 7f80ae724700 1 -- 172.21.15.171:0/172217904 wait complete. 2024-09-13T18:52:02.650 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ae724700 1 Processor -- start 2024-09-13T18:52:02.650 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ae724700 1 -- start start 2024-09-13T18:52:02.650 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ae724700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80a806be40 0x7f80a814bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.650 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ae724700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a806e720 0x7f80a81581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.650 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ae724700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80a8156c10 0x7f80a81586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.650 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ae724700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a81ef800 con 0x7f80a8156c10 2024-09-13T18:52:02.651 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ae724700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a814c1e0 con 0x7f80a806e720 2024-09-13T18:52:02.651 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ae724700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a80ce3e0 con 0x7f80a806be40 2024-09-13T18:52:02.651 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ad722700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80a806be40 0x7f80a814bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.651 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ad722700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80a806be40 0x7f80a814bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.171:36422/0 (socket says 172.21.15.171:36422) 2024-09-13T18:52:02.651 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ad722700 1 -- 172.21.15.171:0/132385262 learned_addr learned my addr 172.21.15.171:0/132385262 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-13T18:52:02.651 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80acf21700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a806e720 0x7f80a81581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.652 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80adf23700 1 --2- 172.21.15.171:0/132385262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80a8156c10 0x7f80a81586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.652 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ad722700 1 -- 172.21.15.171:0/132385262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a806e720 msgr2=0x7f80a81581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.652 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ad722700 1 --2- 172.21.15.171:0/132385262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a806e720 0x7f80a81581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.652 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ad722700 1 -- 172.21.15.171:0/132385262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80a8156c10 msgr2=0x7f80a81586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.652 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ad722700 1 --2- 172.21.15.171:0/132385262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80a8156c10 0x7f80a81586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.652 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ad722700 1 -- 172.21.15.171:0/132385262 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a000bd40 con 0x7f80a806be40 2024-09-13T18:52:02.653 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ad722700 1 --2- 172.21.15.171:0/132385262 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80a806be40 0x7f80a814bca0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f80a400d800 tx=0x7f80a400db10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:02.653 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f809dffb700 1 -- 172.21.15.171:0/132385262 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a40125f0 con 0x7f80a806be40 2024-09-13T18:52:02.653 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f809dffb700 1 -- 172.21.15.171:0/132385262 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a4012750 con 0x7f80a806be40 2024-09-13T18:52:02.653 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ae724700 1 -- 172.21.15.171:0/132385262 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a80cdb40 con 0x7f80a806be40 2024-09-13T18:52:02.653 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f809dffb700 1 -- 172.21.15.171:0/132385262 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4012b00 con 0x7f80a806be40 2024-09-13T18:52:02.653 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ae724700 1 -- 172.21.15.171:0/132385262 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a80ce100 con 0x7f80a806be40 2024-09-13T18:52:02.653 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f809dffb700 1 -- 172.21.15.171:0/132385262 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f80a401e460 con 0x7f80a806be40 2024-09-13T18:52:02.654 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f809dffb700 1 -- 172.21.15.171:0/132385262 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f80a4014bd0 con 0x7f80a806be40 2024-09-13T18:52:02.654 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.638+0000 7f80ae724700 1 -- 172.21.15.171:0/132385262 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a81ef540 con 0x7f80a806be40 2024-09-13T18:52:02.654 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.642+0000 7f809dffb700 1 -- 172.21.15.171:0/132385262 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f80a4014e40 con 0x7f80a806be40 2024-09-13T18:52:02.723 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.718+0000 7f80ae724700 1 -- 172.21.15.171:0/132385262 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e3d83d03-0918-404a-a548-b985f8e18f60", "id": 7} v 0) v1 -- 0x7f80a806b900 con 0x7f80a806be40 2024-09-13T18:52:02.736 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.730+0000 7f809dffb700 1 -- 172.21.15.171:0/132385262 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e3d83d03-0918-404a-a548-b985f8e18f60", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f80a4014640 con 0x7f80a806be40 2024-09-13T18:52:02.736 INFO:teuthology.orchestra.run.smithi171.stdout:7 2024-09-13T18:52:02.738 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.734+0000 7f80ae724700 1 -- 172.21.15.171:0/132385262 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80a806be40 msgr2=0x7f80a814bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.738 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.734+0000 7f80ae724700 1 --2- 172.21.15.171:0/132385262 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80a806be40 0x7f80a814bca0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f80a400d800 tx=0x7f80a400db10 comp rx=0 tx=0).stop 2024-09-13T18:52:02.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.734+0000 7f80ae724700 1 -- 172.21.15.171:0/132385262 shutdown_connections 2024-09-13T18:52:02.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.734+0000 7f80ae724700 1 --2- 172.21.15.171:0/132385262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80a8156c10 0x7f80a81586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.734+0000 7f80ae724700 1 --2- 172.21.15.171:0/132385262 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f80a806e720 0x7f80a81581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.734+0000 7f80ae724700 1 --2- 172.21.15.171:0/132385262 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f80a806be40 0x7f80a814bca0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.734+0000 7f80ae724700 1 -- 172.21.15.171:0/132385262 >> 172.21.15.171:0/132385262 conn(0x7f80a80cf670 msgr2=0x7f80a80b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:02.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.734+0000 7f80ae724700 1 -- 172.21.15.171:0/132385262 shutdown_connections 2024-09-13T18:52:02.739 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-13T18:52:02.734+0000 7f80ae724700 1 -- 172.21.15.171:0/132385262 wait complete. 2024-09-13T18:52:02.750 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T18:52:02.751 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T18:52:02.753 INFO:tasks.ceph.osd.0:Started 2024-09-13T18:52:02.753 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T18:52:02.753 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T18:52:02.755 INFO:tasks.ceph.osd.1:Started 2024-09-13T18:52:02.755 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T18:52:02.755 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T18:52:02.757 INFO:tasks.ceph.osd.2:Started 2024-09-13T18:52:02.757 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T18:52:02.757 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T18:52:02.759 INFO:tasks.ceph.osd.3:Started 2024-09-13T18:52:02.759 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-13T18:52:02.759 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-13T18:52:02.761 INFO:tasks.ceph.osd.4:Started 2024-09-13T18:52:02.761 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T18:52:02.762 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T18:52:02.766 INFO:tasks.ceph.osd.5:Started 2024-09-13T18:52:02.766 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-13T18:52:02.766 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-13T18:52:02.768 INFO:tasks.ceph.osd.6:Started 2024-09-13T18:52:02.768 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-13T18:52:02.768 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-13T18:52:02.770 INFO:tasks.ceph.osd.7:Started 2024-09-13T18:52:02.770 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:02.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.882+0000 7f668a6ba700 1 Processor -- start 2024-09-13T18:52:02.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.882+0000 7f668a6ba700 1 -- start start 2024-09-13T18:52:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f668a6ba700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f66840b4720 0x7f66840b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f668a6ba700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668415b340 con 0x7f6684071a90 2024-09-13T18:52:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f668a6ba700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668415b620 con 0x7f6684151900 2024-09-13T18:52:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f668a6ba700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668415b900 con 0x7f66840b4720 2024-09-13T18:52:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f6689eb9700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6684071a90 0x7f6684071ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33602/0 (socket says 172.21.15.7:33602) 2024-09-13T18:52:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f6689eb9700 1 -- 172.21.15.7:0/1289568927 learned_addr learned my addr 172.21.15.7:0/1289568927 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f6688eb7700 1 --2- 172.21.15.7:0/1289568927 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f66840b4720 0x7f66840b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f667bfff700 1 -- 172.21.15.7:0/1289568927 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 709824684 0 0) 0x7f668415b340 con 0x7f6684071a90 2024-09-13T18:52:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f667bfff700 1 -- 172.21.15.7:0/1289568927 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66700035d0 con 0x7f6684071a90 2024-09-13T18:52:02.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f6688eb7700 1 -- 172.21.15.7:0/1289568927 >> v1:172.21.15.171:6789/0 conn(0x7f6684151900 legacy=0x7f66840b2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f667bfff700 1 -- 172.21.15.7:0/1289568927 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3444587062 0 0) 0x7f668415b620 con 0x7f6684151900 2024-09-13T18:52:02.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f6688eb7700 1 -- 172.21.15.7:0/1289568927 >> v1:172.21.15.7:6789/0 conn(0x7f6684071a90 legacy=0x7f6684071ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f6688eb7700 1 -- 172.21.15.7:0/1289568927 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668415bbe0 con 0x7f66840b4720 2024-09-13T18:52:02.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f6688eb7700 1 --2- 172.21.15.7:0/1289568927 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f66840b4720 0x7f66840b4b60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f667c009dc0 tx=0x7f667c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce02f7299159124d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:02.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f667bfff700 1 -- 172.21.15.7:0/1289568927 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c00e070 con 0x7f66840b4720 2024-09-13T18:52:02.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f667bfff700 1 -- 172.21.15.7:0/1289568927 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c00bb20 con 0x7f66840b4720 2024-09-13T18:52:02.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f667bfff700 1 -- 172.21.15.7:0/1289568927 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c00bed0 con 0x7f66840b4720 2024-09-13T18:52:02.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f668a6ba700 1 -- 172.21.15.7:0/1289568927 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f66840b4720 msgr2=0x7f66840b4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f668a6ba700 1 --2- 172.21.15.7:0/1289568927 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f66840b4720 0x7f66840b4b60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f667c009dc0 tx=0x7f667c00b330 comp rx=0 tx=0).stop 2024-09-13T18:52:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f668a6ba700 1 -- 172.21.15.7:0/1289568927 shutdown_connections 2024-09-13T18:52:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f668a6ba700 1 --2- 172.21.15.7:0/1289568927 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f66840b4720 0x7f66840b4b60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f668a6ba700 1 -- 172.21.15.7:0/1289568927 >> 172.21.15.7:0/1289568927 conn(0x7f66840cbb60 msgr2=0x7f66840cbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f668a6ba700 1 -- 172.21.15.7:0/1289568927 shutdown_connections 2024-09-13T18:52:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.886+0000 7f668a6ba700 1 -- 172.21.15.7:0/1289568927 wait complete. 2024-09-13T18:52:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f668a6ba700 1 Processor -- start 2024-09-13T18:52:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f668a6ba700 1 -- start start 2024-09-13T18:52:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f668a6ba700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6684071a90 0x7f668414e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f668a6ba700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66840b4720 0x7f6684153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f668a6ba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6684151900 0x7f6684154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:02.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f668a6ba700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6684153b20 con 0x7f6684151900 2024-09-13T18:52:02.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f668a6ba700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6684155110 con 0x7f66840b4720 2024-09-13T18:52:02.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f668a6ba700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6684153180 con 0x7f6684071a90 2024-09-13T18:52:02.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f6689eb9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6684151900 0x7f6684154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f66896b8700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6684071a90 0x7f668414e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f6688eb7700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66840b4720 0x7f6684153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:02.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f6689eb9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6684151900 0x7f6684154730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51028/0 (socket says 172.21.15.7:51028) 2024-09-13T18:52:02.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f6689eb9700 1 -- 172.21.15.7:0/48286311 learned_addr learned my addr 172.21.15.7:0/48286311 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:02.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f66896b8700 1 --2- 172.21.15.7:0/48286311 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6684071a90 0x7f668414e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:52694/0 (socket says 172.21.15.7:52694) 2024-09-13T18:52:02.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f6689eb9700 1 -- 172.21.15.7:0/48286311 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6684071a90 msgr2=0x7f668414e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f6689eb9700 1 --2- 172.21.15.7:0/48286311 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6684071a90 0x7f668414e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f6689eb9700 1 -- 172.21.15.7:0/48286311 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66840b4720 msgr2=0x7f6684153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f6689eb9700 1 --2- 172.21.15.7:0/48286311 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66840b4720 0x7f6684153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f6689eb9700 1 -- 172.21.15.7:0/48286311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667c009a70 con 0x7f6684151900 2024-09-13T18:52:02.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f6689eb9700 1 --2- 172.21.15.7:0/48286311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6684151900 0x7f6684154730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f668000c4d0 tx=0x7f668000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:02.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f6679ffb700 1 -- 172.21.15.7:0/48286311 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680005d90 con 0x7f6684151900 2024-09-13T18:52:02.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f6679ffb700 1 -- 172.21.15.7:0/48286311 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668000ce80 con 0x7f6684151900 2024-09-13T18:52:02.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f668a6ba700 1 -- 172.21.15.7:0/48286311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6684153460 con 0x7f6684151900 2024-09-13T18:52:02.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.890+0000 7f6679ffb700 1 -- 172.21.15.7:0/48286311 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66800072f0 con 0x7f6684151900 2024-09-13T18:52:02.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.894+0000 7f668a6ba700 1 -- 172.21.15.7:0/48286311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668415dc40 con 0x7f6684151900 2024-09-13T18:52:02.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.894+0000 7f6679ffb700 1 -- 172.21.15.7:0/48286311 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f668001c440 con 0x7f6684151900 2024-09-13T18:52:02.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.894+0000 7f6679ffb700 1 -- 172.21.15.7:0/48286311 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f66800077f0 con 0x7f6684151900 2024-09-13T18:52:02.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.894+0000 7f668a6ba700 1 -- 172.21.15.7:0/48286311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6684153c60 con 0x7f6684151900 2024-09-13T18:52:02.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.898+0000 7f6679ffb700 1 -- 172.21.15.7:0/48286311 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f6684153c60 con 0x7f6684151900 2024-09-13T18:52:02.928 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:02.922+0000 7f197de502c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T18:52:02.958 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:02.954+0000 7fed851d82c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T18:52:02.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.966+0000 7f668a6ba700 1 -- 172.21.15.7:0/48286311 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f668415af60 con 0x7f6684151900 2024-09-13T18:52:02.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.966+0000 7f6679ffb700 1 -- 172.21.15.7:0/48286311 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f6680017070 con 0x7f6684151900 2024-09-13T18:52:02.972 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:02.972 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:02.723324+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:02.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.970+0000 7f668a6ba700 1 -- 172.21.15.7:0/48286311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6684151900 msgr2=0x7f6684154730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:02.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.970+0000 7f668a6ba700 1 --2- 172.21.15.7:0/48286311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6684151900 0x7f6684154730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f668000c4d0 tx=0x7f668000c7e0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.970+0000 7f668a6ba700 1 -- 172.21.15.7:0/48286311 shutdown_connections 2024-09-13T18:52:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.970+0000 7f668a6ba700 1 --2- 172.21.15.7:0/48286311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6684151900 0x7f6684154730 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.970+0000 7f668a6ba700 1 --2- 172.21.15.7:0/48286311 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f66840b4720 0x7f6684153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.970+0000 7f668a6ba700 1 --2- 172.21.15.7:0/48286311 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6684071a90 0x7f668414e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.970+0000 7f668a6ba700 1 -- 172.21.15.7:0/48286311 >> 172.21.15.7:0/48286311 conn(0x7f66840cbb60 msgr2=0x7f668406d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.970+0000 7f668a6ba700 1 -- 172.21.15.7:0/48286311 shutdown_connections 2024-09-13T18:52:02.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:02.970+0000 7f668a6ba700 1 -- 172.21.15.7:0/48286311 wait complete. 2024-09-13T18:52:02.985 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T18:52:02.985 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T18:52:03.053 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:03.050+0000 7f197de502c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T18:52:03.086 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:03.082+0000 7fed851d82c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T18:52:03.150 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-13T18:52:03.146+0000 7f2b64748880 -1 Falling back to public interface 2024-09-13T18:52:03.154 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-13T18:52:03.150+0000 7f3a5ce4c880 -1 Falling back to public interface 2024-09-13T18:52:03.155 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-13T18:52:03.150+0000 7fb6c5466880 -1 Falling back to public interface 2024-09-13T18:52:03.155 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-13T18:52:03.150+0000 7fa0f0a95880 -1 Falling back to public interface 2024-09-13T18:52:03.155 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-13T18:52:03.150+0000 7f3a5be17700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6806/0: (98) Address already in use 2024-09-13T18:52:03.381 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-13T18:52:03.378+0000 7fea1848c880 -1 Falling back to public interface 2024-09-13T18:52:03.394 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-13T18:52:03.390+0000 7f85a38f4880 -1 Falling back to public interface 2024-09-13T18:52:03.394 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-13T18:52:03.390+0000 7f1005e47880 -1 Falling back to public interface 2024-09-13T18:52:03.411 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-13T18:52:03.406+0000 7f314f239880 -1 Falling back to public interface 2024-09-13T18:52:03.425 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:03.422+0000 7f197de502c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T18:52:03.466 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:03.462+0000 7fed851d82c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T18:52:03.611 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:03.606+0000 7f197de502c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:52:03.660 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:03.654+0000 7fed851d82c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T18:52:03.679 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:03.674+0000 7f197de502c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T18:52:03.730 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:03.726+0000 7fed851d82c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T18:52:03.791 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:03.844 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:03.842+0000 7f197de502c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T18:52:03.897 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.890+0000 7f64663c0700 1 Processor -- start 2024-09-13T18:52:03.897 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.890+0000 7f64663c0700 1 -- start start 2024-09-13T18:52:03.900 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f64663c0700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f646006a580 0x7f646006ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:03.900 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f64663c0700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f64600b3aa0 0x7f646006a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:03.900 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f64663c0700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64600b4340 con 0x7f646006aea0 2024-09-13T18:52:03.900 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f64663c0700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6460159480 con 0x7f64600b3aa0 2024-09-13T18:52:03.900 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f64663c0700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64600b4060 con 0x7f646006a580 2024-09-13T18:52:03.900 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f6465bbf700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f646006aea0 0x7f64600ac250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33606/0 (socket says 172.21.15.7:33606) 2024-09-13T18:52:03.900 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f6465bbf700 1 -- 172.21.15.7:0/3544089774 learned_addr learned my addr 172.21.15.7:0/3544089774 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:03.900 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f64653be700 1 --2- 172.21.15.7:0/3544089774 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f646006a580 0x7f646006ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:03.900 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f6464bbd700 1 --2- 172.21.15.7:0/3544089774 >> v2:172.21.15.171:3300/0 conn(0x7f64600b3aa0 0x7f646006a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:03.900 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f6457fff700 1 -- 172.21.15.7:0/3544089774 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2869593611 0 0) 0x7f64600b4340 con 0x7f646006aea0 2024-09-13T18:52:03.901 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f6457fff700 1 -- 172.21.15.7:0/3544089774 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f644c0035d0 con 0x7f646006aea0 2024-09-13T18:52:03.901 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f64653be700 1 -- 172.21.15.7:0/3544089774 >> v2:172.21.15.171:3300/0 conn(0x7f64600b3aa0 msgr2=0x7f646006a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:03.901 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f64653be700 1 --2- 172.21.15.7:0/3544089774 >> v2:172.21.15.171:3300/0 conn(0x7f64600b3aa0 0x7f646006a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:03.901 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f64653be700 1 -- 172.21.15.7:0/3544089774 >> v1:172.21.15.7:6789/0 conn(0x7f646006aea0 legacy=0x7f64600ac250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:03.901 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f64653be700 1 -- 172.21.15.7:0/3544089774 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646006dd20 con 0x7f646006a580 2024-09-13T18:52:03.902 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f64653be700 1 --2- 172.21.15.7:0/3544089774 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f646006a580 0x7f646006ce90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f645c009850 tx=0x7f645c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=281ffaa9f6841f99 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:03.902 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f6457fff700 1 -- 172.21.15.7:0/3544089774 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c011830 con 0x7f646006a580 2024-09-13T18:52:03.902 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f6457fff700 1 -- 172.21.15.7:0/3544089774 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645c011990 con 0x7f646006a580 2024-09-13T18:52:03.902 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.894+0000 7f6457fff700 1 -- 172.21.15.7:0/3544089774 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c011d40 con 0x7f646006a580 2024-09-13T18:52:03.902 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64663c0700 1 -- 172.21.15.7:0/3544089774 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f646006a580 msgr2=0x7f646006ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:03.902 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64663c0700 1 --2- 172.21.15.7:0/3544089774 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f646006a580 0x7f646006ce90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f645c009850 tx=0x7f645c009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:03.902 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64663c0700 1 -- 172.21.15.7:0/3544089774 shutdown_connections 2024-09-13T18:52:03.902 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64663c0700 1 --2- 172.21.15.7:0/3544089774 >> v2:172.21.15.171:3300/0 conn(0x7f64600b3aa0 0x7f646006a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:03.902 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64663c0700 1 --2- 172.21.15.7:0/3544089774 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f646006a580 0x7f646006ce90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:03.902 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64663c0700 1 -- 172.21.15.7:0/3544089774 >> 172.21.15.7:0/3544089774 conn(0x7f64600b0cd0 msgr2=0x7f64600b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:03.903 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64663c0700 1 -- 172.21.15.7:0/3544089774 shutdown_connections 2024-09-13T18:52:03.903 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64663c0700 1 -- 172.21.15.7:0/3544089774 wait complete. 2024-09-13T18:52:03.905 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64663c0700 1 Processor -- start 2024-09-13T18:52:03.905 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64663c0700 1 -- start start 2024-09-13T18:52:03.905 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64663c0700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f646006a580 0x7f646015a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:03.905 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64663c0700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f646006aea0 0x7f6460151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:03.905 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64663c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64600b3aa0 0x7f64601f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:03.905 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64663c0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6460151560 con 0x7f64600b3aa0 2024-09-13T18:52:03.905 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64663c0700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6460152530 con 0x7f646006a580 2024-09-13T18:52:03.905 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64663c0700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646015ad20 con 0x7f646006aea0 2024-09-13T18:52:03.905 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f6465bbf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64600b3aa0 0x7f64601f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:03.905 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f6465bbf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64600b3aa0 0x7f64601f1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51032/0 (socket says 172.21.15.7:51032) 2024-09-13T18:52:03.905 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f6465bbf700 1 -- 172.21.15.7:0/32940841 learned_addr learned my addr 172.21.15.7:0/32940841 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:03.905 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f6464bbd700 1 --2- 172.21.15.7:0/32940841 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f646006aea0 0x7f6460151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:03.905 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64653be700 1 --2- 172.21.15.7:0/32940841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f646006a580 0x7f646015a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:03.906 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f6465bbf700 1 -- 172.21.15.7:0/32940841 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f646006aea0 msgr2=0x7f6460151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:03.906 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f6465bbf700 1 --2- 172.21.15.7:0/32940841 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f646006aea0 0x7f6460151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:03.906 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f6465bbf700 1 -- 172.21.15.7:0/32940841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f646006a580 msgr2=0x7f646015a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:03.906 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f6465bbf700 1 --2- 172.21.15.7:0/32940841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f646006a580 0x7f646015a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:03.906 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f6465bbf700 1 -- 172.21.15.7:0/32940841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c0094b0 con 0x7f64600b3aa0 2024-09-13T18:52:03.906 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f6465bbf700 1 --2- 172.21.15.7:0/32940841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64600b3aa0 0x7f64601f1d80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f645800ca70 tx=0x7f645800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:03.906 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f6455ffb700 1 -- 172.21.15.7:0/32940841 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645800a980 con 0x7f64600b3aa0 2024-09-13T18:52:03.906 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f6455ffb700 1 -- 172.21.15.7:0/32940841 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645800aae0 con 0x7f64600b3aa0 2024-09-13T18:52:03.906 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64663c0700 1 -- 172.21.15.7:0/32940841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646015b000 con 0x7f64600b3aa0 2024-09-13T18:52:03.906 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f6455ffb700 1 -- 172.21.15.7:0/32940841 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645800ae70 con 0x7f64600b3aa0 2024-09-13T18:52:03.906 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f64663c0700 1 -- 172.21.15.7:0/32940841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646015b5c0 con 0x7f64600b3aa0 2024-09-13T18:52:03.906 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.898+0000 7f6455ffb700 1 -- 172.21.15.7:0/32940841 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f645801c740 con 0x7f64600b3aa0 2024-09-13T18:52:03.906 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.902+0000 7f6455ffb700 1 -- 172.21.15.7:0/32940841 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f6458012400 con 0x7f64600b3aa0 2024-09-13T18:52:03.907 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.902+0000 7f64663c0700 1 -- 172.21.15.7:0/32940841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64600665e0 con 0x7f64600b3aa0 2024-09-13T18:52:03.908 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.902+0000 7f6455ffb700 1 -- 172.21.15.7:0/32940841 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f6458017070 con 0x7f64600b3aa0 2024-09-13T18:52:03.915 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:03.910+0000 7fed851d82c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T18:52:03.971 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.966+0000 7f64663c0700 1 -- 172.21.15.7:0/32940841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6460157cd0 con 0x7f64600b3aa0 2024-09-13T18:52:03.972 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.966+0000 7f6455ffb700 1 -- 172.21.15.7:0/32940841 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f64580063e0 con 0x7f64600b3aa0 2024-09-13T18:52:03.972 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-13T18:52:03.972 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":10,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:02.723324+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:03.973 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.970+0000 7f64663c0700 1 -- 172.21.15.7:0/32940841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64600b3aa0 msgr2=0x7f64601f1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:03.973 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.970+0000 7f64663c0700 1 --2- 172.21.15.7:0/32940841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64600b3aa0 0x7f64601f1d80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f645800ca70 tx=0x7f645800cd80 comp rx=0 tx=0).stop 2024-09-13T18:52:03.973 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.970+0000 7f64663c0700 1 -- 172.21.15.7:0/32940841 shutdown_connections 2024-09-13T18:52:03.973 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.970+0000 7f64663c0700 1 --2- 172.21.15.7:0/32940841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64600b3aa0 0x7f64601f1d80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:03.974 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.970+0000 7f64663c0700 1 --2- 172.21.15.7:0/32940841 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f646006aea0 0x7f6460151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:03.974 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.970+0000 7f64663c0700 1 --2- 172.21.15.7:0/32940841 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f646006a580 0x7f646015a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:03.974 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.970+0000 7f64663c0700 1 -- 172.21.15.7:0/32940841 >> 172.21.15.7:0/32940841 conn(0x7f64600b0cd0 msgr2=0x7f646009cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:03.974 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.970+0000 7f64663c0700 1 -- 172.21.15.7:0/32940841 shutdown_connections 2024-09-13T18:52:03.974 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:03.970+0000 7f64663c0700 1 -- 172.21.15.7:0/32940841 wait complete. 2024-09-13T18:52:03.983 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T18:52:04.679 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:04.674+0000 7fed851d82c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T18:52:04.688 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:04.682+0000 7f197de502c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T18:52:04.749 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:04.746+0000 7fed851d82c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T18:52:04.759 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:04.754+0000 7f197de502c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T18:52:04.913 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:04.910+0000 7fed851d82c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T18:52:04.931 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:04.926+0000 7f197de502c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T18:52:04.983 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:04.978+0000 7fed851d82c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T18:52:05.000 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:04.994+0000 7f197de502c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T18:52:05.061 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:05.054+0000 7fed851d82c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T18:52:05.074 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:05.070+0000 7f197de502c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T18:52:05.391 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:05.386+0000 7fed851d82c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T18:52:05.400 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:05.394+0000 7f197de502c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T18:52:05.469 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:05.466+0000 7fed851d82c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T18:52:05.477 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:05.474+0000 7f197de502c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T18:52:06.175 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:06.170+0000 7f197de502c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T18:52:06.212 INFO:tasks.ceph.mgr.y.smithi007.stderr:2024-09-13T18:52:06.206+0000 7fed851d82c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T18:52:06.789 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-13T18:52:06.782+0000 7f3a5ce4c880 -1 osd.3 0 log_to_monitors true 2024-09-13T18:52:07.035 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-13T18:52:07.030+0000 7fa0f0a95880 -1 osd.1 0 log_to_monitors true 2024-09-13T18:52:07.041 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-13T18:52:07.038+0000 7fb6c5466880 -1 osd.0 0 log_to_monitors true 2024-09-13T18:52:07.068 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-13T18:52:07.062+0000 7f2b64748880 -1 osd.5 0 log_to_monitors true 2024-09-13T18:52:07.216 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:07.214+0000 7f19649c6700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T18:52:07.217 INFO:tasks.ceph.mgr.x.smithi171.stderr:2024-09-13T18:52:07.214+0000 7f19649c6700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T18:52:07.230 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-13T18:52:07.226+0000 7fea1848c880 -1 osd.6 0 log_to_monitors true 2024-09-13T18:52:07.239 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-13T18:52:07.234+0000 7f1005e47880 -1 osd.7 0 log_to_monitors true 2024-09-13T18:52:07.283 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-13T18:52:07.278+0000 7f85a38f4880 -1 osd.4 0 log_to_monitors true 2024-09-13T18:52:07.305 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-13T18:52:07.302+0000 7f314f239880 -1 osd.2 0 log_to_monitors true 2024-09-13T18:52:09.697 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-13T18:52:09.690+0000 7fb6c0a82700 -1 osd.0 0 waiting for initial osdmap 2024-09-13T18:52:09.702 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-13T18:52:09.698+0000 7fb6b8d57700 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:52:09.702 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-13T18:52:09.698+0000 7fa0ec0b1700 -1 osd.1 0 waiting for initial osdmap 2024-09-13T18:52:09.714 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-13T18:52:09.710+0000 7fa0e4386700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:52:09.720 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-13T18:52:09.714+0000 7f3a58468700 -1 osd.3 0 waiting for initial osdmap 2024-09-13T18:52:09.726 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-13T18:52:09.722+0000 7f3a5073d700 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:52:10.177 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-13T18:52:10.174+0000 7fea13aa8700 -1 osd.6 0 waiting for initial osdmap 2024-09-13T18:52:10.186 INFO:tasks.ceph.osd.6.smithi171.stderr:2024-09-13T18:52:10.182+0000 7fea0bd7d700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:52:10.233 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-13T18:52:10.230+0000 7f2b6170f700 -1 osd.5 0 waiting for initial osdmap 2024-09-13T18:52:10.240 INFO:tasks.ceph.osd.5.smithi171.stderr:2024-09-13T18:52:10.234+0000 7f2b58039700 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:52:10.248 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-13T18:52:10.246+0000 7f1002e0e700 -1 osd.7 0 waiting for initial osdmap 2024-09-13T18:52:10.255 INFO:tasks.ceph.osd.7.smithi171.stderr:2024-09-13T18:52:10.250+0000 7f0ff9738700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:52:10.539 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-13T18:52:10.534+0000 7f314a855700 -1 osd.2 0 waiting for initial osdmap 2024-09-13T18:52:10.545 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-13T18:52:10.542+0000 7f3142b2a700 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:52:10.683 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-13T18:52:10.678+0000 7f85a08bb700 -1 osd.4 0 waiting for initial osdmap 2024-09-13T18:52:10.687 INFO:tasks.ceph.osd.4.smithi171.stderr:2024-09-13T18:52:10.682+0000 7f85971e5700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T18:52:10.789 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:10.901 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bd5264700 1 Processor -- start 2024-09-13T18:52:10.901 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bd5264700 1 -- start start 2024-09-13T18:52:10.904 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bd5264700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bd015ceb0 0x7f7bd015a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:10.904 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bd5264700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f7bd006dbb0 0x7f7bd015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:10.904 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bd5264700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd006d810 con 0x7f7bd015b4f0 2024-09-13T18:52:10.904 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bd5264700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd006a310 con 0x7f7bd006dbb0 2024-09-13T18:52:10.904 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bd5264700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd006a560 con 0x7f7bd015ceb0 2024-09-13T18:52:10.904 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bd4a63700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7bd015b4f0 0x7f7bd006b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33622/0 (socket says 172.21.15.7:33622) 2024-09-13T18:52:10.904 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bd4a63700 1 -- 172.21.15.7:0/4095929771 learned_addr learned my addr 172.21.15.7:0/4095929771 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:10.905 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bcffff700 1 --2- 172.21.15.7:0/4095929771 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bd015ceb0 0x7f7bd015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:10.905 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bcf7fe700 1 --2- 172.21.15.7:0/4095929771 >> v2:172.21.15.171:3300/0 conn(0x7f7bd006dbb0 0x7f7bd015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:10.905 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bceffd700 1 -- 172.21.15.7:0/4095929771 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008562236 0 0) 0x7f7bd006d810 con 0x7f7bd015b4f0 2024-09-13T18:52:10.905 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bceffd700 1 -- 172.21.15.7:0/4095929771 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb80035d0 con 0x7f7bd015b4f0 2024-09-13T18:52:10.908 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bceffd700 1 -- 172.21.15.7:0/4095929771 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2926968692 0 0) 0x7f7bb80035d0 con 0x7f7bd015b4f0 2024-09-13T18:52:10.908 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bceffd700 1 -- 172.21.15.7:0/4095929771 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bd015ceb0 msgr2=0x7f7bd015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:10.908 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bceffd700 1 --2- 172.21.15.7:0/4095929771 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bd015ceb0 0x7f7bd015a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:10.908 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bceffd700 1 -- 172.21.15.7:0/4095929771 >> v2:172.21.15.171:3300/0 conn(0x7f7bd006dbb0 msgr2=0x7f7bd015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:10.908 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bceffd700 1 --2- 172.21.15.7:0/4095929771 >> v2:172.21.15.171:3300/0 conn(0x7f7bd006dbb0 0x7f7bd015afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:10.908 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bceffd700 1 -- 172.21.15.7:0/4095929771 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd006a840 con 0x7f7bd015b4f0 2024-09-13T18:52:10.908 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bceffd700 1 -- 172.21.15.7:0/4095929771 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7f7bc80019f0 con 0x7f7bd015b4f0 2024-09-13T18:52:10.908 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bcf7fe700 1 --2- 172.21.15.7:0/4095929771 >> v2:172.21.15.171:3300/0 conn(0x7f7bd006dbb0 0x7f7bd015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:10.908 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.898+0000 7f7bceffd700 1 -- 172.21.15.7:0/4095929771 >> v1:172.21.15.7:6789/0 conn(0x7f7bd015b4f0 legacy=0x7f7bd006b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:10.909 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bceffd700 1 --2- 172.21.15.7:0/4095929771 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bd015ceb0 0x7f7bb8003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:10.909 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bceffd700 1 --2- 172.21.15.7:0/4095929771 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7bb8003fa0 0x7f7bb8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:10.909 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bceffd700 1 --2- 172.21.15.7:0/4095929771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb8004db0 0x7f7bb8005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:10.909 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bceffd700 1 -- 172.21.15.7:0/4095929771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd006a310 con 0x7f7bb8004db0 2024-09-13T18:52:10.909 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bceffd700 1 -- 172.21.15.7:0/4095929771 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd006a560 con 0x7f7bb8003fa0 2024-09-13T18:52:10.909 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bceffd700 1 -- 172.21.15.7:0/4095929771 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb800ba30 con 0x7f7bd015ceb0 2024-09-13T18:52:10.909 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bd4a63700 1 --2- 172.21.15.7:0/4095929771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb8004db0 0x7f7bb8005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:10.909 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bcffff700 1 --2- 172.21.15.7:0/4095929771 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bd015ceb0 0x7f7bb8003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:10.909 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bcf7fe700 1 --2- 172.21.15.7:0/4095929771 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7bb8003fa0 0x7f7bb8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:10.909 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bd4a63700 1 -- 172.21.15.7:0/4095929771 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bd015ceb0 msgr2=0x7f7bb8003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:10.910 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bd4a63700 1 --2- 172.21.15.7:0/4095929771 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bd015ceb0 0x7f7bb8003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:10.910 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bd4a63700 1 -- 172.21.15.7:0/4095929771 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7bb8003fa0 msgr2=0x7f7bb8004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:10.910 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bd4a63700 1 --2- 172.21.15.7:0/4095929771 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7bb8003fa0 0x7f7bb8004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:10.910 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bd4a63700 1 -- 172.21.15.7:0/4095929771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb80035d0 con 0x7f7bb8004db0 2024-09-13T18:52:10.910 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bd4a63700 1 --2- 172.21.15.7:0/4095929771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb8004db0 0x7f7bb8005230 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800a860 tx=0x7f7bb80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809b01e67ec9cf97 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:10.910 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bceffd700 1 -- 172.21.15.7:0/4095929771 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc80072d0 con 0x7f7bb8004db0 2024-09-13T18:52:10.910 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bceffd700 1 -- 172.21.15.7:0/4095929771 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc8007430 con 0x7f7bb8004db0 2024-09-13T18:52:10.910 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bceffd700 1 -- 172.21.15.7:0/4095929771 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc80077e0 con 0x7f7bb8004db0 2024-09-13T18:52:10.910 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bd5264700 1 -- 172.21.15.7:0/4095929771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb8004db0 msgr2=0x7f7bb8005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:10.910 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bd5264700 1 --2- 172.21.15.7:0/4095929771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb8004db0 0x7f7bb8005230 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800a860 tx=0x7f7bb80058a0 comp rx=0 tx=0).stop 2024-09-13T18:52:10.911 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bd4a63700 1 -- 172.21.15.7:0/4095929771 reap_dead start 2024-09-13T18:52:10.911 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bd5264700 1 -- 172.21.15.7:0/4095929771 shutdown_connections 2024-09-13T18:52:10.911 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bd5264700 1 -- 172.21.15.7:0/4095929771 >> 172.21.15.7:0/4095929771 conn(0x7f7bd00b0cd0 msgr2=0x7f7bd00b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:10.911 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bd5264700 1 -- 172.21.15.7:0/4095929771 shutdown_connections 2024-09-13T18:52:10.911 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.902+0000 7f7bd5264700 1 -- 172.21.15.7:0/4095929771 wait complete. 2024-09-13T18:52:10.913 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bd5264700 1 Processor -- start 2024-09-13T18:52:10.913 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bd5264700 1 -- start start 2024-09-13T18:52:10.913 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bd5264700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb8004db0 0x7f7bd009b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:10.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bd5264700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bd014c7e0 0x7f7bd00afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:10.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bd5264700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7bd01566e0 0x7f7bd0156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:10.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bd5264700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd0154aa0 con 0x7f7bb8004db0 2024-09-13T18:52:10.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bd5264700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd0154470 con 0x7f7bd01566e0 2024-09-13T18:52:10.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bd5264700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd01570a0 con 0x7f7bd014c7e0 2024-09-13T18:52:10.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bcffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb8004db0 0x7f7bd009b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:10.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bcffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb8004db0 0x7f7bd009b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51090/0 (socket says 172.21.15.7:51090) 2024-09-13T18:52:10.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bcffff700 1 -- 172.21.15.7:0/1004573551 learned_addr learned my addr 172.21.15.7:0/1004573551 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:10.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bcf7fe700 1 --2- 172.21.15.7:0/1004573551 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bd014c7e0 0x7f7bd00afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:10.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bd4a63700 1 --2- 172.21.15.7:0/1004573551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7bd01566e0 0x7f7bd0156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:10.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bcffff700 1 -- 172.21.15.7:0/1004573551 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bd014c7e0 msgr2=0x7f7bd00afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:10.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bcffff700 1 --2- 172.21.15.7:0/1004573551 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bd014c7e0 0x7f7bd00afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:10.915 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bcffff700 1 -- 172.21.15.7:0/1004573551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7bd01566e0 msgr2=0x7f7bd0156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:10.915 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bcffff700 1 --2- 172.21.15.7:0/1004573551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7bd01566e0 0x7f7bd0156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:10.915 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bcffff700 1 -- 172.21.15.7:0/1004573551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd009bea0 con 0x7f7bb8004db0 2024-09-13T18:52:10.915 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bd4a63700 1 --2- 172.21.15.7:0/1004573551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7bd01566e0 0x7f7bd0156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:10.915 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bcffff700 1 --2- 172.21.15.7:0/1004573551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb8004db0 0x7f7bd009b5a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0013de0 tx=0x7f7bc000e510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:10.915 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bccff9700 1 -- 172.21.15.7:0/1004573551 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0018070 con 0x7f7bb8004db0 2024-09-13T18:52:10.915 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bccff9700 1 -- 172.21.15.7:0/1004573551 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc000ecb0 con 0x7f7bb8004db0 2024-09-13T18:52:10.915 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bd5264700 1 -- 172.21.15.7:0/1004573551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd009c180 con 0x7f7bb8004db0 2024-09-13T18:52:10.915 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bccff9700 1 -- 172.21.15.7:0/1004573551 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc000ee40 con 0x7f7bb8004db0 2024-09-13T18:52:10.915 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.906+0000 7f7bd5264700 1 -- 172.21.15.7:0/1004573551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd009bae0 con 0x7f7bb8004db0 2024-09-13T18:52:10.916 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.910+0000 7f7bd5264700 1 -- 172.21.15.7:0/1004573551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd015afb0 con 0x7f7bb8004db0 2024-09-13T18:52:10.916 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.910+0000 7f7bccff9700 1 -- 172.21.15.7:0/1004573551 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f7bc001c410 con 0x7f7bb8004db0 2024-09-13T18:52:10.917 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.910+0000 7f7bccff9700 1 --2- 172.21.15.7:0/1004573551 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7bbc0666a0 0x7f7bbc068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:10.917 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.910+0000 7f7bccff9700 1 -- 172.21.15.7:0/1004573551 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f7bc00876b0 con 0x7f7bb8004db0 2024-09-13T18:52:10.917 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.910+0000 7f7bcf7fe700 1 --2- 172.21.15.7:0/1004573551 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7bbc0666a0 0x7f7bbc068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:10.917 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.914+0000 7f7bcf7fe700 1 --2- 172.21.15.7:0/1004573551 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7bbc0666a0 0x7f7bbc068b60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7bd015ae40 tx=0x7f7bc4009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:10.925 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:10.922+0000 7f7bccff9700 1 -- 172.21.15.7:0/1004573551 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7bc00551d0 con 0x7f7bb8004db0 2024-09-13T18:52:11.096 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:11.090+0000 7f7bd5264700 1 -- 172.21.15.7:0/1004573551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bd00ac8f0 con 0x7f7bb8004db0 2024-09-13T18:52:11.097 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:11.094+0000 7f7bccff9700 1 -- 172.21.15.7:0/1004573551 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10860 (secure 0 0 0) 0x7f7bc0021020 con 0x7f7bb8004db0 2024-09-13T18:52:11.098 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-13T18:52:11.098 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":14,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:10.708691+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:11.100 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:11.094+0000 7f7bd5264700 1 -- 172.21.15.7:0/1004573551 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7bbc0666a0 msgr2=0x7f7bbc068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:11.101 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:11.094+0000 7f7bd5264700 1 --2- 172.21.15.7:0/1004573551 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7bbc0666a0 0x7f7bbc068b60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7bd015ae40 tx=0x7f7bc4009240 comp rx=0 tx=0).stop 2024-09-13T18:52:11.101 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:11.094+0000 7f7bd5264700 1 -- 172.21.15.7:0/1004573551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb8004db0 msgr2=0x7f7bd009b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:11.101 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:11.094+0000 7f7bd5264700 1 --2- 172.21.15.7:0/1004573551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb8004db0 0x7f7bd009b5a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0013de0 tx=0x7f7bc000e510 comp rx=0 tx=0).stop 2024-09-13T18:52:11.101 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:11.094+0000 7f7bd5264700 1 -- 172.21.15.7:0/1004573551 shutdown_connections 2024-09-13T18:52:11.101 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:11.094+0000 7f7bd5264700 1 --2- 172.21.15.7:0/1004573551 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7bbc0666a0 0x7f7bbc068b60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.101 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:11.094+0000 7f7bd5264700 1 --2- 172.21.15.7:0/1004573551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7bd01566e0 0x7f7bd0156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.101 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:11.094+0000 7f7bd5264700 1 --2- 172.21.15.7:0/1004573551 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bd014c7e0 0x7f7bd00afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.101 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:11.094+0000 7f7bd5264700 1 --2- 172.21.15.7:0/1004573551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bb8004db0 0x7f7bd009b5a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.101 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:11.094+0000 7f7bd5264700 1 -- 172.21.15.7:0/1004573551 >> 172.21.15.7:0/1004573551 conn(0x7f7bd00b0cd0 msgr2=0x7f7bd00b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:11.101 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:11.094+0000 7f7bd5264700 1 -- 172.21.15.7:0/1004573551 shutdown_connections 2024-09-13T18:52:11.101 INFO:teuthology.misc.health.smithi007.stderr:2024-09-13T18:52:11.094+0000 7f7bd5264700 1 -- 172.21.15.7:0/1004573551 wait complete. 2024-09-13T18:52:11.112 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-13T18:52:11.113 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T18:52:11.113 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T18:52:11.113 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T18:52:11.116 INFO:tasks.ceph.mds.a:Started 2024-09-13T18:52:11.116 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T18:52:11.116 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T18:52:11.118 INFO:tasks.ceph.mds.c:Started 2024-09-13T18:52:11.118 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T18:52:11.118 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T18:52:11.121 INFO:tasks.ceph.mds.b:Started 2024-09-13T18:52:11.121 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T18:52:11.122 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T18:52:11.124 INFO:tasks.ceph.mds.d:Started 2024-09-13T18:52:11.124 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-13T18:52:11.125 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:11.188 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-09-13T18:52:11.192 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-13T18:52:11.205 INFO:tasks.ceph.mds.d.smithi171.stdout:starting mds.d at 2024-09-13T18:52:11.208 INFO:tasks.ceph.mds.b.smithi171.stdout:starting mds.b at 2024-09-13T18:52:11.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.226+0000 7f45fb45b700 1 Processor -- start 2024-09-13T18:52:11.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.226+0000 7f45fb45b700 1 -- start start 2024-09-13T18:52:11.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.226+0000 7f45fb45b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f45f40cdcd0 0x7f45f415f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.226+0000 7f45fb45b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f415e570 con 0x7f45f40cd890 2024-09-13T18:52:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.226+0000 7f45fb45b700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f41614a0 con 0x7f45f41433f0 2024-09-13T18:52:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.226+0000 7f45fb45b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f415ebc0 con 0x7f45f40cdcd0 2024-09-13T18:52:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.226+0000 7f45f89f6700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f45f40cd890 0x7f45f415efe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52136/0 (socket says 172.21.15.7:52136) 2024-09-13T18:52:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.226+0000 7f45f89f6700 1 -- 172.21.15.7:0/1996529760 learned_addr learned my addr 172.21.15.7:0/1996529760 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.226+0000 7f45f99f8700 1 --2- 172.21.15.7:0/1996529760 >> v2:172.21.15.7:3300/0 conn(0x7f45f40cdcd0 0x7f45f415f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.226+0000 7f45ebfff700 1 -- 172.21.15.7:0/1996529760 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103470259 0 0) 0x7f45f415e570 con 0x7f45f40cd890 2024-09-13T18:52:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.226+0000 7f45ebfff700 1 -- 172.21.15.7:0/1996529760 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45e00035d0 con 0x7f45f40cd890 2024-09-13T18:52:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.226+0000 7f45f99f8700 1 -- 172.21.15.7:0/1996529760 >> v1:172.21.15.171:6789/0 conn(0x7f45f41433f0 legacy=0x7f45f4160a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:11.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.226+0000 7f45f99f8700 1 -- 172.21.15.7:0/1996529760 >> v1:172.21.15.7:6789/0 conn(0x7f45f40cd890 legacy=0x7f45f415efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:11.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.226+0000 7f45f99f8700 1 -- 172.21.15.7:0/1996529760 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f4161780 con 0x7f45f40cdcd0 2024-09-13T18:52:11.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.226+0000 7f45f99f8700 1 --2- 172.21.15.7:0/1996529760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45f40cdcd0 0x7f45f415f6f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009d90 tx=0x7f45ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8682025968cacfe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:11.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.230+0000 7f45ebfff700 1 -- 172.21.15.7:0/1996529760 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec00e070 con 0x7f45f40cdcd0 2024-09-13T18:52:11.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.230+0000 7f45ebfff700 1 -- 172.21.15.7:0/1996529760 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec00bb20 con 0x7f45f40cdcd0 2024-09-13T18:52:11.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.230+0000 7f45ebfff700 1 -- 172.21.15.7:0/1996529760 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec00be90 con 0x7f45f40cdcd0 2024-09-13T18:52:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.230+0000 7f45fb45b700 1 -- 172.21.15.7:0/1996529760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45f40cdcd0 msgr2=0x7f45f415f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.230+0000 7f45fb45b700 1 --2- 172.21.15.7:0/1996529760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45f40cdcd0 0x7f45f415f6f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009d90 tx=0x7f45ec00b330 comp rx=0 tx=0).stop 2024-09-13T18:52:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.230+0000 7f45fb45b700 1 -- 172.21.15.7:0/1996529760 shutdown_connections 2024-09-13T18:52:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.230+0000 7f45fb45b700 1 --2- 172.21.15.7:0/1996529760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45f40cdcd0 0x7f45f415f6f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.230+0000 7f45fb45b700 1 -- 172.21.15.7:0/1996529760 >> 172.21.15.7:0/1996529760 conn(0x7f45f40cb420 msgr2=0x7f45f40cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:11.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.230+0000 7f45fb45b700 1 -- 172.21.15.7:0/1996529760 shutdown_connections 2024-09-13T18:52:11.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.230+0000 7f45fb45b700 1 -- 172.21.15.7:0/1996529760 wait complete. 2024-09-13T18:52:11.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45fb45b700 1 Processor -- start 2024-09-13T18:52:11.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45fb45b700 1 -- start start 2024-09-13T18:52:11.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45fb45b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f45f40cd890 0x7f45f413dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:11.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45fb45b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45f40cdcd0 0x7f45f413c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:11.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45fb45b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45f41433f0 0x7f45f41f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:11.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45fb45b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f41445d0 con 0x7f45f41433f0 2024-09-13T18:52:11.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45fb45b700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f41439e0 con 0x7f45f40cdcd0 2024-09-13T18:52:11.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45fb45b700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f4143c30 con 0x7f45f40cd890 2024-09-13T18:52:11.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45f99f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45f41433f0 0x7f45f41f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:11.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45f91f7700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f45f40cd890 0x7f45f413dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:11.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45f99f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45f41433f0 0x7f45f41f29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55638/0 (socket says 172.21.15.7:55638) 2024-09-13T18:52:11.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45f99f8700 1 -- 172.21.15.7:0/2143828325 learned_addr learned my addr 172.21.15.7:0/2143828325 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:11.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45f89f6700 1 --2- 172.21.15.7:0/2143828325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45f40cdcd0 0x7f45f413c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:11.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45f99f8700 1 -- 172.21.15.7:0/2143828325 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f45f40cd890 msgr2=0x7f45f413dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45f99f8700 1 --2- 172.21.15.7:0/2143828325 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f45f40cd890 0x7f45f413dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45f99f8700 1 -- 172.21.15.7:0/2143828325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45f40cdcd0 msgr2=0x7f45f413c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45f99f8700 1 --2- 172.21.15.7:0/2143828325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45f40cdcd0 0x7f45f413c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45f99f8700 1 -- 172.21.15.7:0/2143828325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ec009a70 con 0x7f45f41433f0 2024-09-13T18:52:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45f99f8700 1 --2- 172.21.15.7:0/2143828325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45f41433f0 0x7f45f41f29e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f45f40b1690 tx=0x7f45ec004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45e9ffb700 1 -- 172.21.15.7:0/2143828325 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec004900 con 0x7f45f41433f0 2024-09-13T18:52:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45e9ffb700 1 -- 172.21.15.7:0/2143828325 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec012430 con 0x7f45f41433f0 2024-09-13T18:52:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45e9ffb700 1 -- 172.21.15.7:0/2143828325 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0127e0 con 0x7f45f41433f0 2024-09-13T18:52:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.234+0000 7f45fb45b700 1 -- 172.21.15.7:0/2143828325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f4143f10 con 0x7f45f41433f0 2024-09-13T18:52:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.238+0000 7f45fb45b700 1 -- 172.21.15.7:0/2143828325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f4146620 con 0x7f45f41433f0 2024-09-13T18:52:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.238+0000 7f45e9ffb700 1 -- 172.21.15.7:0/2143828325 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f45ec012940 con 0x7f45f41433f0 2024-09-13T18:52:11.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.238+0000 7f45fb45b700 1 -- 172.21.15.7:0/2143828325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f4144710 con 0x7f45f41433f0 2024-09-13T18:52:11.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.238+0000 7f45e9ffb700 1 --2- 172.21.15.7:0/2143828325 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f45d80666a0 0x7f45d8068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:11.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.238+0000 7f45e9ffb700 1 -- 172.21.15.7:0/2143828325 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f45ec085830 con 0x7f45f41433f0 2024-09-13T18:52:11.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.238+0000 7f45f91f7700 1 --2- 172.21.15.7:0/2143828325 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f45d80666a0 0x7f45d8068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:11.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.238+0000 7f45f91f7700 1 --2- 172.21.15.7:0/2143828325 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f45d80666a0 0x7f45d8068b60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f45f00018a0 tx=0x7f45f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:11.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.250+0000 7f45e9ffb700 1 -- 172.21.15.7:0/2143828325 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f45f4144710 con 0x7f45f41433f0 2024-09-13T18:52:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.422+0000 7f45fb45b700 1 -- 172.21.15.7:0/2143828325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45f4144710 con 0x7f45f41433f0 2024-09-13T18:52:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.422+0000 7f45e9ffb700 1 -- 172.21.15.7:0/2143828325 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10860 (secure 0 0 0) 0x7f45f4144710 con 0x7f45f41433f0 2024-09-13T18:52:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:11.428 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":14,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:10.708691+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.426+0000 7f45fb45b700 1 -- 172.21.15.7:0/2143828325 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f45d80666a0 msgr2=0x7f45d8068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.426+0000 7f45fb45b700 1 --2- 172.21.15.7:0/2143828325 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f45d80666a0 0x7f45d8068b60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f45f00018a0 tx=0x7f45f0009040 comp rx=0 tx=0).stop 2024-09-13T18:52:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.426+0000 7f45fb45b700 1 -- 172.21.15.7:0/2143828325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45f41433f0 msgr2=0x7f45f41f29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.426+0000 7f45fb45b700 1 --2- 172.21.15.7:0/2143828325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45f41433f0 0x7f45f41f29e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f45f40b1690 tx=0x7f45ec004af0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.426+0000 7f45fb45b700 1 -- 172.21.15.7:0/2143828325 shutdown_connections 2024-09-13T18:52:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.426+0000 7f45fb45b700 1 --2- 172.21.15.7:0/2143828325 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f45d80666a0 0x7f45d8068b60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.426+0000 7f45fb45b700 1 --2- 172.21.15.7:0/2143828325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45f41433f0 0x7f45f41f29e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.426+0000 7f45fb45b700 1 --2- 172.21.15.7:0/2143828325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f45f40cdcd0 0x7f45f413c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.426+0000 7f45fb45b700 1 --2- 172.21.15.7:0/2143828325 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f45f40cd890 0x7f45f413dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.426+0000 7f45fb45b700 1 -- 172.21.15.7:0/2143828325 >> 172.21.15.7:0/2143828325 conn(0x7f45f40cb420 msgr2=0x7f45f40b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.426+0000 7f45fb45b700 1 -- 172.21.15.7:0/2143828325 shutdown_connections 2024-09-13T18:52:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.426+0000 7f45fb45b700 1 -- 172.21.15.7:0/2143828325 wait complete. 2024-09-13T18:52:11.444 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-09-13T18:52:11.444 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:11.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.558+0000 7f95526bf700 1 Processor -- start 2024-09-13T18:52:11.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.558+0000 7f95526bf700 1 -- start start 2024-09-13T18:52:11.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f95526bf700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f954c159720 0x7f954c06cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:11.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f95526bf700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f954c06aa90 0x7f954c0b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f95526bf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f954c0b3910 0x7f954c070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f95526bf700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f954c0b3dc0 con 0x7f954c0b3910 2024-09-13T18:52:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f95526bf700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f954c06aed0 con 0x7f954c06aa90 2024-09-13T18:52:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f95526bf700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954c06d690 con 0x7f954c159720 2024-09-13T18:52:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f9551ebe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f954c0b3910 0x7f954c070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f9551ebe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f954c0b3910 0x7f954c070c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55640/0 (socket says 172.21.15.7:55640) 2024-09-13T18:52:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f9551ebe700 1 -- 172.21.15.7:0/1991596606 learned_addr learned my addr 172.21.15.7:0/1991596606 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f95516bd700 1 --2- 172.21.15.7:0/1991596606 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f954c159720 0x7f954c06cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f9550ebc700 1 --2- 172.21.15.7:0/1991596606 >> v2:172.21.15.171:3300/0 conn(0x7f954c06aa90 0x7f954c0b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f9551ebe700 1 -- 172.21.15.7:0/1991596606 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f954c159720 msgr2=0x7f954c06cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f9551ebe700 1 --2- 172.21.15.7:0/1991596606 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f954c159720 0x7f954c06cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f9551ebe700 1 -- 172.21.15.7:0/1991596606 >> v2:172.21.15.171:3300/0 conn(0x7f954c06aa90 msgr2=0x7f954c0b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f9551ebe700 1 --2- 172.21.15.7:0/1991596606 >> v2:172.21.15.171:3300/0 conn(0x7f954c06aa90 0x7f954c0b33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f9551ebe700 1 -- 172.21.15.7:0/1991596606 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c0cbfe0 con 0x7f954c0b3910 2024-09-13T18:52:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f9551ebe700 1 --2- 172.21.15.7:0/1991596606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c0b3910 0x7f954c070c40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f954c072080 tx=0x7f954400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fb0017bf9b5db9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f9543fff700 1 -- 172.21.15.7:0/1991596606 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954400e070 con 0x7f954c0b3910 2024-09-13T18:52:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f9543fff700 1 -- 172.21.15.7:0/1991596606 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954400b910 con 0x7f954c0b3910 2024-09-13T18:52:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f9543fff700 1 -- 172.21.15.7:0/1991596606 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954400bcc0 con 0x7f954c0b3910 2024-09-13T18:52:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f95526bf700 1 -- 172.21.15.7:0/1991596606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c0b3910 msgr2=0x7f954c070c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f95526bf700 1 --2- 172.21.15.7:0/1991596606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c0b3910 0x7f954c070c40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f954c072080 tx=0x7f954400b330 comp rx=0 tx=0).stop 2024-09-13T18:52:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f95526bf700 1 -- 172.21.15.7:0/1991596606 shutdown_connections 2024-09-13T18:52:11.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f95526bf700 1 --2- 172.21.15.7:0/1991596606 >> v2:172.21.15.171:3300/0 conn(0x7f954c06aa90 0x7f954c0b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f95526bf700 1 --2- 172.21.15.7:0/1991596606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c0b3910 0x7f954c070c40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f95526bf700 1 --2- 172.21.15.7:0/1991596606 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f954c159720 0x7f954c06cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f95526bf700 1 -- 172.21.15.7:0/1991596606 >> 172.21.15.7:0/1991596606 conn(0x7f954c0b0f00 msgr2=0x7f954c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f95526bf700 1 -- 172.21.15.7:0/1991596606 shutdown_connections 2024-09-13T18:52:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.562+0000 7f95526bf700 1 -- 172.21.15.7:0/1991596606 wait complete. 2024-09-13T18:52:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f95526bf700 1 Processor -- start 2024-09-13T18:52:11.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f95526bf700 1 -- start start 2024-09-13T18:52:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f95526bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c06aa90 0x7f954c136840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f95526bf700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f954c0b3910 0x7f954c136f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f95526bf700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f954c159720 0x7f954c15c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f95526bf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c1374d0 con 0x7f954c06aa90 2024-09-13T18:52:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f95526bf700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954c15cd90 con 0x7f954c0b3910 2024-09-13T18:52:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f95526bf700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f954c156460 con 0x7f954c159720 2024-09-13T18:52:11.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f95516bd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c06aa90 0x7f954c136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:11.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f9550ebc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f954c0b3910 0x7f954c136f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:11.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f95516bd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c06aa90 0x7f954c136840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55648/0 (socket says 172.21.15.7:55648) 2024-09-13T18:52:11.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f95516bd700 1 -- 172.21.15.7:0/3919339805 learned_addr learned my addr 172.21.15.7:0/3919339805 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:11.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f9551ebe700 1 --2- 172.21.15.7:0/3919339805 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f954c159720 0x7f954c15c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f95516bd700 1 -- 172.21.15.7:0/3919339805 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f954c159720 msgr2=0x7f954c15c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f95516bd700 1 --2- 172.21.15.7:0/3919339805 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f954c159720 0x7f954c15c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f95516bd700 1 -- 172.21.15.7:0/3919339805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f954c0b3910 msgr2=0x7f954c136f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f95516bd700 1 --2- 172.21.15.7:0/3919339805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f954c0b3910 0x7f954c136f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f95516bd700 1 -- 172.21.15.7:0/3919339805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9544009b30 con 0x7f954c06aa90 2024-09-13T18:52:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.566+0000 7f95516bd700 1 --2- 172.21.15.7:0/3919339805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c06aa90 0x7f954c136840 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f954800e840 tx=0x7f954800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.570+0000 7f9541ffb700 1 -- 172.21.15.7:0/3919339805 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95480095a0 con 0x7f954c06aa90 2024-09-13T18:52:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.570+0000 7f9541ffb700 1 -- 172.21.15.7:0/3919339805 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9548013680 con 0x7f954c06aa90 2024-09-13T18:52:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.570+0000 7f95526bf700 1 -- 172.21.15.7:0/3919339805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f954c156740 con 0x7f954c06aa90 2024-09-13T18:52:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.570+0000 7f9541ffb700 1 -- 172.21.15.7:0/3919339805 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95480139f0 con 0x7f954c06aa90 2024-09-13T18:52:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.570+0000 7f95526bf700 1 -- 172.21.15.7:0/3919339805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f954c1560a0 con 0x7f954c06aa90 2024-09-13T18:52:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.570+0000 7f95526bf700 1 -- 172.21.15.7:0/3919339805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954c06b200 con 0x7f954c06aa90 2024-09-13T18:52:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.570+0000 7f9541ffb700 1 -- 172.21.15.7:0/3919339805 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f9548013b50 con 0x7f954c06aa90 2024-09-13T18:52:11.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.574+0000 7f9541ffb700 1 --2- 172.21.15.7:0/3919339805 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f95300666f0 0x7f9530068bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:11.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.574+0000 7f9541ffb700 1 -- 172.21.15.7:0/3919339805 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f9548083990 con 0x7f954c06aa90 2024-09-13T18:52:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.574+0000 7f9550ebc700 1 --2- 172.21.15.7:0/3919339805 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f95300666f0 0x7f9530068bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.574+0000 7f9550ebc700 1 --2- 172.21.15.7:0/3919339805 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f95300666f0 0x7f9530068bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f954c1f4010 tx=0x7f953c009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.578+0000 7f9541ffb700 1 -- 172.21.15.7:0/3919339805 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9548051300 con 0x7f954c06aa90 2024-09-13T18:52:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.790+0000 7f95526bf700 1 -- 172.21.15.7:0/3919339805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f954c0b33d0 con 0x7f954c06aa90 2024-09-13T18:52:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.790+0000 7f9541ffb700 1 -- 172.21.15.7:0/3919339805 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1250 (secure 0 0 0) 0x7f9548021090 con 0x7f954c06aa90 2024-09-13T18:52:11.797 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:11.797 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/951828843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":951828843},{"type":"v1","addr":"172.21.15.7:6833","nonce":951828843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-09-13T18:52:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.794+0000 7f95526bf700 1 -- 172.21.15.7:0/3919339805 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f95300666f0 msgr2=0x7f9530068bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.794+0000 7f95526bf700 1 --2- 172.21.15.7:0/3919339805 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f95300666f0 0x7f9530068bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f954c1f4010 tx=0x7f953c009240 comp rx=0 tx=0).stop 2024-09-13T18:52:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.794+0000 7f95526bf700 1 -- 172.21.15.7:0/3919339805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c06aa90 msgr2=0x7f954c136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.794+0000 7f95526bf700 1 --2- 172.21.15.7:0/3919339805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c06aa90 0x7f954c136840 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f954800e840 tx=0x7f954800eb50 comp rx=0 tx=0).stop 2024-09-13T18:52:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.794+0000 7f95526bf700 1 -- 172.21.15.7:0/3919339805 shutdown_connections 2024-09-13T18:52:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.794+0000 7f95526bf700 1 --2- 172.21.15.7:0/3919339805 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f95300666f0 0x7f9530068bb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.794+0000 7f95526bf700 1 --2- 172.21.15.7:0/3919339805 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f954c159720 0x7f954c15c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.794+0000 7f95526bf700 1 --2- 172.21.15.7:0/3919339805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f954c0b3910 0x7f954c136f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.794+0000 7f95526bf700 1 --2- 172.21.15.7:0/3919339805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f954c06aa90 0x7f954c136840 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.794+0000 7f95526bf700 1 -- 172.21.15.7:0/3919339805 >> 172.21.15.7:0/3919339805 conn(0x7f954c0b0f00 msgr2=0x7f954c070140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.794+0000 7f95526bf700 1 -- 172.21.15.7:0/3919339805 shutdown_connections 2024-09-13T18:52:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:11.794+0000 7f95526bf700 1 -- 172.21.15.7:0/3919339805 wait complete. 2024-09-13T18:52:11.802 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 2 2024-09-13T18:52:13.813 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:13.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.930+0000 7f0ccabb1700 1 Processor -- start 2024-09-13T18:52:13.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.930+0000 7f0ccabb1700 1 -- start start 2024-09-13T18:52:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.930+0000 7f0ccabb1700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f0cc40722c0 0x7f0cc406c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.930+0000 7f0ccabb1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0cc40b29a0 0x7f0cc40b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.930+0000 7f0ccabb1700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc40b3710 con 0x7f0cc4153da0 2024-09-13T18:52:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.930+0000 7f0ccabb1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc415abb0 con 0x7f0cc40b29a0 2024-09-13T18:52:13.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.930+0000 7f0ccabb1700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc406be30 con 0x7f0cc40722c0 2024-09-13T18:52:13.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0cca3b0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0cc40b29a0 0x7f0cc40b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:13.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0cca3b0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0cc40b29a0 0x7f0cc40b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55650/0 (socket says 172.21.15.7:55650) 2024-09-13T18:52:13.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0cca3b0700 1 -- 172.21.15.7:0/2972810068 learned_addr learned my addr 172.21.15.7:0/2972810068 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:13.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0cc93ae700 1 --2- 172.21.15.7:0/2972810068 >> v2:172.21.15.171:3300/0 conn(0x7f0cc40722c0 0x7f0cc406c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:13.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0cca3b0700 1 -- 172.21.15.7:0/2972810068 >> v2:172.21.15.171:3300/0 conn(0x7f0cc40722c0 msgr2=0x7f0cc406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:13.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0cca3b0700 1 --2- 172.21.15.7:0/2972810068 >> v2:172.21.15.171:3300/0 conn(0x7f0cc40722c0 0x7f0cc406c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:13.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0cca3b0700 1 -- 172.21.15.7:0/2972810068 >> v1:172.21.15.171:6789/0 conn(0x7f0cc4153da0 legacy=0x7f0cc4071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:13.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0cca3b0700 1 -- 172.21.15.7:0/2972810068 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc406c110 con 0x7f0cc40b29a0 2024-09-13T18:52:13.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0cca3b0700 1 --2- 172.21.15.7:0/2972810068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cc40b29a0 0x7f0cc40b2dd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0cc406ddf0 tx=0x7f0cbc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c92cc65e108ad0a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:13.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0cc8bad700 1 -- 172.21.15.7:0/2972810068 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc00e070 con 0x7f0cc40b29a0 2024-09-13T18:52:13.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0cc8bad700 1 -- 172.21.15.7:0/2972810068 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cbc00b910 con 0x7f0cc40b29a0 2024-09-13T18:52:13.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0cc8bad700 1 -- 172.21.15.7:0/2972810068 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc00bcc0 con 0x7f0cc40b29a0 2024-09-13T18:52:13.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0ccabb1700 1 -- 172.21.15.7:0/2972810068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cc40b29a0 msgr2=0x7f0cc40b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:13.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0ccabb1700 1 --2- 172.21.15.7:0/2972810068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cc40b29a0 0x7f0cc40b2dd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0cc406ddf0 tx=0x7f0cbc00b330 comp rx=0 tx=0).stop 2024-09-13T18:52:13.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0ccabb1700 1 -- 172.21.15.7:0/2972810068 shutdown_connections 2024-09-13T18:52:13.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0ccabb1700 1 --2- 172.21.15.7:0/2972810068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cc40b29a0 0x7f0cc40b2dd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:13.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0ccabb1700 1 --2- 172.21.15.7:0/2972810068 >> v2:172.21.15.171:3300/0 conn(0x7f0cc40722c0 0x7f0cc406c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:13.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0ccabb1700 1 -- 172.21.15.7:0/2972810068 >> 172.21.15.7:0/2972810068 conn(0x7f0cc40cf910 msgr2=0x7f0cc40cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:13.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0ccabb1700 1 -- 172.21.15.7:0/2972810068 shutdown_connections 2024-09-13T18:52:13.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.934+0000 7f0ccabb1700 1 -- 172.21.15.7:0/2972810068 wait complete. 2024-09-13T18:52:13.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0ccabb1700 1 Processor -- start 2024-09-13T18:52:13.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0ccabb1700 1 -- start start 2024-09-13T18:52:13.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0ccabb1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cc40722c0 0x7f0cc415b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:13.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0ccabb1700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0cc40b29a0 0x7f0cc415d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:13.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0ccabb1700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cc4153da0 0x7f0cc41f0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:13.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0ccabb1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc415d670 con 0x7f0cc40722c0 2024-09-13T18:52:13.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0ccabb1700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc40ae3f0 con 0x7f0cc4153da0 2024-09-13T18:52:13.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0ccabb1700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc415be00 con 0x7f0cc40b29a0 2024-09-13T18:52:13.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0cc9baf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cc40722c0 0x7f0cc415b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:13.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0cc9baf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cc40722c0 0x7f0cc415b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55666/0 (socket says 172.21.15.7:55666) 2024-09-13T18:52:13.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0cc9baf700 1 -- 172.21.15.7:0/4223115462 learned_addr learned my addr 172.21.15.7:0/4223115462 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:13.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0cc93ae700 1 --2- 172.21.15.7:0/4223115462 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0cc40b29a0 0x7f0cc415d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:13.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0cca3b0700 1 --2- 172.21.15.7:0/4223115462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cc4153da0 0x7f0cc41f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:13.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0cca3b0700 1 -- 172.21.15.7:0/4223115462 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0cc40b29a0 msgr2=0x7f0cc415d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:13.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0cca3b0700 1 --2- 172.21.15.7:0/4223115462 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0cc40b29a0 0x7f0cc415d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:13.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0cca3b0700 1 -- 172.21.15.7:0/4223115462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cc40722c0 msgr2=0x7f0cc415b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:13.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0cca3b0700 1 --2- 172.21.15.7:0/4223115462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cc40722c0 0x7f0cc415b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:13.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0cca3b0700 1 -- 172.21.15.7:0/4223115462 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc009b30 con 0x7f0cc4153da0 2024-09-13T18:52:13.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0cc9baf700 1 --2- 172.21.15.7:0/4223115462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cc40722c0 0x7f0cc415b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:13.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.938+0000 7f0cca3b0700 1 --2- 172.21.15.7:0/4223115462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cc4153da0 0x7f0cc41f0430 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc015000 tx=0x7f0cbc004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:13.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.942+0000 7f0cb67fc700 1 -- 172.21.15.7:0/4223115462 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc004880 con 0x7f0cc4153da0 2024-09-13T18:52:13.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.942+0000 7f0cb67fc700 1 -- 172.21.15.7:0/4223115462 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cbc0092e0 con 0x7f0cc4153da0 2024-09-13T18:52:13.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.942+0000 7f0ccabb1700 1 -- 172.21.15.7:0/4223115462 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc415c0e0 con 0x7f0cc4153da0 2024-09-13T18:52:13.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.942+0000 7f0cb67fc700 1 -- 172.21.15.7:0/4223115462 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc01c660 con 0x7f0cc4153da0 2024-09-13T18:52:13.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.942+0000 7f0ccabb1700 1 -- 172.21.15.7:0/4223115462 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc415c6a0 con 0x7f0cc4153da0 2024-09-13T18:52:13.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.942+0000 7f0ccabb1700 1 -- 172.21.15.7:0/4223115462 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc415c830 con 0x7f0cc4153da0 2024-09-13T18:52:13.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.942+0000 7f0cb67fc700 1 -- 172.21.15.7:0/4223115462 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f0cbc01c7c0 con 0x7f0cc4153da0 2024-09-13T18:52:13.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.942+0000 7f0cb67fc700 1 --2- 172.21.15.7:0/4223115462 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0ca80666a0 0x7f0ca8068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:13.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.946+0000 7f0cb67fc700 1 -- 172.21.15.7:0/4223115462 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4846+0+0 (secure 0 0 0) 0x7f0cbc085490 con 0x7f0cc4153da0 2024-09-13T18:52:13.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.946+0000 7f0cc9baf700 1 --2- 172.21.15.7:0/4223115462 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0ca80666a0 0x7f0ca8068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:13.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.946+0000 7f0cc9baf700 1 --2- 172.21.15.7:0/4223115462 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0ca80666a0 0x7f0ca8068b60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0cc41ee020 tx=0x7f0cb8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:13.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:13.958+0000 7f0cb67fc700 1 -- 172.21.15.7:0/4223115462 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0cbc052cd0 con 0x7f0cc4153da0 2024-09-13T18:52:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.166+0000 7f0ccabb1700 1 -- 172.21.15.7:0/4223115462 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cc406c910 con 0x7f0cc4153da0 2024-09-13T18:52:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.166+0000 7f0cb67fc700 1 -- 172.21.15.7:0/4223115462 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3521 (secure 0 0 0) 0x7f0cbc0562f0 con 0x7f0cc4153da0 2024-09-13T18:52:14.172 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:14.172 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1309419125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1309419125},{"type":"v1","addr":"172.21.15.7:6835","nonce":1309419125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/951828843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":951828843},{"type":"v1","addr":"172.21.15.7:6833","nonce":951828843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4226627580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4226627580},{"type":"v1","addr":"172.21.15.171:6835","nonce":4226627580}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3658296803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3658296803},{"type":"v1","addr":"172.21.15.171:6837","nonce":3658296803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-09-13T18:52:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.170+0000 7f0ccabb1700 1 -- 172.21.15.7:0/4223115462 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0ca80666a0 msgr2=0x7f0ca8068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.170+0000 7f0ccabb1700 1 --2- 172.21.15.7:0/4223115462 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0ca80666a0 0x7f0ca8068b60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0cc41ee020 tx=0x7f0cb8009040 comp rx=0 tx=0).stop 2024-09-13T18:52:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.170+0000 7f0ccabb1700 1 -- 172.21.15.7:0/4223115462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cc4153da0 msgr2=0x7f0cc41f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.170+0000 7f0ccabb1700 1 --2- 172.21.15.7:0/4223115462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cc4153da0 0x7f0cc41f0430 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc015000 tx=0x7f0cbc004af0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.170+0000 7f0ccabb1700 1 -- 172.21.15.7:0/4223115462 shutdown_connections 2024-09-13T18:52:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.170+0000 7f0ccabb1700 1 --2- 172.21.15.7:0/4223115462 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0ca80666a0 0x7f0ca8068b60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.170+0000 7f0ccabb1700 1 --2- 172.21.15.7:0/4223115462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0cc4153da0 0x7f0cc41f0430 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.170+0000 7f0ccabb1700 1 --2- 172.21.15.7:0/4223115462 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0cc40b29a0 0x7f0cc415d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.170+0000 7f0ccabb1700 1 --2- 172.21.15.7:0/4223115462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cc40722c0 0x7f0cc415b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.170+0000 7f0ccabb1700 1 -- 172.21.15.7:0/4223115462 >> 172.21.15.7:0/4223115462 conn(0x7f0cc40cf910 msgr2=0x7f0cc4159e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:14.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.170+0000 7f0ccabb1700 1 -- 172.21.15.7:0/4223115462 shutdown_connections 2024-09-13T18:52:14.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.170+0000 7f0ccabb1700 1 -- 172.21.15.7:0/4223115462 wait complete. 2024-09-13T18:52:14.176 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 3 2024-09-13T18:52:14.186 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.298+0000 7fabf5f39700 1 Processor -- start 2024-09-13T18:52:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.298+0000 7fabf5f39700 1 -- start start 2024-09-13T18:52:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.298+0000 7fabf5f39700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabf00cb130 0x7fabf015d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.298+0000 7fabf5f39700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabf00b33d0 0x7fabf00b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.298+0000 7fabf5f39700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf00cc860 con 0x7fabf015d640 2024-09-13T18:52:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.298+0000 7fabf5f39700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf00713f0 con 0x7fabf00b33d0 2024-09-13T18:52:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.298+0000 7fabf5f39700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf00b3dc0 con 0x7fabf00cb130 2024-09-13T18:52:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.298+0000 7fabf5738700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabf00b33d0 0x7fabf00b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.298+0000 7fabeffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fabf015d640 0x7fabf0071810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52142/0 (socket says 172.21.15.7:52142) 2024-09-13T18:52:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.298+0000 7fabeffff700 1 -- 172.21.15.7:0/3723790956 learned_addr learned my addr 172.21.15.7:0/3723790956 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:14.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.298+0000 7fabf4f37700 1 --2- 172.21.15.7:0/3723790956 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabf00cb130 0x7fabf015d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:14.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.298+0000 7fabf5738700 1 -- 172.21.15.7:0/3723790956 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabf00cb130 msgr2=0x7fabf015d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.298+0000 7fabf5738700 1 --2- 172.21.15.7:0/3723790956 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabf00cb130 0x7fabf015d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.298+0000 7fabf5738700 1 -- 172.21.15.7:0/3723790956 >> v1:172.21.15.7:6789/0 conn(0x7fabf015d640 legacy=0x7fabf0071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.298+0000 7fabef7fe700 1 -- 172.21.15.7:0/3723790956 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510294184 0 0) 0x7fabf00cc860 con 0x7fabf015d640 2024-09-13T18:52:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.298+0000 7fabf5738700 1 -- 172.21.15.7:0/3723790956 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf00ce0e0 con 0x7fabf00b33d0 2024-09-13T18:52:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.298+0000 7fabf5738700 1 --2- 172.21.15.7:0/3723790956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf00b33d0 0x7fabf00b3880 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009850 tx=0x7fabe8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80528c16558e0f87 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.302+0000 7fabef7fe700 1 -- 172.21.15.7:0/3723790956 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8011830 con 0x7fabf00b33d0 2024-09-13T18:52:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.302+0000 7fabef7fe700 1 -- 172.21.15.7:0/3723790956 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe8011990 con 0x7fabf00b33d0 2024-09-13T18:52:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.302+0000 7fabef7fe700 1 -- 172.21.15.7:0/3723790956 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8011d40 con 0x7fabf00b33d0 2024-09-13T18:52:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.302+0000 7fabf5f39700 1 -- 172.21.15.7:0/3723790956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf00b33d0 msgr2=0x7fabf00b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.302+0000 7fabf5f39700 1 --2- 172.21.15.7:0/3723790956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf00b33d0 0x7fabf00b3880 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009850 tx=0x7fabe8009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.302+0000 7fabf5f39700 1 -- 172.21.15.7:0/3723790956 shutdown_connections 2024-09-13T18:52:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.302+0000 7fabf5f39700 1 --2- 172.21.15.7:0/3723790956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf00b33d0 0x7fabf00b3880 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.302+0000 7fabf5f39700 1 --2- 172.21.15.7:0/3723790956 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabf00cb130 0x7fabf015d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.302+0000 7fabf5f39700 1 -- 172.21.15.7:0/3723790956 >> 172.21.15.7:0/3723790956 conn(0x7fabf00b0f00 msgr2=0x7fabf00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.302+0000 7fabf5f39700 1 -- 172.21.15.7:0/3723790956 shutdown_connections 2024-09-13T18:52:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.302+0000 7fabf5f39700 1 -- 172.21.15.7:0/3723790956 wait complete. 2024-09-13T18:52:14.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.302+0000 7fabf5f39700 1 Processor -- start 2024-09-13T18:52:14.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.302+0000 7fabf5f39700 1 -- start start 2024-09-13T18:52:14.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabf5f39700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabf00b33d0 0x7fabf0138b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:14.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabf5f39700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabf00cb130 0x7fabf00c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:14.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabf5f39700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf015d640 0x7fabf015c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:14.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabf5f39700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf00c6250 con 0x7fabf015d640 2024-09-13T18:52:14.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabf5f39700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf00c64d0 con 0x7fabf00cb130 2024-09-13T18:52:14.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabf5f39700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf00c67b0 con 0x7fabf00b33d0 2024-09-13T18:52:14.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabf5738700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf015d640 0x7fabf015c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:14.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabf5738700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf015d640 0x7fabf015c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55690/0 (socket says 172.21.15.7:55690) 2024-09-13T18:52:14.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabf5738700 1 -- 172.21.15.7:0/2290082341 learned_addr learned my addr 172.21.15.7:0/2290082341 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:14.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabf4f37700 1 --2- 172.21.15.7:0/2290082341 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabf00b33d0 0x7fabf0138b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:14.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabeffff700 1 --2- 172.21.15.7:0/2290082341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabf00cb130 0x7fabf00c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:14.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabf5738700 1 -- 172.21.15.7:0/2290082341 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabf00b33d0 msgr2=0x7fabf0138b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:14.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabf5738700 1 --2- 172.21.15.7:0/2290082341 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabf00b33d0 0x7fabf0138b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabf5738700 1 -- 172.21.15.7:0/2290082341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabf00cb130 msgr2=0x7fabf00c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:14.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabf5738700 1 --2- 172.21.15.7:0/2290082341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabf00cb130 0x7fabf00c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabf5738700 1 -- 172.21.15.7:0/2290082341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe80094b0 con 0x7fabf015d640 2024-09-13T18:52:14.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabf5738700 1 --2- 172.21.15.7:0/2290082341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf015d640 0x7fabf015c7a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009ba0 tx=0x7fabe8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:14.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabed7fa700 1 -- 172.21.15.7:0/2290082341 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8004550 con 0x7fabf015d640 2024-09-13T18:52:14.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabed7fa700 1 -- 172.21.15.7:0/2290082341 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe801f070 con 0x7fabf015d640 2024-09-13T18:52:14.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabf5f39700 1 -- 172.21.15.7:0/2290082341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf00c6a90 con 0x7fabf015d640 2024-09-13T18:52:14.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabed7fa700 1 -- 172.21.15.7:0/2290082341 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8003900 con 0x7fabf015d640 2024-09-13T18:52:14.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.306+0000 7fabf5f39700 1 -- 172.21.15.7:0/2290082341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf00c7050 con 0x7fabf015d640 2024-09-13T18:52:14.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.310+0000 7fabf5f39700 1 -- 172.21.15.7:0/2290082341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf00cca20 con 0x7fabf015d640 2024-09-13T18:52:14.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.310+0000 7fabed7fa700 1 -- 172.21.15.7:0/2290082341 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7fabe8003a60 con 0x7fabf015d640 2024-09-13T18:52:14.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.310+0000 7fabed7fa700 1 --2- 172.21.15.7:0/2290082341 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fabdc0666a0 0x7fabdc068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:14.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.310+0000 7fabed7fa700 1 -- 172.21.15.7:0/2290082341 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fabe8021070 con 0x7fabf015d640 2024-09-13T18:52:14.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.310+0000 7fabf4f37700 1 --2- 172.21.15.7:0/2290082341 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fabdc0666a0 0x7fabdc068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:14.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.314+0000 7fabf4f37700 1 --2- 172.21.15.7:0/2290082341 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fabdc0666a0 0x7fabdc068b60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009bf0 tx=0x7fabe4009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:14.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.318+0000 7fabed7fa700 1 -- 172.21.15.7:0/2290082341 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fabe8052ab0 con 0x7fabf015d640 2024-09-13T18:52:14.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.494+0000 7fabf5f39700 1 -- 172.21.15.7:0/2290082341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabf00ce0e0 con 0x7fabf015d640 2024-09-13T18:52:14.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.494+0000 7fabed7fa700 1 -- 172.21.15.7:0/2290082341 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12815 (secure 0 0 0) 0x7fabf00ce0e0 con 0x7fabf015d640 2024-09-13T18:52:14.500 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:14.501 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:14.203896+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:14.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.498+0000 7fabf5f39700 1 -- 172.21.15.7:0/2290082341 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fabdc0666a0 msgr2=0x7fabdc068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:14.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.498+0000 7fabf5f39700 1 --2- 172.21.15.7:0/2290082341 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fabdc0666a0 0x7fabdc068b60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009bf0 tx=0x7fabe4009240 comp rx=0 tx=0).stop 2024-09-13T18:52:14.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.498+0000 7fabf5f39700 1 -- 172.21.15.7:0/2290082341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf015d640 msgr2=0x7fabf015c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:14.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.498+0000 7fabf5f39700 1 --2- 172.21.15.7:0/2290082341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf015d640 0x7fabf015c7a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009ba0 tx=0x7fabe8004770 comp rx=0 tx=0).stop 2024-09-13T18:52:14.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.498+0000 7fabf5f39700 1 -- 172.21.15.7:0/2290082341 shutdown_connections 2024-09-13T18:52:14.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.498+0000 7fabf5f39700 1 --2- 172.21.15.7:0/2290082341 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fabdc0666a0 0x7fabdc068b60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.498+0000 7fabf5f39700 1 --2- 172.21.15.7:0/2290082341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabf015d640 0x7fabf015c7a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.498+0000 7fabf5f39700 1 --2- 172.21.15.7:0/2290082341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabf00cb130 0x7fabf00c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.498+0000 7fabf5f39700 1 --2- 172.21.15.7:0/2290082341 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabf00b33d0 0x7fabf0138b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.498+0000 7fabf5f39700 1 -- 172.21.15.7:0/2290082341 >> 172.21.15.7:0/2290082341 conn(0x7fabf00b0f00 msgr2=0x7fabf015db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:14.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.498+0000 7fabf5f39700 1 -- 172.21.15.7:0/2290082341 shutdown_connections 2024-09-13T18:52:14.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.498+0000 7fabf5f39700 1 -- 172.21.15.7:0/2290082341 wait complete. 2024-09-13T18:52:14.515 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:52:14.516 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:52:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c3342700 1 Processor -- start 2024-09-13T18:52:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c3342700 1 -- start start 2024-09-13T18:52:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c3342700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06bc15cda0 0x7f06bc0d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c3342700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f06bc0b5e00 0x7f06bc0b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c3342700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06bc0b6780 0x7f06bc06a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c3342700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06bc0cde50 con 0x7f06bc15cda0 2024-09-13T18:52:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c3342700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06bc0d0880 con 0x7f06bc0b5e00 2024-09-13T18:52:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c3342700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0d16e0 con 0x7f06bc0b6780 2024-09-13T18:52:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c10de700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06bc15cda0 0x7f06bc0d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c10de700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06bc15cda0 0x7f06bc0d1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55692/0 (socket says 172.21.15.7:55692) 2024-09-13T18:52:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c10de700 1 -- 172.21.15.7:0/4050526276 learned_addr learned my addr 172.21.15.7:0/4050526276 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c18df700 1 --2- 172.21.15.7:0/4050526276 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06bc0b6780 0x7f06bc06a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c08dd700 1 --2- 172.21.15.7:0/4050526276 >> v2:172.21.15.171:3300/0 conn(0x7f06bc0b5e00 0x7f06bc0b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c10de700 1 -- 172.21.15.7:0/4050526276 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06bc0b6780 msgr2=0x7f06bc06a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c10de700 1 --2- 172.21.15.7:0/4050526276 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06bc0b6780 0x7f06bc06a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c10de700 1 -- 172.21.15.7:0/4050526276 >> v2:172.21.15.171:3300/0 conn(0x7f06bc0b5e00 msgr2=0x7f06bc0b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c10de700 1 --2- 172.21.15.7:0/4050526276 >> v2:172.21.15.171:3300/0 conn(0x7f06bc0b5e00 0x7f06bc0b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c10de700 1 -- 172.21.15.7:0/4050526276 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06bc0d0ad0 con 0x7f06bc15cda0 2024-09-13T18:52:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c18df700 1 --2- 172.21.15.7:0/4050526276 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06bc0b6780 0x7f06bc06a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.638+0000 7f06c10de700 1 --2- 172.21.15.7:0/4050526276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06bc15cda0 0x7f06bc0d1b70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f06bc06c470 tx=0x7f06b0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549a3ec0e2b4442b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.642+0000 7f06affff700 1 -- 172.21.15.7:0/4050526276 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b000b8a0 con 0x7f06bc15cda0 2024-09-13T18:52:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.642+0000 7f06affff700 1 -- 172.21.15.7:0/4050526276 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b000ba00 con 0x7f06bc15cda0 2024-09-13T18:52:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.642+0000 7f06affff700 1 -- 172.21.15.7:0/4050526276 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b000bd90 con 0x7f06bc15cda0 2024-09-13T18:52:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.642+0000 7f06c3342700 1 -- 172.21.15.7:0/4050526276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06bc15cda0 msgr2=0x7f06bc0d1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.642+0000 7f06c3342700 1 --2- 172.21.15.7:0/4050526276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06bc15cda0 0x7f06bc0d1b70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f06bc06c470 tx=0x7f06b0009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.642+0000 7f06c3342700 1 -- 172.21.15.7:0/4050526276 shutdown_connections 2024-09-13T18:52:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.642+0000 7f06c3342700 1 --2- 172.21.15.7:0/4050526276 >> v2:172.21.15.171:3300/0 conn(0x7f06bc0b5e00 0x7f06bc0b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.642+0000 7f06c3342700 1 --2- 172.21.15.7:0/4050526276 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06bc0b6780 0x7f06bc06a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.642+0000 7f06c3342700 1 --2- 172.21.15.7:0/4050526276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06bc15cda0 0x7f06bc0d1b70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.642+0000 7f06c3342700 1 -- 172.21.15.7:0/4050526276 >> 172.21.15.7:0/4050526276 conn(0x7f06bc0d2cb0 msgr2=0x7f06bc0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.642+0000 7f06c3342700 1 -- 172.21.15.7:0/4050526276 shutdown_connections 2024-09-13T18:52:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.642+0000 7f06c3342700 1 -- 172.21.15.7:0/4050526276 wait complete. 2024-09-13T18:52:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.642+0000 7f06c3342700 1 Processor -- start 2024-09-13T18:52:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.642+0000 7f06c3342700 1 -- start start 2024-09-13T18:52:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c3342700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06bc0b5e00 0x7f06bc0cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c3342700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06bc0b6780 0x7f06bc153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c3342700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06bc15cda0 0x7f06bc1f5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c3342700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0cbf10 con 0x7f06bc0b6780 2024-09-13T18:52:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c3342700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0cb300 con 0x7f06bc0b5e00 2024-09-13T18:52:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c3342700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06bc0cb5e0 con 0x7f06bc15cda0 2024-09-13T18:52:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c08dd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06bc0b6780 0x7f06bc153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c08dd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06bc0b6780 0x7f06bc153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55706/0 (socket says 172.21.15.7:55706) 2024-09-13T18:52:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c08dd700 1 -- 172.21.15.7:0/4292503533 learned_addr learned my addr 172.21.15.7:0/4292503533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c10de700 1 --2- 172.21.15.7:0/4292503533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06bc0b5e00 0x7f06bc0cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c18df700 1 --2- 172.21.15.7:0/4292503533 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06bc15cda0 0x7f06bc1f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:14.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c08dd700 1 -- 172.21.15.7:0/4292503533 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06bc15cda0 msgr2=0x7f06bc1f5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:14.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c08dd700 1 --2- 172.21.15.7:0/4292503533 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06bc15cda0 0x7f06bc1f5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c08dd700 1 -- 172.21.15.7:0/4292503533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06bc0b5e00 msgr2=0x7f06bc0cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:14.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c08dd700 1 --2- 172.21.15.7:0/4292503533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06bc0b5e00 0x7f06bc0cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c08dd700 1 -- 172.21.15.7:0/4292503533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b0009570 con 0x7f06bc0b6780 2024-09-13T18:52:14.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c08dd700 1 --2- 172.21.15.7:0/4292503533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06bc0b6780 0x7f06bc153a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f06b800d800 tx=0x7f06b800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:14.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06adffb700 1 -- 172.21.15.7:0/4292503533 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b80125f0 con 0x7f06bc0b6780 2024-09-13T18:52:14.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06adffb700 1 -- 172.21.15.7:0/4292503533 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b8012750 con 0x7f06bc0b6780 2024-09-13T18:52:14.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06adffb700 1 -- 172.21.15.7:0/4292503533 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8012ae0 con 0x7f06bc0b6780 2024-09-13T18:52:14.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c3342700 1 -- 172.21.15.7:0/4292503533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06bc0cb8c0 con 0x7f06bc0b6780 2024-09-13T18:52:14.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.646+0000 7f06c3342700 1 -- 172.21.15.7:0/4292503533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06bc153fc0 con 0x7f06bc0b6780 2024-09-13T18:52:14.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.650+0000 7f06c3342700 1 -- 172.21.15.7:0/4292503533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06bc0cc080 con 0x7f06bc0b6780 2024-09-13T18:52:14.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.650+0000 7f06adffb700 1 -- 172.21.15.7:0/4292503533 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f06b8012c40 con 0x7f06bc0b6780 2024-09-13T18:52:14.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.650+0000 7f06adffb700 1 --2- 172.21.15.7:0/4292503533 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f06a00666a0 0x7f06a0068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:14.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.650+0000 7f06adffb700 1 -- 172.21.15.7:0/4292503533 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f06b8083fc0 con 0x7f06bc0b6780 2024-09-13T18:52:14.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.650+0000 7f06c10de700 1 --2- 172.21.15.7:0/4292503533 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f06a00666a0 0x7f06a0068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:14.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.654+0000 7f06c10de700 1 --2- 172.21.15.7:0/4292503533 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f06a00666a0 0x7f06a0068b60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f06b0000c00 tx=0x7f06b0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:14.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.658+0000 7f06adffb700 1 -- 172.21.15.7:0/4292503533 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f06b80516c0 con 0x7f06bc0b6780 2024-09-13T18:52:14.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.830+0000 7f06c3342700 1 -- 172.21.15.7:0/4292503533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f06bc0b6240 con 0x7f06bc0b6780 2024-09-13T18:52:14.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.830+0000 7f06adffb700 1 -- 172.21.15.7:0/4292503533 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f06b800b440 con 0x7f06bc0b6780 2024-09-13T18:52:14.835 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:52:14.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.834+0000 7f06c3342700 1 -- 172.21.15.7:0/4292503533 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f06a00666a0 msgr2=0x7f06a0068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:14.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.834+0000 7f06c3342700 1 --2- 172.21.15.7:0/4292503533 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f06a00666a0 0x7f06a0068b60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f06b0000c00 tx=0x7f06b0008030 comp rx=0 tx=0).stop 2024-09-13T18:52:14.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.834+0000 7f06c3342700 1 -- 172.21.15.7:0/4292503533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06bc0b6780 msgr2=0x7f06bc153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:14.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.834+0000 7f06c3342700 1 --2- 172.21.15.7:0/4292503533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06bc0b6780 0x7f06bc153a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f06b800d800 tx=0x7f06b800db10 comp rx=0 tx=0).stop 2024-09-13T18:52:14.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.834+0000 7f06c3342700 1 -- 172.21.15.7:0/4292503533 shutdown_connections 2024-09-13T18:52:14.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.834+0000 7f06c3342700 1 --2- 172.21.15.7:0/4292503533 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f06a00666a0 0x7f06a0068b60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.834+0000 7f06c3342700 1 --2- 172.21.15.7:0/4292503533 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06bc15cda0 0x7f06bc1f5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.834+0000 7f06c3342700 1 --2- 172.21.15.7:0/4292503533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06bc0b6780 0x7f06bc153a80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.834+0000 7f06c3342700 1 --2- 172.21.15.7:0/4292503533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06bc0b5e00 0x7f06bc0cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.834+0000 7f06c3342700 1 -- 172.21.15.7:0/4292503533 >> 172.21.15.7:0/4292503533 conn(0x7f06bc0d2cb0 msgr2=0x7f06bc0d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:14.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.834+0000 7f06c3342700 1 -- 172.21.15.7:0/4292503533 shutdown_connections 2024-09-13T18:52:14.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.834+0000 7f06c3342700 1 -- 172.21.15.7:0/4292503533 wait complete. 2024-09-13T18:52:14.850 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T18:52:14.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.958+0000 7f0c19d93700 1 Processor -- start 2024-09-13T18:52:14.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.958+0000 7f0c19d93700 1 -- start start 2024-09-13T18:52:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c19d93700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0c0c00bb60 0x7f0c0c04eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c19d93700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c0c0e0d20 con 0x7f0c0c005b60 2024-09-13T18:52:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c19d93700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c0c0e0110 con 0x7f0c0c0df9c0 2024-09-13T18:52:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c19d93700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c0c0e03f0 con 0x7f0c0c00bb60 2024-09-13T18:52:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c13fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0c0c005b60 0x7f0c0c00b450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52154/0 (socket says 172.21.15.7:52154) 2024-09-13T18:52:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c13fff700 1 -- 172.21.15.7:0/1952746358 learned_addr learned my addr 172.21.15.7:0/1952746358 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c19592700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0c0c00bb60 0x7f0c0c04eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c137fe700 1 -- 172.21.15.7:0/1952746358 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160847362 0 0) 0x7f0c0c0e0d20 con 0x7f0c0c005b60 2024-09-13T18:52:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c137fe700 1 -- 172.21.15.7:0/1952746358 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bfc0035d0 con 0x7f0c0c005b60 2024-09-13T18:52:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c19592700 1 -- 172.21.15.7:0/1952746358 >> v1:172.21.15.171:6789/0 conn(0x7f0c0c0df9c0 legacy=0x7f0c0c01c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:14.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c19592700 1 -- 172.21.15.7:0/1952746358 >> v1:172.21.15.7:6789/0 conn(0x7f0c0c005b60 legacy=0x7f0c0c00b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:14.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c19592700 1 -- 172.21.15.7:0/1952746358 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c0e06d0 con 0x7f0c0c00bb60 2024-09-13T18:52:14.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c19592700 1 --2- 172.21.15.7:0/1952746358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c00bb60 0x7f0c0c04eef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0c1404f480 tx=0x7f0c14074840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565a9b245395e53a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:14.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c137fe700 1 -- 172.21.15.7:0/1952746358 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1404f010 con 0x7f0c0c00bb60 2024-09-13T18:52:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c137fe700 1 -- 172.21.15.7:0/1952746358 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c140730f0 con 0x7f0c0c00bb60 2024-09-13T18:52:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c137fe700 1 -- 172.21.15.7:0/1952746358 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c140734c0 con 0x7f0c0c00bb60 2024-09-13T18:52:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c19d93700 1 -- 172.21.15.7:0/1952746358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c00bb60 msgr2=0x7f0c0c04eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c19d93700 1 --2- 172.21.15.7:0/1952746358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c00bb60 0x7f0c0c04eef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0c1404f480 tx=0x7f0c14074840 comp rx=0 tx=0).stop 2024-09-13T18:52:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c19d93700 1 -- 172.21.15.7:0/1952746358 shutdown_connections 2024-09-13T18:52:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c19d93700 1 --2- 172.21.15.7:0/1952746358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c00bb60 0x7f0c0c04eef0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c19d93700 1 -- 172.21.15.7:0/1952746358 >> 172.21.15.7:0/1952746358 conn(0x7f0c0c036140 msgr2=0x7f0c0c036550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c19d93700 1 -- 172.21.15.7:0/1952746358 shutdown_connections 2024-09-13T18:52:14.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c19d93700 1 -- 172.21.15.7:0/1952746358 wait complete. 2024-09-13T18:52:14.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c19d93700 1 Processor -- start 2024-09-13T18:52:14.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.962+0000 7f0c19d93700 1 -- start start 2024-09-13T18:52:14.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c19d93700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0c0c005b60 0x7f0c0c032b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:14.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c19d93700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c00bb60 0x7f0c0c0d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:14.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c19d93700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c0c0df9c0 0x7f0c0c0d2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:14.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c19d93700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c0d1a80 con 0x7f0c0c00bb60 2024-09-13T18:52:14.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c19d93700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c0d3960 con 0x7f0c0c0df9c0 2024-09-13T18:52:14.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c19d93700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c0d1750 con 0x7f0c0c005b60 2024-09-13T18:52:14.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c13fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c00bb60 0x7f0c0c0d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:14.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c13fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c00bb60 0x7f0c0c0d1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55720/0 (socket says 172.21.15.7:55720) 2024-09-13T18:52:14.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c13fff700 1 -- 172.21.15.7:0/972381286 learned_addr learned my addr 172.21.15.7:0/972381286 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:14.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c19592700 1 --2- 172.21.15.7:0/972381286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c0c0df9c0 0x7f0c0c0d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:14.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c18d91700 1 --2- 172.21.15.7:0/972381286 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0c0c005b60 0x7f0c0c032b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:14.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c13fff700 1 -- 172.21.15.7:0/972381286 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0c0c005b60 msgr2=0x7f0c0c032b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:14.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c13fff700 1 --2- 172.21.15.7:0/972381286 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0c0c005b60 0x7f0c0c032b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c13fff700 1 -- 172.21.15.7:0/972381286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c0c0df9c0 msgr2=0x7f0c0c0d2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:14.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c13fff700 1 --2- 172.21.15.7:0/972381286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c0c0df9c0 0x7f0c0c0d2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:14.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c13fff700 1 -- 172.21.15.7:0/972381286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1404f5c0 con 0x7f0c0c00bb60 2024-09-13T18:52:14.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c13fff700 1 --2- 172.21.15.7:0/972381286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c00bb60 0x7f0c0c0d1f10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c4d0 tx=0x7f0c0400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:14.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c117fa700 1 -- 172.21.15.7:0/972381286 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04005d90 con 0x7f0c0c00bb60 2024-09-13T18:52:14.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c117fa700 1 -- 172.21.15.7:0/972381286 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0400ce60 con 0x7f0c0c00bb60 2024-09-13T18:52:14.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c19d93700 1 -- 172.21.15.7:0/972381286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0c0d33a0 con 0x7f0c0c00bb60 2024-09-13T18:52:14.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c117fa700 1 -- 172.21.15.7:0/972381286 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04007310 con 0x7f0c0c00bb60 2024-09-13T18:52:14.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.966+0000 7f0c19d93700 1 -- 172.21.15.7:0/972381286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0c0d2990 con 0x7f0c0c00bb60 2024-09-13T18:52:14.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.970+0000 7f0c19d93700 1 -- 172.21.15.7:0/972381286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0c004800 con 0x7f0c0c00bb60 2024-09-13T18:52:14.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.970+0000 7f0c117fa700 1 -- 172.21.15.7:0/972381286 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7f0c04007470 con 0x7f0c0c00bb60 2024-09-13T18:52:14.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.970+0000 7f0c117fa700 1 --2- 172.21.15.7:0/972381286 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0c000666d0 0x7f0c00068b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:14.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.970+0000 7f0c117fa700 1 -- 172.21.15.7:0/972381286 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0c04082f70 con 0x7f0c0c00bb60 2024-09-13T18:52:14.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.970+0000 7f0c18d91700 1 --2- 172.21.15.7:0/972381286 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0c000666d0 0x7f0c00068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:14.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.970+0000 7f0c18d91700 1 --2- 172.21.15.7:0/972381286 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0c000666d0 0x7f0c00068b90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c03a6c0 tx=0x7f0c08009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:14.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:14.978+0000 7f0c117fa700 1 -- 172.21.15.7:0/972381286 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0c040507a0 con 0x7f0c0c00bb60 2024-09-13T18:52:15.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.162+0000 7f0c19d93700 1 -- 172.21.15.7:0/972381286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0c0c0d1890 con 0x7f0c0c00bb60 2024-09-13T18:52:15.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.162+0000 7f0c117fa700 1 -- 172.21.15.7:0/972381286 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0c04017090 con 0x7f0c0c00bb60 2024-09-13T18:52:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:15.169 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-13T18:52:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:52:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-09-13T18:52:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-09-13T18:52:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:52:15.170 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-13T18:52:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.170+0000 7f0c19d93700 1 -- 172.21.15.7:0/972381286 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0c000666d0 msgr2=0x7f0c00068b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.170+0000 7f0c19d93700 1 --2- 172.21.15.7:0/972381286 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0c000666d0 0x7f0c00068b90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c03a6c0 tx=0x7f0c08009040 comp rx=0 tx=0).stop 2024-09-13T18:52:15.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.170+0000 7f0c19d93700 1 -- 172.21.15.7:0/972381286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c00bb60 msgr2=0x7f0c0c0d1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.170+0000 7f0c19d93700 1 --2- 172.21.15.7:0/972381286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c00bb60 0x7f0c0c0d1f10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c4d0 tx=0x7f0c0400c7e0 comp rx=0 tx=0).stop 2024-09-13T18:52:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.170+0000 7f0c19d93700 1 -- 172.21.15.7:0/972381286 shutdown_connections 2024-09-13T18:52:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.170+0000 7f0c19d93700 1 --2- 172.21.15.7:0/972381286 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0c000666d0 0x7f0c00068b90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.170+0000 7f0c19d93700 1 --2- 172.21.15.7:0/972381286 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0c0c0df9c0 0x7f0c0c0d2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.170+0000 7f0c19d93700 1 --2- 172.21.15.7:0/972381286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c00bb60 0x7f0c0c0d1f10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.170+0000 7f0c19d93700 1 --2- 172.21.15.7:0/972381286 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0c0c005b60 0x7f0c0c032b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.170+0000 7f0c19d93700 1 -- 172.21.15.7:0/972381286 >> 172.21.15.7:0/972381286 conn(0x7f0c0c036140 msgr2=0x7f0c0c0d40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.170+0000 7f0c19d93700 1 -- 172.21.15.7:0/972381286 shutdown_connections 2024-09-13T18:52:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.170+0000 7f0c19d93700 1 -- 172.21.15.7:0/972381286 wait complete. 2024-09-13T18:52:15.188 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T18:52:15.188 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T18:52:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc057fff700 1 Processor -- start 2024-09-13T18:52:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc057fff700 1 -- start start 2024-09-13T18:52:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc057fff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fc05806ccf0 0x7fc058159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc057fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc058070ba0 0x7fc0580720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc057fff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc058071030 con 0x7fc05806d0d0 2024-09-13T18:52:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc057fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc058071310 con 0x7fc058070ba0 2024-09-13T18:52:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc057fff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0580715f0 con 0x7fc05806ccf0 2024-09-13T18:52:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc0567fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc058070ba0 0x7fc0580720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc0567fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc058070ba0 0x7fc0580720d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55724/0 (socket says 172.21.15.7:55724) 2024-09-13T18:52:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc0567fc700 1 -- 172.21.15.7:0/1808900928 learned_addr learned my addr 172.21.15.7:0/1808900928 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc056ffd700 1 --2- 172.21.15.7:0/1808900928 >> v2:172.21.15.171:3300/0 conn(0x7fc05806ccf0 0x7fc058159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:15.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc0567fc700 1 -- 172.21.15.7:0/1808900928 >> v2:172.21.15.171:3300/0 conn(0x7fc05806ccf0 msgr2=0x7fc058159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc0567fc700 1 --2- 172.21.15.7:0/1808900928 >> v2:172.21.15.171:3300/0 conn(0x7fc05806ccf0 0x7fc058159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc0567fc700 1 -- 172.21.15.7:0/1808900928 >> v1:172.21.15.171:6789/0 conn(0x7fc05806d0d0 legacy=0x7fc05815af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc0567fc700 1 -- 172.21.15.7:0/1808900928 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0580718d0 con 0x7fc058070ba0 2024-09-13T18:52:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc056ffd700 1 --2- 172.21.15.7:0/1808900928 >> v2:172.21.15.171:3300/0 conn(0x7fc05806ccf0 0x7fc058159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc0567fc700 1 --2- 172.21.15.7:0/1808900928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc058070ba0 0x7fc0580720d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc0580727d0 tx=0x7fc04c00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed00943dce0a592a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc055ffb700 1 -- 172.21.15.7:0/1808900928 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c00dd50 con 0x7fc058070ba0 2024-09-13T18:52:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc055ffb700 1 -- 172.21.15.7:0/1808900928 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04c004030 con 0x7fc058070ba0 2024-09-13T18:52:15.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.306+0000 7fc055ffb700 1 -- 172.21.15.7:0/1808900928 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c0043c0 con 0x7fc058070ba0 2024-09-13T18:52:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc057fff700 1 -- 172.21.15.7:0/1808900928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc058070ba0 msgr2=0x7fc0580720d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc057fff700 1 --2- 172.21.15.7:0/1808900928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc058070ba0 0x7fc0580720d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc0580727d0 tx=0x7fc04c00be90 comp rx=0 tx=0).stop 2024-09-13T18:52:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc057fff700 1 -- 172.21.15.7:0/1808900928 shutdown_connections 2024-09-13T18:52:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc057fff700 1 --2- 172.21.15.7:0/1808900928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc058070ba0 0x7fc0580720d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc057fff700 1 --2- 172.21.15.7:0/1808900928 >> v2:172.21.15.171:3300/0 conn(0x7fc05806ccf0 0x7fc058159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc057fff700 1 -- 172.21.15.7:0/1808900928 >> 172.21.15.7:0/1808900928 conn(0x7fc0580cfad0 msgr2=0x7fc0580cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc057fff700 1 -- 172.21.15.7:0/1808900928 shutdown_connections 2024-09-13T18:52:15.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc057fff700 1 -- 172.21.15.7:0/1808900928 wait complete. 2024-09-13T18:52:15.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc057fff700 1 Processor -- start 2024-09-13T18:52:15.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc057fff700 1 -- start start 2024-09-13T18:52:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc057fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc05806ccf0 0x7fc058149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc057fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc05806d0d0 0x7fc0581579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc057fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0580aedc0 0x7fc058157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc057fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0580af380 con 0x7fc05806ccf0 2024-09-13T18:52:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc057fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05815ab90 con 0x7fc0580aedc0 2024-09-13T18:52:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc057fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0580ad400 con 0x7fc05806d0d0 2024-09-13T18:52:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc056ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc05806ccf0 0x7fc058149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc056ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc05806ccf0 0x7fc058149780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55728/0 (socket says 172.21.15.7:55728) 2024-09-13T18:52:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.310+0000 7fc056ffd700 1 -- 172.21.15.7:0/448128426 learned_addr learned my addr 172.21.15.7:0/448128426 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.314+0000 7fc0577fe700 1 --2- 172.21.15.7:0/448128426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0580aedc0 0x7fc058157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.314+0000 7fc0567fc700 1 --2- 172.21.15.7:0/448128426 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc05806d0d0 0x7fc0581579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.314+0000 7fc056ffd700 1 -- 172.21.15.7:0/448128426 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc05806d0d0 msgr2=0x7fc0581579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.314+0000 7fc056ffd700 1 --2- 172.21.15.7:0/448128426 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc05806d0d0 0x7fc0581579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.314+0000 7fc056ffd700 1 -- 172.21.15.7:0/448128426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0580aedc0 msgr2=0x7fc058157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.314+0000 7fc056ffd700 1 --2- 172.21.15.7:0/448128426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0580aedc0 0x7fc058157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.314+0000 7fc056ffd700 1 -- 172.21.15.7:0/448128426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c00b810 con 0x7fc05806ccf0 2024-09-13T18:52:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.314+0000 7fc056ffd700 1 --2- 172.21.15.7:0/448128426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc05806ccf0 0x7fc058149780 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc04800c7a0 tx=0x7fc04800cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.314+0000 7fc03ffff700 1 -- 172.21.15.7:0/448128426 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048011800 con 0x7fc05806ccf0 2024-09-13T18:52:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.314+0000 7fc03ffff700 1 -- 172.21.15.7:0/448128426 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc048011960 con 0x7fc05806ccf0 2024-09-13T18:52:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.314+0000 7fc057fff700 1 -- 172.21.15.7:0/448128426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0580ad6e0 con 0x7fc05806ccf0 2024-09-13T18:52:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.314+0000 7fc03ffff700 1 -- 172.21.15.7:0/448128426 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048011d10 con 0x7fc05806ccf0 2024-09-13T18:52:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.314+0000 7fc057fff700 1 -- 172.21.15.7:0/448128426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc058149cc0 con 0x7fc05806ccf0 2024-09-13T18:52:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.314+0000 7fc057fff700 1 -- 172.21.15.7:0/448128426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0580b39b0 con 0x7fc05806ccf0 2024-09-13T18:52:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.318+0000 7fc03ffff700 1 -- 172.21.15.7:0/448128426 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7fc048018360 con 0x7fc05806ccf0 2024-09-13T18:52:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.318+0000 7fc03ffff700 1 --2- 172.21.15.7:0/448128426 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc0440666a0 0x7fc044068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.318+0000 7fc03ffff700 1 -- 172.21.15.7:0/448128426 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc048083c70 con 0x7fc05806ccf0 2024-09-13T18:52:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.318+0000 7fc0567fc700 1 --2- 172.21.15.7:0/448128426 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc0440666a0 0x7fc044068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.318+0000 7fc0567fc700 1 --2- 172.21.15.7:0/448128426 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc0440666a0 0x7fc044068b60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc04c000c00 tx=0x7fc04c01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:15.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.322+0000 7fc03ffff700 1 -- 172.21.15.7:0/448128426 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc048051420 con 0x7fc05806ccf0 2024-09-13T18:52:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:15.514+0000 7fc057fff700 1 -- 172.21.15.7:0/448128426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc0580b0780 con 0x7fc05806ccf0 2024-09-13T18:52:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.510+0000 7fc03ffff700 1 -- 172.21.15.7:0/448128426 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc048054a40 con 0x7fc05806ccf0 2024-09-13T18:52:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.518+0000 7fc057fff700 1 -- 172.21.15.7:0/448128426 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc0440666a0 msgr2=0x7fc044068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.518+0000 7fc057fff700 1 --2- 172.21.15.7:0/448128426 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc0440666a0 0x7fc044068b60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc04c000c00 tx=0x7fc04c01c040 comp rx=0 tx=0).stop 2024-09-13T18:52:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.518+0000 7fc057fff700 1 -- 172.21.15.7:0/448128426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc05806ccf0 msgr2=0x7fc058149780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.518+0000 7fc057fff700 1 --2- 172.21.15.7:0/448128426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc05806ccf0 0x7fc058149780 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc04800c7a0 tx=0x7fc04800cab0 comp rx=0 tx=0).stop 2024-09-13T18:52:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.518+0000 7fc057fff700 1 -- 172.21.15.7:0/448128426 shutdown_connections 2024-09-13T18:52:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.518+0000 7fc057fff700 1 --2- 172.21.15.7:0/448128426 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc0440666a0 0x7fc044068b60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.518+0000 7fc057fff700 1 --2- 172.21.15.7:0/448128426 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0580aedc0 0x7fc058157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.518+0000 7fc057fff700 1 --2- 172.21.15.7:0/448128426 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc05806d0d0 0x7fc0581579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.518+0000 7fc057fff700 1 --2- 172.21.15.7:0/448128426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc05806ccf0 0x7fc058149780 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.518+0000 7fc057fff700 1 -- 172.21.15.7:0/448128426 >> 172.21.15.7:0/448128426 conn(0x7fc0580cfad0 msgr2=0x7fc05806bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.518+0000 7fc057fff700 1 -- 172.21.15.7:0/448128426 shutdown_connections 2024-09-13T18:52:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.518+0000 7fc057fff700 1 -- 172.21.15.7:0/448128426 wait complete. 2024-09-13T18:52:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-09-13T18:52:16.536 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T18:52:16.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa578977700 1 Processor -- start 2024-09-13T18:52:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa578977700 1 -- start start 2024-09-13T18:52:16.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa578977700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa57415cda0 0x7fa57406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:16.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa578977700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa57406a520 0x7fa57406a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:16.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa578977700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57406aeb0 con 0x7fa5740ceb70 2024-09-13T18:52:16.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa578977700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa57406be40 con 0x7fa57406a520 2024-09-13T18:52:16.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa578977700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740b64f0 con 0x7fa57415cda0 2024-09-13T18:52:16.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa57259c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa5740ceb70 0x7fa5740b5de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52166/0 (socket says 172.21.15.7:52166) 2024-09-13T18:52:16.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa57259c700 1 -- 172.21.15.7:0/213439341 learned_addr learned my addr 172.21.15.7:0/213439341 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa57359e700 1 --2- 172.21.15.7:0/213439341 >> v2:172.21.15.171:3300/0 conn(0x7fa57406a520 0x7fa57406a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa572d9d700 1 --2- 172.21.15.7:0/213439341 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa57415cda0 0x7fa57406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa571d9b700 1 -- 172.21.15.7:0/213439341 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1490450639 0 0) 0x7fa55c001110 con 0x7fa5740ceb70 2024-09-13T18:52:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa571d9b700 1 -- 172.21.15.7:0/213439341 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5600035d0 con 0x7fa5740ceb70 2024-09-13T18:52:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa57359e700 1 -- 172.21.15.7:0/213439341 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa57415cda0 msgr2=0x7fa57406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa57359e700 1 --2- 172.21.15.7:0/213439341 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa57415cda0 0x7fa57406b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa57359e700 1 -- 172.21.15.7:0/213439341 >> v1:172.21.15.7:6789/0 conn(0x7fa5740ceb70 legacy=0x7fa5740b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa57359e700 1 -- 172.21.15.7:0/213439341 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5740b8430 con 0x7fa57406a520 2024-09-13T18:52:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa57359e700 1 --2- 172.21.15.7:0/213439341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa57406a520 0x7fa57406a970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009d90 tx=0x7fa56c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ce0d51c661e05c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa571d9b700 1 -- 172.21.15.7:0/213439341 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c00e070 con 0x7fa57406a520 2024-09-13T18:52:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa571d9b700 1 -- 172.21.15.7:0/213439341 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56c00bb20 con 0x7fa57406a520 2024-09-13T18:52:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa571d9b700 1 -- 172.21.15.7:0/213439341 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c012430 con 0x7fa57406a520 2024-09-13T18:52:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa578977700 1 -- 172.21.15.7:0/213439341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa57406a520 msgr2=0x7fa57406a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa578977700 1 --2- 172.21.15.7:0/213439341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa57406a520 0x7fa57406a970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009d90 tx=0x7fa56c00b330 comp rx=0 tx=0).stop 2024-09-13T18:52:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa578977700 1 -- 172.21.15.7:0/213439341 shutdown_connections 2024-09-13T18:52:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa578977700 1 --2- 172.21.15.7:0/213439341 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa57406a520 0x7fa57406a970 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa578977700 1 --2- 172.21.15.7:0/213439341 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa57415cda0 0x7fa57406b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa578977700 1 -- 172.21.15.7:0/213439341 >> 172.21.15.7:0/213439341 conn(0x7fa5740d2cb0 msgr2=0x7fa5740d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa578977700 1 -- 172.21.15.7:0/213439341 shutdown_connections 2024-09-13T18:52:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.650+0000 7fa578977700 1 -- 172.21.15.7:0/213439341 wait complete. 2024-09-13T18:52:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa578977700 1 Processor -- start 2024-09-13T18:52:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa578977700 1 -- start start 2024-09-13T18:52:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa578977700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa57406a520 0x7fa57414afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa578977700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5740ceb70 0x7fa5740b6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa578977700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa57415cda0 0x7fa5740b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa578977700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740c8a00 con 0x7fa57406a520 2024-09-13T18:52:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa578977700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740c8c30 con 0x7fa5740ceb70 2024-09-13T18:52:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa578977700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740c8ed0 con 0x7fa57415cda0 2024-09-13T18:52:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa572d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa57406a520 0x7fa57414afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa572d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa57406a520 0x7fa57414afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55742/0 (socket says 172.21.15.7:55742) 2024-09-13T18:52:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa572d9d700 1 -- 172.21.15.7:0/483082177 learned_addr learned my addr 172.21.15.7:0/483082177 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa57359e700 1 --2- 172.21.15.7:0/483082177 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa57415cda0 0x7fa5740b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa57259c700 1 --2- 172.21.15.7:0/483082177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5740ceb70 0x7fa5740b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa572d9d700 1 -- 172.21.15.7:0/483082177 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa57415cda0 msgr2=0x7fa5740b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa572d9d700 1 --2- 172.21.15.7:0/483082177 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa57415cda0 0x7fa5740b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa572d9d700 1 -- 172.21.15.7:0/483082177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5740ceb70 msgr2=0x7fa5740b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa572d9d700 1 --2- 172.21.15.7:0/483082177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5740ceb70 0x7fa5740b6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa572d9d700 1 -- 172.21.15.7:0/483082177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c009a70 con 0x7fa57406a520 2024-09-13T18:52:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa57259c700 1 --2- 172.21.15.7:0/483082177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5740ceb70 0x7fa5740b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa572d9d700 1 --2- 172.21.15.7:0/483082177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa57406a520 0x7fa57414afa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa56400d800 tx=0x7fa56400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa56b7fe700 1 -- 172.21.15.7:0/483082177 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5640125f0 con 0x7fa57406a520 2024-09-13T18:52:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa56b7fe700 1 -- 172.21.15.7:0/483082177 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa564012750 con 0x7fa57406a520 2024-09-13T18:52:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa578977700 1 -- 172.21.15.7:0/483082177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5740c91b0 con 0x7fa57406a520 2024-09-13T18:52:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.654+0000 7fa578977700 1 -- 172.21.15.7:0/483082177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5740c9770 con 0x7fa57406a520 2024-09-13T18:52:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.658+0000 7fa56b7fe700 1 -- 172.21.15.7:0/483082177 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564012ac0 con 0x7fa57406a520 2024-09-13T18:52:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.658+0000 7fa578977700 1 -- 172.21.15.7:0/483082177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5740d1120 con 0x7fa57406a520 2024-09-13T18:52:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.658+0000 7fa56b7fe700 1 -- 172.21.15.7:0/483082177 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 86036+0+0 (secure 0 0 0) 0x7fa564012c20 con 0x7fa57406a520 2024-09-13T18:52:16.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.658+0000 7fa56b7fe700 1 --2- 172.21.15.7:0/483082177 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa5540666a0 0x7fa554068b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:16.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.658+0000 7fa56b7fe700 1 -- 172.21.15.7:0/483082177 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fa564083f70 con 0x7fa57406a520 2024-09-13T18:52:16.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.658+0000 7fa57259c700 1 --2- 172.21.15.7:0/483082177 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa5540666a0 0x7fa554068b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:16.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.662+0000 7fa57259c700 1 --2- 172.21.15.7:0/483082177 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa5540666a0 0x7fa554068b60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa5741558d0 tx=0x7fa55c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:16.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.666+0000 7fa56b7fe700 1 -- 172.21.15.7:0/483082177 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa564051520 con 0x7fa57406a520 2024-09-13T18:52:16.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:16.850+0000 7fa578977700 1 -- 172.21.15.7:0/483082177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa5740b2cf0 con 0x7fa57406a520 2024-09-13T18:52:17.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.026+0000 7fa56b7fe700 1 -- 172.21.15.7:0/483082177 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa5640518f0 con 0x7fa57406a520 2024-09-13T18:52:17.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.518+0000 7fa56b7fe700 1 -- 172.21.15.7:0/483082177 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa564070110 con 0x7fa57406a520 2024-09-13T18:52:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.534+0000 7fa5697fa700 1 -- 172.21.15.7:0/483082177 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa5540666a0 msgr2=0x7fa554068b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.534+0000 7fa5697fa700 1 --2- 172.21.15.7:0/483082177 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa5540666a0 0x7fa554068b60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa5741558d0 tx=0x7fa55c009040 comp rx=0 tx=0).stop 2024-09-13T18:52:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.534+0000 7fa5697fa700 1 -- 172.21.15.7:0/483082177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa57406a520 msgr2=0x7fa57414afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.534+0000 7fa5697fa700 1 --2- 172.21.15.7:0/483082177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa57406a520 0x7fa57414afa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa56400d800 tx=0x7fa56400db10 comp rx=0 tx=0).stop 2024-09-13T18:52:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.534+0000 7fa5697fa700 1 -- 172.21.15.7:0/483082177 shutdown_connections 2024-09-13T18:52:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.534+0000 7fa5697fa700 1 --2- 172.21.15.7:0/483082177 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa5540666a0 0x7fa554068b60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.534+0000 7fa5697fa700 1 --2- 172.21.15.7:0/483082177 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa57415cda0 0x7fa5740b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:17.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.534+0000 7fa5697fa700 1 --2- 172.21.15.7:0/483082177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5740ceb70 0x7fa5740b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:17.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.534+0000 7fa5697fa700 1 --2- 172.21.15.7:0/483082177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa57406a520 0x7fa57414afa0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:17.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.534+0000 7fa5697fa700 1 -- 172.21.15.7:0/483082177 >> 172.21.15.7:0/483082177 conn(0x7fa5740d2cb0 msgr2=0x7fa5740d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:17.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.534+0000 7fa5697fa700 1 -- 172.21.15.7:0/483082177 shutdown_connections 2024-09-13T18:52:17.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.538+0000 7fa5697fa700 1 -- 172.21.15.7:0/483082177 wait complete. 2024-09-13T18:52:17.543 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-09-13T18:52:17.557 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T18:52:17.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.670+0000 7fdececf0700 1 Processor -- start 2024-09-13T18:52:17.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.670+0000 7fdececf0700 1 -- start start 2024-09-13T18:52:17.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.670+0000 7fdececf0700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdec815cda0 0x7fdec806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:17.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.670+0000 7fdececf0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdec806a520 0x7fdec806a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:17.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.670+0000 7fdececf0700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec806aeb0 con 0x7fdec80ceb70 2024-09-13T18:52:17.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.670+0000 7fdececf0700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec806be40 con 0x7fdec806a520 2024-09-13T18:52:17.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.670+0000 7fdececf0700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec80b64f0 con 0x7fdec815cda0 2024-09-13T18:52:17.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.670+0000 7fdece4ef700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdec806a520 0x7fdec806a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:17.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.670+0000 7fdece4ef700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdec806a520 0x7fdec806a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55746/0 (socket says 172.21.15.7:55746) 2024-09-13T18:52:17.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.670+0000 7fdece4ef700 1 -- 172.21.15.7:0/1529148830 learned_addr learned my addr 172.21.15.7:0/1529148830 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:17.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.670+0000 7fdecdcee700 1 --2- 172.21.15.7:0/1529148830 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdec815cda0 0x7fdec806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:17.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdece4ef700 1 -- 172.21.15.7:0/1529148830 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdec815cda0 msgr2=0x7fdec806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdece4ef700 1 --2- 172.21.15.7:0/1529148830 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdec815cda0 0x7fdec806b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdece4ef700 1 -- 172.21.15.7:0/1529148830 >> v1:172.21.15.171:6789/0 conn(0x7fdec80ceb70 legacy=0x7fdec80b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdece4ef700 1 -- 172.21.15.7:0/1529148830 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec80b8430 con 0x7fdec806a520 2024-09-13T18:52:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdecccec700 1 -- 172.21.15.7:0/1529148830 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992172541 0 0) 0x7fdeb8001110 con 0x7fdec80ceb70 2024-09-13T18:52:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdece4ef700 1 --2- 172.21.15.7:0/1529148830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdec806a520 0x7fdec806a970 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009e50 tx=0x7fdec000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2508f8204e908a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdecccec700 1 -- 172.21.15.7:0/1529148830 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec000e070 con 0x7fdec806a520 2024-09-13T18:52:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdecccec700 1 -- 172.21.15.7:0/1529148830 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec000b910 con 0x7fdec806a520 2024-09-13T18:52:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdecccec700 1 -- 172.21.15.7:0/1529148830 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec000bcc0 con 0x7fdec806a520 2024-09-13T18:52:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdececf0700 1 -- 172.21.15.7:0/1529148830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdec806a520 msgr2=0x7fdec806a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdececf0700 1 --2- 172.21.15.7:0/1529148830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdec806a520 0x7fdec806a970 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009e50 tx=0x7fdec000b330 comp rx=0 tx=0).stop 2024-09-13T18:52:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdececf0700 1 -- 172.21.15.7:0/1529148830 shutdown_connections 2024-09-13T18:52:17.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdececf0700 1 --2- 172.21.15.7:0/1529148830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdec806a520 0x7fdec806a970 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:17.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdececf0700 1 --2- 172.21.15.7:0/1529148830 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdec815cda0 0x7fdec806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:17.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdececf0700 1 -- 172.21.15.7:0/1529148830 >> 172.21.15.7:0/1529148830 conn(0x7fdec80d2cb0 msgr2=0x7fdec80d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:17.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdececf0700 1 -- 172.21.15.7:0/1529148830 shutdown_connections 2024-09-13T18:52:17.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdececf0700 1 -- 172.21.15.7:0/1529148830 wait complete. 2024-09-13T18:52:17.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdececf0700 1 Processor -- start 2024-09-13T18:52:17.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.674+0000 7fdececf0700 1 -- start start 2024-09-13T18:52:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdececf0700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdec806a520 0x7fdec80c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdececf0700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdec80ceb70 0x7fdec81f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdececf0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdec815cda0 0x7fdec81f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdececf0700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec81f31a0 con 0x7fdec815cda0 2024-09-13T18:52:17.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdececf0700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec81f3480 con 0x7fdec806a520 2024-09-13T18:52:17.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdececf0700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec81f3760 con 0x7fdec80ceb70 2024-09-13T18:52:17.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdece4ef700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdec815cda0 0x7fdec81f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:17.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdecdcee700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdec806a520 0x7fdec80c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:17.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdece4ef700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdec815cda0 0x7fdec81f39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55752/0 (socket says 172.21.15.7:55752) 2024-09-13T18:52:17.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdece4ef700 1 -- 172.21.15.7:0/3860818224 learned_addr learned my addr 172.21.15.7:0/3860818224 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:17.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdecd4ed700 1 --2- 172.21.15.7:0/3860818224 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdec80ceb70 0x7fdec81f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:17.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdece4ef700 1 -- 172.21.15.7:0/3860818224 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdec80ceb70 msgr2=0x7fdec81f27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:17.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdece4ef700 1 --2- 172.21.15.7:0/3860818224 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdec80ceb70 0x7fdec81f27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:17.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdece4ef700 1 -- 172.21.15.7:0/3860818224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdec806a520 msgr2=0x7fdec80c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:17.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdece4ef700 1 --2- 172.21.15.7:0/3860818224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdec806a520 0x7fdec80c6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:17.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdece4ef700 1 -- 172.21.15.7:0/3860818224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec0009b30 con 0x7fdec815cda0 2024-09-13T18:52:17.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdece4ef700 1 --2- 172.21.15.7:0/3860818224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdec815cda0 0x7fdec81f39f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdec000b680 tx=0x7fdec0004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:17.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdebe7fc700 1 -- 172.21.15.7:0/3860818224 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec00048b0 con 0x7fdec815cda0 2024-09-13T18:52:17.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdebe7fc700 1 -- 172.21.15.7:0/3860818224 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec00099b0 con 0x7fdec815cda0 2024-09-13T18:52:17.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdececf0700 1 -- 172.21.15.7:0/3860818224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec80c89e0 con 0x7fdec815cda0 2024-09-13T18:52:17.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdebe7fc700 1 -- 172.21.15.7:0/3860818224 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec001b670 con 0x7fdec815cda0 2024-09-13T18:52:17.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.678+0000 7fdececf0700 1 -- 172.21.15.7:0/3860818224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec80c7250 con 0x7fdec815cda0 2024-09-13T18:52:17.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.682+0000 7fdececf0700 1 -- 172.21.15.7:0/3860818224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec80c8840 con 0x7fdec815cda0 2024-09-13T18:52:17.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.682+0000 7fdebe7fc700 1 -- 172.21.15.7:0/3860818224 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdec001b7d0 con 0x7fdec815cda0 2024-09-13T18:52:17.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.682+0000 7fdebe7fc700 1 --2- 172.21.15.7:0/3860818224 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fdeac066730 0x7fdeac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:17.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.682+0000 7fdecdcee700 1 --2- 172.21.15.7:0/3860818224 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fdeac066730 0x7fdeac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:17.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.682+0000 7fdebe7fc700 1 -- 172.21.15.7:0/3860818224 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fdec0085980 con 0x7fdec815cda0 2024-09-13T18:52:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.686+0000 7fdecdcee700 1 --2- 172.21.15.7:0/3860818224 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fdeac066730 0x7fdeac068bf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009bf0 tx=0x7fdec4009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:17.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.694+0000 7fdebe7fc700 1 -- 172.21.15.7:0/3860818224 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdec80c8840 con 0x7fdec815cda0 2024-09-13T18:52:17.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.906+0000 7fdececf0700 1 -- 172.21.15.7:0/3860818224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdec80c8840 con 0x7fdec815cda0 2024-09-13T18:52:17.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.910+0000 7fdebe7fc700 1 -- 172.21.15.7:0/3860818224 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fdec80c8840 con 0x7fdec815cda0 2024-09-13T18:52:17.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.918+0000 7fdececf0700 1 -- 172.21.15.7:0/3860818224 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fdeac066730 msgr2=0x7fdeac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:17.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.918+0000 7fdececf0700 1 --2- 172.21.15.7:0/3860818224 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fdeac066730 0x7fdeac068bf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009bf0 tx=0x7fdec4009240 comp rx=0 tx=0).stop 2024-09-13T18:52:17.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.918+0000 7fdececf0700 1 -- 172.21.15.7:0/3860818224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdec815cda0 msgr2=0x7fdec81f39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:17.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.918+0000 7fdececf0700 1 --2- 172.21.15.7:0/3860818224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdec815cda0 0x7fdec81f39f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdec000b680 tx=0x7fdec0004af0 comp rx=0 tx=0).stop 2024-09-13T18:52:17.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.918+0000 7fdececf0700 1 -- 172.21.15.7:0/3860818224 shutdown_connections 2024-09-13T18:52:17.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.918+0000 7fdececf0700 1 --2- 172.21.15.7:0/3860818224 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fdeac066730 0x7fdeac068bf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:17.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.918+0000 7fdececf0700 1 --2- 172.21.15.7:0/3860818224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdec815cda0 0x7fdec81f39f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:17.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.918+0000 7fdececf0700 1 --2- 172.21.15.7:0/3860818224 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdec80ceb70 0x7fdec81f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:17.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.918+0000 7fdececf0700 1 --2- 172.21.15.7:0/3860818224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdec806a520 0x7fdec80c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:17.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.918+0000 7fdececf0700 1 -- 172.21.15.7:0/3860818224 >> 172.21.15.7:0/3860818224 conn(0x7fdec80d2cb0 msgr2=0x7fdec815bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:17.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.918+0000 7fdececf0700 1 -- 172.21.15.7:0/3860818224 shutdown_connections 2024-09-13T18:52:17.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:17.918+0000 7fdececf0700 1 -- 172.21.15.7:0/3860818224 wait complete. 2024-09-13T18:52:17.926 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 2 and data pool 3 2024-09-13T18:52:17.940 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:18.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.042+0000 7fe8d8bdb700 1 Processor -- start 2024-09-13T18:52:18.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.042+0000 7fe8d8bdb700 1 -- start start 2024-09-13T18:52:18.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.042+0000 7fe8d8bdb700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fe8d406b310 0x7fe8d40cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.042+0000 7fe8d8bdb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d40d21e0 con 0x7fe8d415cae0 2024-09-13T18:52:18.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.042+0000 7fe8d8bdb700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d40d1940 con 0x7fe8d415cec0 2024-09-13T18:52:18.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.042+0000 7fe8d8bdb700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d40d1c20 con 0x7fe8d406b310 2024-09-13T18:52:18.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.042+0000 7fe8d37fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe8d415cae0 0x7fe8d406ac00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52176/0 (socket says 172.21.15.7:52176) 2024-09-13T18:52:18.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.042+0000 7fe8d37fe700 1 -- 172.21.15.7:0/588346163 learned_addr learned my addr 172.21.15.7:0/588346163 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:18.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.042+0000 7fe8d3fff700 1 --2- 172.21.15.7:0/588346163 >> v2:172.21.15.171:3300/0 conn(0x7fe8d406b310 0x7fe8d40cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.042+0000 7fe8d27fc700 1 -- 172.21.15.7:0/588346163 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 814263271 0 0) 0x7fe8d40d21e0 con 0x7fe8d415cae0 2024-09-13T18:52:18.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.042+0000 7fe8d27fc700 1 -- 172.21.15.7:0/588346163 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8bc0035d0 con 0x7fe8d415cae0 2024-09-13T18:52:18.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.046+0000 7fe8d3fff700 1 -- 172.21.15.7:0/588346163 >> v1:172.21.15.171:6789/0 conn(0x7fe8d415cec0 legacy=0x7fe8d40b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.046+0000 7fe8d27fc700 1 -- 172.21.15.7:0/588346163 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417777646 0 0) 0x7fe8d40d1940 con 0x7fe8d415cec0 2024-09-13T18:52:18.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.046+0000 7fe8d3fff700 1 -- 172.21.15.7:0/588346163 >> v1:172.21.15.7:6789/0 conn(0x7fe8d415cae0 legacy=0x7fe8d406ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.046+0000 7fe8d3fff700 1 -- 172.21.15.7:0/588346163 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d40d1f00 con 0x7fe8d406b310 2024-09-13T18:52:18.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.046+0000 7fe8d3fff700 1 --2- 172.21.15.7:0/588346163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8d406b310 0x7fe8d40cf030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009dc0 tx=0x7fe8cc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=746b1152f0d23aa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:18.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.046+0000 7fe8d27fc700 1 -- 172.21.15.7:0/588346163 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc00e070 con 0x7fe8d406b310 2024-09-13T18:52:18.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.046+0000 7fe8d27fc700 1 -- 172.21.15.7:0/588346163 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8cc00bae0 con 0x7fe8d406b310 2024-09-13T18:52:18.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.046+0000 7fe8d27fc700 1 -- 172.21.15.7:0/588346163 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc00be90 con 0x7fe8d406b310 2024-09-13T18:52:18.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.046+0000 7fe8d8bdb700 1 -- 172.21.15.7:0/588346163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8d406b310 msgr2=0x7fe8d40cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.046+0000 7fe8d8bdb700 1 --2- 172.21.15.7:0/588346163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8d406b310 0x7fe8d40cf030 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009dc0 tx=0x7fe8cc00b330 comp rx=0 tx=0).stop 2024-09-13T18:52:18.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.046+0000 7fe8d8bdb700 1 -- 172.21.15.7:0/588346163 shutdown_connections 2024-09-13T18:52:18.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.046+0000 7fe8d8bdb700 1 --2- 172.21.15.7:0/588346163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8d406b310 0x7fe8d40cf030 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.046+0000 7fe8d8bdb700 1 -- 172.21.15.7:0/588346163 >> 172.21.15.7:0/588346163 conn(0x7fe8d40d2c80 msgr2=0x7fe8d40d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:18.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.046+0000 7fe8d8bdb700 1 -- 172.21.15.7:0/588346163 shutdown_connections 2024-09-13T18:52:18.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.046+0000 7fe8d8bdb700 1 -- 172.21.15.7:0/588346163 wait complete. 2024-09-13T18:52:18.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.050+0000 7fe8d8bdb700 1 Processor -- start 2024-09-13T18:52:18.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.050+0000 7fe8d8bdb700 1 -- start start 2024-09-13T18:52:18.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.050+0000 7fe8d8bdb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d406b310 0x7fe8d4157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.050+0000 7fe8d8bdb700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe8d415cae0 0x7fe8d415fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.050+0000 7fe8d8bdb700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8d415cec0 0x7fe8d40cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.050+0000 7fe8d8bdb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d415dc70 con 0x7fe8d406b310 2024-09-13T18:52:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.050+0000 7fe8d8bdb700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d415de50 con 0x7fe8d415cec0 2024-09-13T18:52:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.050+0000 7fe8d8bdb700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d415e130 con 0x7fe8d415cae0 2024-09-13T18:52:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.050+0000 7fe8d37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d406b310 0x7fe8d4157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.050+0000 7fe8d37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d406b310 0x7fe8d4157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55764/0 (socket says 172.21.15.7:55764) 2024-09-13T18:52:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.050+0000 7fe8d37fe700 1 -- 172.21.15.7:0/1549939258 learned_addr learned my addr 172.21.15.7:0/1549939258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.054+0000 7fe8d3fff700 1 --2- 172.21.15.7:0/1549939258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8d415cec0 0x7fe8d40cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.054+0000 7fe8d2ffd700 1 --2- 172.21.15.7:0/1549939258 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe8d415cae0 0x7fe8d415fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.054+0000 7fe8d37fe700 1 -- 172.21.15.7:0/1549939258 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe8d415cae0 msgr2=0x7fe8d415fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.054+0000 7fe8d37fe700 1 --2- 172.21.15.7:0/1549939258 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe8d415cae0 0x7fe8d415fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.054+0000 7fe8d37fe700 1 -- 172.21.15.7:0/1549939258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8d415cec0 msgr2=0x7fe8d40cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.054+0000 7fe8d37fe700 1 --2- 172.21.15.7:0/1549939258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8d415cec0 0x7fe8d40cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.054+0000 7fe8d37fe700 1 -- 172.21.15.7:0/1549939258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc009a70 con 0x7fe8d406b310 2024-09-13T18:52:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.054+0000 7fe8d37fe700 1 --2- 172.21.15.7:0/1549939258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d406b310 0x7fe8d4157750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800c4f0 tx=0x7fe8c800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:18.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.054+0000 7fe8bbfff700 1 -- 172.21.15.7:0/1549939258 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8005d90 con 0x7fe8d406b310 2024-09-13T18:52:18.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.054+0000 7fe8bbfff700 1 -- 172.21.15.7:0/1549939258 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c80070d0 con 0x7fe8d406b310 2024-09-13T18:52:18.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.054+0000 7fe8bbfff700 1 -- 172.21.15.7:0/1549939258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c8007440 con 0x7fe8d406b310 2024-09-13T18:52:18.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.054+0000 7fe8d8bdb700 1 -- 172.21.15.7:0/1549939258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d415e410 con 0x7fe8d406b310 2024-09-13T18:52:18.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.054+0000 7fe8d8bdb700 1 -- 172.21.15.7:0/1549939258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d415e9d0 con 0x7fe8d406b310 2024-09-13T18:52:18.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.054+0000 7fe8d8bdb700 1 -- 172.21.15.7:0/1549939258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d40b2050 con 0x7fe8d406b310 2024-09-13T18:52:18.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.054+0000 7fe8bbfff700 1 -- 172.21.15.7:0/1549939258 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe8c80075a0 con 0x7fe8d406b310 2024-09-13T18:52:18.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.058+0000 7fe8bbfff700 1 --2- 172.21.15.7:0/1549939258 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe8c0066730 0x7fe8c0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.058+0000 7fe8bbfff700 1 -- 172.21.15.7:0/1549939258 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8c8083500 con 0x7fe8d406b310 2024-09-13T18:52:18.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.058+0000 7fe8d2ffd700 1 --2- 172.21.15.7:0/1549939258 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe8c0066730 0x7fe8c0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.058+0000 7fe8d2ffd700 1 --2- 172.21.15.7:0/1549939258 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe8c0066730 0x7fe8c0068bf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40b19d0 tx=0x7fe8c4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:18.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.066+0000 7fe8bbfff700 1 -- 172.21.15.7:0/1549939258 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe8c80509b0 con 0x7fe8d406b310 2024-09-13T18:52:18.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.242+0000 7fe8d8bdb700 1 -- 172.21.15.7:0/1549939258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8d40cf030 con 0x7fe8d406b310 2024-09-13T18:52:18.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.242+0000 7fe8bbfff700 1 -- 172.21.15.7:0/1549939258 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16470 (secure 0 0 0) 0x7fe8c8053fd0 con 0x7fe8d406b310 2024-09-13T18:52:18.249 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:18.249 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:17.908202+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:18.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.246+0000 7fe8d8bdb700 1 -- 172.21.15.7:0/1549939258 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe8c0066730 msgr2=0x7fe8c0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.246+0000 7fe8d8bdb700 1 --2- 172.21.15.7:0/1549939258 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe8c0066730 0x7fe8c0068bf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40b19d0 tx=0x7fe8c4009040 comp rx=0 tx=0).stop 2024-09-13T18:52:18.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.246+0000 7fe8d8bdb700 1 -- 172.21.15.7:0/1549939258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d406b310 msgr2=0x7fe8d4157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.246+0000 7fe8d8bdb700 1 --2- 172.21.15.7:0/1549939258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d406b310 0x7fe8d4157750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800c4f0 tx=0x7fe8c800c8b0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.246+0000 7fe8d8bdb700 1 -- 172.21.15.7:0/1549939258 shutdown_connections 2024-09-13T18:52:18.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.246+0000 7fe8d8bdb700 1 --2- 172.21.15.7:0/1549939258 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe8c0066730 0x7fe8c0068bf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.246+0000 7fe8d8bdb700 1 --2- 172.21.15.7:0/1549939258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe8d415cec0 0x7fe8d40cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.246+0000 7fe8d8bdb700 1 --2- 172.21.15.7:0/1549939258 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe8d415cae0 0x7fe8d415fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.246+0000 7fe8d8bdb700 1 --2- 172.21.15.7:0/1549939258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d406b310 0x7fe8d4157750 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.246+0000 7fe8d8bdb700 1 -- 172.21.15.7:0/1549939258 >> 172.21.15.7:0/1549939258 conn(0x7fe8d40d2c80 msgr2=0x7fe8d40b7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:18.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.246+0000 7fe8d8bdb700 1 -- 172.21.15.7:0/1549939258 shutdown_connections 2024-09-13T18:52:18.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.246+0000 7fe8d8bdb700 1 -- 172.21.15.7:0/1549939258 wait complete. 2024-09-13T18:52:18.264 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.374+0000 7fd23f190700 1 Processor -- start 2024-09-13T18:52:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.374+0000 7fd23f190700 1 -- start start 2024-09-13T18:52:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.374+0000 7fd23f190700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd23806bbf0 0x7fd2380b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.374+0000 7fd23f190700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd2380b3910 0x7fd23806a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.374+0000 7fd23f190700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd23813dcb0 con 0x7fd23806aaf0 2024-09-13T18:52:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.374+0000 7fd23f190700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd23815be60 con 0x7fd23806bbf0 2024-09-13T18:52:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.374+0000 7fd23f190700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23813ffd0 con 0x7fd2380b3910 2024-09-13T18:52:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.374+0000 7fd23e18e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd23806bbf0 0x7fd2380b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.374+0000 7fd23d98d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd2380b3910 0x7fd23806a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.374+0000 7fd23e98f700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7fd23806aaf0 0x7fd2380cd580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:58990/0 (socket says 172.21.15.7:58990) 2024-09-13T18:52:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.374+0000 7fd23e98f700 1 -- 172.21.15.7:0/1910662673 learned_addr learned my addr 172.21.15.7:0/1910662673 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.374+0000 7fd23d18c700 1 -- 172.21.15.7:0/1910662673 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367032578 0 0) 0x7fd23813dcb0 con 0x7fd23806aaf0 2024-09-13T18:52:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.374+0000 7fd23d18c700 1 -- 172.21.15.7:0/1910662673 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2240035d0 con 0x7fd23806aaf0 2024-09-13T18:52:18.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.378+0000 7fd23e18e700 1 -- 172.21.15.7:0/1910662673 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd2380b3910 msgr2=0x7fd23806a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.378+0000 7fd23e18e700 1 --2- 172.21.15.7:0/1910662673 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd2380b3910 0x7fd23806a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.378+0000 7fd23e18e700 1 -- 172.21.15.7:0/1910662673 >> v1:172.21.15.171:6789/0 conn(0x7fd23806aaf0 legacy=0x7fd2380cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.378+0000 7fd23e18e700 1 -- 172.21.15.7:0/1910662673 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2380b3d70 con 0x7fd23806bbf0 2024-09-13T18:52:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.378+0000 7fd23e18e700 1 --2- 172.21.15.7:0/1910662673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd23806bbf0 0x7fd2380b33d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd228009850 tx=0x7fd228009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8809b3e24a1ac9b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.378+0000 7fd23d18c700 1 -- 172.21.15.7:0/1910662673 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228011830 con 0x7fd23806bbf0 2024-09-13T18:52:18.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.378+0000 7fd23d18c700 1 -- 172.21.15.7:0/1910662673 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd228011990 con 0x7fd23806bbf0 2024-09-13T18:52:18.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.378+0000 7fd23d18c700 1 -- 172.21.15.7:0/1910662673 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd228011d00 con 0x7fd23806bbf0 2024-09-13T18:52:18.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.378+0000 7fd23f190700 1 -- 172.21.15.7:0/1910662673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd23806bbf0 msgr2=0x7fd2380b33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.378+0000 7fd23f190700 1 --2- 172.21.15.7:0/1910662673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd23806bbf0 0x7fd2380b33d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd228009850 tx=0x7fd228009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:18.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.378+0000 7fd23f190700 1 -- 172.21.15.7:0/1910662673 shutdown_connections 2024-09-13T18:52:18.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.378+0000 7fd23f190700 1 --2- 172.21.15.7:0/1910662673 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd2380b3910 0x7fd23806a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.378+0000 7fd23f190700 1 --2- 172.21.15.7:0/1910662673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd23806bbf0 0x7fd2380b33d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.378+0000 7fd23f190700 1 -- 172.21.15.7:0/1910662673 >> 172.21.15.7:0/1910662673 conn(0x7fd2380b0f00 msgr2=0x7fd2380b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:18.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.378+0000 7fd23f190700 1 -- 172.21.15.7:0/1910662673 shutdown_connections 2024-09-13T18:52:18.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.378+0000 7fd23f190700 1 -- 172.21.15.7:0/1910662673 wait complete. 2024-09-13T18:52:18.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23f190700 1 Processor -- start 2024-09-13T18:52:18.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23f190700 1 -- start start 2024-09-13T18:52:18.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23f190700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23806aaf0 0x7fd238143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23f190700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd23806bbf0 0x7fd2381f0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23f190700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2380b3910 0x7fd2381f0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23f190700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd238143a20 con 0x7fd23806aaf0 2024-09-13T18:52:18.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23f190700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2381f1340 con 0x7fd2380b3910 2024-09-13T18:52:18.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23f190700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd238142e00 con 0x7fd23806bbf0 2024-09-13T18:52:18.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23e18e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23806aaf0 0x7fd238143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23e18e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23806aaf0 0x7fd238143e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55778/0 (socket says 172.21.15.7:55778) 2024-09-13T18:52:18.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23e18e700 1 -- 172.21.15.7:0/224533340 learned_addr learned my addr 172.21.15.7:0/224533340 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:18.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23d98d700 1 --2- 172.21.15.7:0/224533340 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd23806bbf0 0x7fd2381f0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23e98f700 1 --2- 172.21.15.7:0/224533340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2380b3910 0x7fd2381f0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23e18e700 1 -- 172.21.15.7:0/224533340 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd23806bbf0 msgr2=0x7fd2381f0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23e18e700 1 --2- 172.21.15.7:0/224533340 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd23806bbf0 0x7fd2381f0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23e18e700 1 -- 172.21.15.7:0/224533340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2380b3910 msgr2=0x7fd2381f0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23e18e700 1 --2- 172.21.15.7:0/224533340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2380b3910 0x7fd2381f0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23e18e700 1 -- 172.21.15.7:0/224533340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2280094b0 con 0x7fd23806aaf0 2024-09-13T18:52:18.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23e98f700 1 --2- 172.21.15.7:0/224533340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2380b3910 0x7fd2381f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23d98d700 1 --2- 172.21.15.7:0/224533340 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd23806bbf0 0x7fd2381f0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23e18e700 1 --2- 172.21.15.7:0/224533340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23806aaf0 0x7fd238143e40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd2280090b0 tx=0x7fd228004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd22effd700 1 -- 172.21.15.7:0/224533340 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2280045d0 con 0x7fd23806aaf0 2024-09-13T18:52:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd22effd700 1 -- 172.21.15.7:0/224533340 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22801f070 con 0x7fd23806aaf0 2024-09-13T18:52:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23f190700 1 -- 172.21.15.7:0/224533340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2381f1090 con 0x7fd23806aaf0 2024-09-13T18:52:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd22effd700 1 -- 172.21.15.7:0/224533340 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2280038e0 con 0x7fd23806aaf0 2024-09-13T18:52:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.382+0000 7fd23f190700 1 -- 172.21.15.7:0/224533340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd238143360 con 0x7fd23806aaf0 2024-09-13T18:52:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.386+0000 7fd23f190700 1 -- 172.21.15.7:0/224533340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd238143b60 con 0x7fd23806aaf0 2024-09-13T18:52:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.386+0000 7fd22effd700 1 -- 172.21.15.7:0/224533340 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd228003a40 con 0x7fd23806aaf0 2024-09-13T18:52:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.386+0000 7fd22effd700 1 --2- 172.21.15.7:0/224533340 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd21c066730 0x7fd21c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.386+0000 7fd22effd700 1 -- 172.21.15.7:0/224533340 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd228021070 con 0x7fd23806aaf0 2024-09-13T18:52:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.386+0000 7fd23d98d700 1 --2- 172.21.15.7:0/224533340 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd21c066730 0x7fd21c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.386+0000 7fd23d98d700 1 --2- 172.21.15.7:0/224533340 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd21c066730 0x7fd21c068bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd238155650 tx=0x7fd234008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:18.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.398+0000 7fd22effd700 1 -- 172.21.15.7:0/224533340 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd238143b60 con 0x7fd23806aaf0 2024-09-13T18:52:18.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.574+0000 7fd23f190700 1 -- 172.21.15.7:0/224533340 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd238143b60 con 0x7fd23806aaf0 2024-09-13T18:52:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.578+0000 7fd22effd700 1 -- 172.21.15.7:0/224533340 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16470 (secure 0 0 0) 0x7fd238143b60 con 0x7fd23806aaf0 2024-09-13T18:52:18.583 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:18.584 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:17.908202+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.582+0000 7fd23f190700 1 -- 172.21.15.7:0/224533340 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd21c066730 msgr2=0x7fd21c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.582+0000 7fd23f190700 1 --2- 172.21.15.7:0/224533340 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd21c066730 0x7fd21c068bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd238155650 tx=0x7fd234008040 comp rx=0 tx=0).stop 2024-09-13T18:52:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.582+0000 7fd23f190700 1 -- 172.21.15.7:0/224533340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23806aaf0 msgr2=0x7fd238143e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.582+0000 7fd23f190700 1 --2- 172.21.15.7:0/224533340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23806aaf0 0x7fd238143e40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd2280090b0 tx=0x7fd228004770 comp rx=0 tx=0).stop 2024-09-13T18:52:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.582+0000 7fd23f190700 1 -- 172.21.15.7:0/224533340 shutdown_connections 2024-09-13T18:52:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.582+0000 7fd23f190700 1 --2- 172.21.15.7:0/224533340 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd21c066730 0x7fd21c068bf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.582+0000 7fd23f190700 1 --2- 172.21.15.7:0/224533340 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2380b3910 0x7fd2381f0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.582+0000 7fd23f190700 1 --2- 172.21.15.7:0/224533340 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd23806bbf0 0x7fd2381f0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.582+0000 7fd23f190700 1 --2- 172.21.15.7:0/224533340 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23806aaf0 0x7fd238143e40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.582+0000 7fd23f190700 1 -- 172.21.15.7:0/224533340 >> 172.21.15.7:0/224533340 conn(0x7fd2380b0f00 msgr2=0x7fd238159070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.582+0000 7fd23f190700 1 -- 172.21.15.7:0/224533340 shutdown_connections 2024-09-13T18:52:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.582+0000 7fd23f190700 1 -- 172.21.15.7:0/224533340 wait complete. 2024-09-13T18:52:18.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T18:52:18.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.714+0000 7f4cfb376700 1 Processor -- start 2024-09-13T18:52:18.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.714+0000 7f4cfb376700 1 -- start start 2024-09-13T18:52:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cfb376700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4cf415cda0 0x7f4cf406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cfb376700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4cf40ceb70 0x7f4cf40b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cfb376700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf40b8430 con 0x7f4cf406a520 2024-09-13T18:52:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cfb376700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf406be40 con 0x7f4cf415cda0 2024-09-13T18:52:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cfb376700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf40cf350 con 0x7f4cf40ceb70 2024-09-13T18:52:18.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cfa374700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4cf415cda0 0x7f4cf406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cfa374700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4cf415cda0 0x7f4cf406b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55784/0 (socket says 172.21.15.7:55784) 2024-09-13T18:52:18.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cfa374700 1 -- 172.21.15.7:0/3741287230 learned_addr learned my addr 172.21.15.7:0/3741287230 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:18.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cf9b73700 1 --2- 172.21.15.7:0/3741287230 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4cf40ceb70 0x7f4cf40b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cfa374700 1 -- 172.21.15.7:0/3741287230 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4cf40ceb70 msgr2=0x7f4cf40b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:18.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cfa374700 1 --2- 172.21.15.7:0/3741287230 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4cf40ceb70 0x7f4cf40b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cfa374700 1 -- 172.21.15.7:0/3741287230 >> v1:172.21.15.171:6789/0 conn(0x7f4cf406a520 legacy=0x7f4cf40d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cfa374700 1 -- 172.21.15.7:0/3741287230 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf40d0880 con 0x7f4cf415cda0 2024-09-13T18:52:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cfa374700 1 --2- 172.21.15.7:0/3741287230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf415cda0 0x7f4cf406b900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009910 tx=0x7f4cf0009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6fa6a83091e832 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cf9372700 1 -- 172.21.15.7:0/3741287230 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0011830 con 0x7f4cf415cda0 2024-09-13T18:52:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cf9372700 1 -- 172.21.15.7:0/3741287230 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf0011990 con 0x7f4cf415cda0 2024-09-13T18:52:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cf9372700 1 -- 172.21.15.7:0/3741287230 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0011d40 con 0x7f4cf415cda0 2024-09-13T18:52:18.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cfb376700 1 -- 172.21.15.7:0/3741287230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf415cda0 msgr2=0x7f4cf406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.718+0000 7f4cfb376700 1 --2- 172.21.15.7:0/3741287230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf415cda0 0x7f4cf406b900 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009910 tx=0x7f4cf0009c20 comp rx=0 tx=0).stop 2024-09-13T18:52:18.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.722+0000 7f4cfb376700 1 -- 172.21.15.7:0/3741287230 shutdown_connections 2024-09-13T18:52:18.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.722+0000 7f4cfb376700 1 --2- 172.21.15.7:0/3741287230 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4cf40ceb70 0x7f4cf40b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.722+0000 7f4cfb376700 1 --2- 172.21.15.7:0/3741287230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf415cda0 0x7f4cf406b900 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.722+0000 7f4cfb376700 1 -- 172.21.15.7:0/3741287230 >> 172.21.15.7:0/3741287230 conn(0x7f4cf40d2cb0 msgr2=0x7f4cf40d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:18.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.722+0000 7f4cfb376700 1 -- 172.21.15.7:0/3741287230 shutdown_connections 2024-09-13T18:52:18.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.722+0000 7f4cfb376700 1 -- 172.21.15.7:0/3741287230 wait complete. 2024-09-13T18:52:18.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.722+0000 7f4cfb376700 1 Processor -- start 2024-09-13T18:52:18.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.722+0000 7f4cfb376700 1 -- start start 2024-09-13T18:52:18.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfb376700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cf406a520 0x7f4cf4146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfb376700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4cf40ceb70 0x7f4cf4152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfb376700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf415cda0 0x7f4cf41584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfb376700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf40c7c90 con 0x7f4cf415cda0 2024-09-13T18:52:18.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfb376700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf40c8810 con 0x7f4cf406a520 2024-09-13T18:52:18.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfb376700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf40c7ee0 con 0x7f4cf40ceb70 2024-09-13T18:52:18.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfab75700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf415cda0 0x7f4cf41584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfa374700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cf406a520 0x7f4cf4146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfab75700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf415cda0 0x7f4cf41584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55798/0 (socket says 172.21.15.7:55798) 2024-09-13T18:52:18.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfab75700 1 -- 172.21.15.7:0/2871406278 learned_addr learned my addr 172.21.15.7:0/2871406278 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cf9b73700 1 --2- 172.21.15.7:0/2871406278 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4cf40ceb70 0x7f4cf4152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfab75700 1 -- 172.21.15.7:0/2871406278 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4cf40ceb70 msgr2=0x7f4cf4152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfab75700 1 --2- 172.21.15.7:0/2871406278 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4cf40ceb70 0x7f4cf4152960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfab75700 1 -- 172.21.15.7:0/2871406278 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cf406a520 msgr2=0x7f4cf4146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfab75700 1 --2- 172.21.15.7:0/2871406278 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cf406a520 0x7f4cf4146ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfab75700 1 -- 172.21.15.7:0/2871406278 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf0009570 con 0x7f4cf415cda0 2024-09-13T18:52:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfab75700 1 --2- 172.21.15.7:0/2871406278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf415cda0 0x7f4cf41584b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4cec00cc50 tx=0x7f4cec00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4ceaffd700 1 -- 172.21.15.7:0/2871406278 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec003c20 con 0x7f4cf415cda0 2024-09-13T18:52:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4ceaffd700 1 -- 172.21.15.7:0/2871406278 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cec003d80 con 0x7f4cf415cda0 2024-09-13T18:52:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfb376700 1 -- 172.21.15.7:0/2871406278 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf40c81c0 con 0x7f4cf415cda0 2024-09-13T18:52:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4ceaffd700 1 -- 172.21.15.7:0/2871406278 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec00aab0 con 0x7f4cf415cda0 2024-09-13T18:52:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfb376700 1 -- 172.21.15.7:0/2871406278 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf41589f0 con 0x7f4cf415cda0 2024-09-13T18:52:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.726+0000 7f4cfb376700 1 -- 172.21.15.7:0/2871406278 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf40c8980 con 0x7f4cf415cda0 2024-09-13T18:52:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.730+0000 7f4ceaffd700 1 -- 172.21.15.7:0/2871406278 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4cec00ac10 con 0x7f4cf415cda0 2024-09-13T18:52:18.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.730+0000 7f4ceaffd700 1 --2- 172.21.15.7:0/2871406278 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4cd8066730 0x7f4cd8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:18.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.730+0000 7f4ceaffd700 1 -- 172.21.15.7:0/2871406278 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4cec0841a0 con 0x7f4cf415cda0 2024-09-13T18:52:18.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.730+0000 7f4cfa374700 1 --2- 172.21.15.7:0/2871406278 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4cd8066730 0x7f4cd8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.730+0000 7f4cfa374700 1 --2- 172.21.15.7:0/2871406278 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4cd8066730 0x7f4cd8068bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0000c00 tx=0x7f4cf0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:18.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.738+0000 7f4ceaffd700 1 -- 172.21.15.7:0/2871406278 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4cec051470 con 0x7f4cf415cda0 2024-09-13T18:52:18.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:18.942+0000 7f4cfb376700 1 -- 172.21.15.7:0/2871406278 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4cf4158b80 con 0x7f4cf415cda0 2024-09-13T18:52:19.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:19.930+0000 7f4ceaffd700 1 -- 172.21.15.7:0/2871406278 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4cf4158b80 con 0x7f4cf415cda0 2024-09-13T18:52:19.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:19.934+0000 7f4cfb376700 1 -- 172.21.15.7:0/2871406278 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4cd8066730 msgr2=0x7f4cd8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:19.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:19.934+0000 7f4cfb376700 1 --2- 172.21.15.7:0/2871406278 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4cd8066730 0x7f4cd8068bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0000c00 tx=0x7f4cf0008030 comp rx=0 tx=0).stop 2024-09-13T18:52:19.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:19.934+0000 7f4cfb376700 1 -- 172.21.15.7:0/2871406278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf415cda0 msgr2=0x7f4cf41584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:19.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:19.934+0000 7f4cfb376700 1 --2- 172.21.15.7:0/2871406278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf415cda0 0x7f4cf41584b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4cec00cc50 tx=0x7f4cec00cf60 comp rx=0 tx=0).stop 2024-09-13T18:52:19.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:19.934+0000 7f4cfb376700 1 -- 172.21.15.7:0/2871406278 shutdown_connections 2024-09-13T18:52:19.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:19.934+0000 7f4cfb376700 1 --2- 172.21.15.7:0/2871406278 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4cd8066730 0x7f4cd8068bf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:19.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:19.934+0000 7f4cfb376700 1 --2- 172.21.15.7:0/2871406278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cf415cda0 0x7f4cf41584b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:19.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:19.934+0000 7f4cfb376700 1 --2- 172.21.15.7:0/2871406278 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4cf40ceb70 0x7f4cf4152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:19.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:19.934+0000 7f4cfb376700 1 --2- 172.21.15.7:0/2871406278 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cf406a520 0x7f4cf4146ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:19.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:19.938+0000 7f4cfb376700 1 -- 172.21.15.7:0/2871406278 >> 172.21.15.7:0/2871406278 conn(0x7f4cf40d2cb0 msgr2=0x7f4cf40b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:19.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:19.938+0000 7f4cfb376700 1 -- 172.21.15.7:0/2871406278 shutdown_connections 2024-09-13T18:52:19.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:19.938+0000 7f4cfb376700 1 -- 172.21.15.7:0/2871406278 wait complete. 2024-09-13T18:52:19.959 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T18:52:19.959 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T18:52:20.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.078+0000 7fe2f8911700 1 Processor -- start 2024-09-13T18:52:20.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.078+0000 7fe2f8911700 1 -- start start 2024-09-13T18:52:20.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f8911700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fe2f406b900 0x7fe2f40b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:20.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f8911700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe2f406a520 0x7fe2f406a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:20.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f8911700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f40b8430 con 0x7fe2f415cda0 2024-09-13T18:52:20.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f8911700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f406ae70 con 0x7fe2f406a520 2024-09-13T18:52:20.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f8911700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f406bd60 con 0x7fe2f406b900 2024-09-13T18:52:20.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f359e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe2f406a520 0x7fe2f406a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:20.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f2d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe2f415cda0 0x7fe2f40ceb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52192/0 (socket says 172.21.15.7:52192) 2024-09-13T18:52:20.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f2d9d700 1 -- 172.21.15.7:0/1778332627 learned_addr learned my addr 172.21.15.7:0/1778332627 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:20.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f259c700 1 --2- 172.21.15.7:0/1778332627 >> v2:172.21.15.171:3300/0 conn(0x7fe2f406b900 0x7fe2f40b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:20.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f359e700 1 -- 172.21.15.7:0/1778332627 >> v2:172.21.15.171:3300/0 conn(0x7fe2f406b900 msgr2=0x7fe2f40b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:20.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f359e700 1 --2- 172.21.15.7:0/1778332627 >> v2:172.21.15.171:3300/0 conn(0x7fe2f406b900 0x7fe2f40b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:20.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f359e700 1 -- 172.21.15.7:0/1778332627 >> v1:172.21.15.7:6789/0 conn(0x7fe2f415cda0 legacy=0x7fe2f40ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:20.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f1d9b700 1 -- 172.21.15.7:0/1778332627 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240513565 0 0) 0x7fe2f40b8430 con 0x7fe2f415cda0 2024-09-13T18:52:20.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f359e700 1 -- 172.21.15.7:0/1778332627 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f40cf350 con 0x7fe2f406a520 2024-09-13T18:52:20.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f259c700 1 --2- 172.21.15.7:0/1778332627 >> v2:172.21.15.171:3300/0 conn(0x7fe2f406b900 0x7fe2f40b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:20.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f359e700 1 --2- 172.21.15.7:0/1778332627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f406a520 0x7fe2f406a930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40cf280 tx=0x7fe2ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e47a18a889938e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:20.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f1d9b700 1 -- 172.21.15.7:0/1778332627 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec00e070 con 0x7fe2f406a520 2024-09-13T18:52:20.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f1d9b700 1 -- 172.21.15.7:0/1778332627 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec00bb20 con 0x7fe2f406a520 2024-09-13T18:52:20.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f1d9b700 1 -- 172.21.15.7:0/1778332627 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec012430 con 0x7fe2f406a520 2024-09-13T18:52:20.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f8911700 1 -- 172.21.15.7:0/1778332627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f406a520 msgr2=0x7fe2f406a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:20.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f8911700 1 --2- 172.21.15.7:0/1778332627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f406a520 0x7fe2f406a930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40cf280 tx=0x7fe2ec00b330 comp rx=0 tx=0).stop 2024-09-13T18:52:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f8911700 1 -- 172.21.15.7:0/1778332627 shutdown_connections 2024-09-13T18:52:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f8911700 1 --2- 172.21.15.7:0/1778332627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f406a520 0x7fe2f406a930 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f8911700 1 --2- 172.21.15.7:0/1778332627 >> v2:172.21.15.171:3300/0 conn(0x7fe2f406b900 0x7fe2f40b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f8911700 1 -- 172.21.15.7:0/1778332627 >> 172.21.15.7:0/1778332627 conn(0x7fe2f40d2cb0 msgr2=0x7fe2f40d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f8911700 1 -- 172.21.15.7:0/1778332627 shutdown_connections 2024-09-13T18:52:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.082+0000 7fe2f8911700 1 -- 172.21.15.7:0/1778332627 wait complete. 2024-09-13T18:52:20.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.086+0000 7fe2f8911700 1 Processor -- start 2024-09-13T18:52:20.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.086+0000 7fe2f8911700 1 -- start start 2024-09-13T18:52:20.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.086+0000 7fe2f8911700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f406a520 0x7fe2f40c7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.086+0000 7fe2f8911700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2f406b900 0x7fe2f41f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.086+0000 7fe2f8911700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe2f415cda0 0x7fe2f41f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.086+0000 7fe2f8911700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f40c8b10 con 0x7fe2f406a520 2024-09-13T18:52:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.086+0000 7fe2f8911700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f41f34c0 con 0x7fe2f406b900 2024-09-13T18:52:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.086+0000 7fe2f8911700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f4155dc0 con 0x7fe2f415cda0 2024-09-13T18:52:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.086+0000 7fe2f2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f406a520 0x7fe2f40c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.086+0000 7fe2f2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f406a520 0x7fe2f40c7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55812/0 (socket says 172.21.15.7:55812) 2024-09-13T18:52:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.086+0000 7fe2f2d9d700 1 -- 172.21.15.7:0/2277713324 learned_addr learned my addr 172.21.15.7:0/2277713324 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.086+0000 7fe2f259c700 1 --2- 172.21.15.7:0/2277713324 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2f406b900 0x7fe2f41f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.086+0000 7fe2f359e700 1 --2- 172.21.15.7:0/2277713324 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe2f415cda0 0x7fe2f41f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.086+0000 7fe2f2d9d700 1 -- 172.21.15.7:0/2277713324 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe2f415cda0 msgr2=0x7fe2f41f1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.090+0000 7fe2f2d9d700 1 --2- 172.21.15.7:0/2277713324 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe2f415cda0 0x7fe2f41f1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.090+0000 7fe2f2d9d700 1 -- 172.21.15.7:0/2277713324 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2f406b900 msgr2=0x7fe2f41f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.090+0000 7fe2f2d9d700 1 --2- 172.21.15.7:0/2277713324 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2f406b900 0x7fe2f41f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.090+0000 7fe2f2d9d700 1 -- 172.21.15.7:0/2277713324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e40094b0 con 0x7fe2f406a520 2024-09-13T18:52:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.090+0000 7fe2f359e700 1 --2- 172.21.15.7:0/2277713324 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe2f415cda0 0x7fe2f41f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.090+0000 7fe2f2d9d700 1 --2- 172.21.15.7:0/2277713324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f406a520 0x7fe2f40c7410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe2e800c4e0 tx=0x7fe2e800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.090+0000 7fe2db7fe700 1 -- 172.21.15.7:0/2277713324 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8004ca0 con 0x7fe2f406a520 2024-09-13T18:52:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.090+0000 7fe2db7fe700 1 -- 172.21.15.7:0/2277713324 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e8004e20 con 0x7fe2f406a520 2024-09-13T18:52:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.090+0000 7fe2f8911700 1 -- 172.21.15.7:0/2277713324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ec009a70 con 0x7fe2f406a520 2024-09-13T18:52:20.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.090+0000 7fe2db7fe700 1 -- 172.21.15.7:0/2277713324 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e8007310 con 0x7fe2f406a520 2024-09-13T18:52:20.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.090+0000 7fe2f8911700 1 -- 172.21.15.7:0/2277713324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f40c6f80 con 0x7fe2f406a520 2024-09-13T18:52:20.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.090+0000 7fe2db7fe700 1 -- 172.21.15.7:0/2277713324 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe2e8007470 con 0x7fe2f406a520 2024-09-13T18:52:20.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.090+0000 7fe2f8911700 1 -- 172.21.15.7:0/2277713324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f406a930 con 0x7fe2f406a520 2024-09-13T18:52:20.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.090+0000 7fe2db7fe700 1 --2- 172.21.15.7:0/2277713324 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe2e0066730 0x7fe2e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:20.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.090+0000 7fe2db7fe700 1 -- 172.21.15.7:0/2277713324 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2e8083de0 con 0x7fe2f406a520 2024-09-13T18:52:20.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.094+0000 7fe2f259c700 1 --2- 172.21.15.7:0/2277713324 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe2e0066730 0x7fe2e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:20.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.094+0000 7fe2f259c700 1 --2- 172.21.15.7:0/2277713324 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe2e0066730 0x7fe2e0068bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4155910 tx=0x7fe2e4009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:20.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.098+0000 7fe2db7fe700 1 -- 172.21.15.7:0/2277713324 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe2e8051160 con 0x7fe2f406a520 2024-09-13T18:52:20.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.314+0000 7fe2f8911700 1 -- 172.21.15.7:0/2277713324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe2f40cece0 con 0x7fe2f406a520 2024-09-13T18:52:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.938+0000 7fe2db7fe700 1 -- 172.21.15.7:0/2277713324 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe2e8007720 con 0x7fe2f406a520 2024-09-13T18:52:20.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.942+0000 7fe2f8911700 1 -- 172.21.15.7:0/2277713324 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe2e0066730 msgr2=0x7fe2e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:20.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.942+0000 7fe2f8911700 1 --2- 172.21.15.7:0/2277713324 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe2e0066730 0x7fe2e0068bf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4155910 tx=0x7fe2e4009240 comp rx=0 tx=0).stop 2024-09-13T18:52:20.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.942+0000 7fe2f8911700 1 -- 172.21.15.7:0/2277713324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f406a520 msgr2=0x7fe2f40c7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.942+0000 7fe2f8911700 1 --2- 172.21.15.7:0/2277713324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f406a520 0x7fe2f40c7410 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe2e800c4e0 tx=0x7fe2e800c7f0 comp rx=0 tx=0).stop 2024-09-13T18:52:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.942+0000 7fe2f8911700 1 -- 172.21.15.7:0/2277713324 shutdown_connections 2024-09-13T18:52:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.942+0000 7fe2f8911700 1 --2- 172.21.15.7:0/2277713324 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe2e0066730 0x7fe2e0068bf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.942+0000 7fe2f8911700 1 --2- 172.21.15.7:0/2277713324 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe2f415cda0 0x7fe2f41f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.942+0000 7fe2f8911700 1 --2- 172.21.15.7:0/2277713324 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe2f406b900 0x7fe2f41f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.942+0000 7fe2f8911700 1 --2- 172.21.15.7:0/2277713324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2f406a520 0x7fe2f40c7410 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.942+0000 7fe2f8911700 1 -- 172.21.15.7:0/2277713324 >> 172.21.15.7:0/2277713324 conn(0x7fe2f40d2cb0 msgr2=0x7fe2f40d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.942+0000 7fe2f8911700 1 -- 172.21.15.7:0/2277713324 shutdown_connections 2024-09-13T18:52:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:20.942+0000 7fe2f8911700 1 -- 172.21.15.7:0/2277713324 wait complete. 2024-09-13T18:52:20.967 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T18:52:20.967 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:21.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc68ffff700 1 Processor -- start 2024-09-13T18:52:21.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc68ffff700 1 -- start start 2024-09-13T18:52:21.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc68ffff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fc690157040 0x7fc6900ae180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:21.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc68ffff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc69015d0f0 0x7fc6900b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:21.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc68ffff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69015d550 con 0x7fc6900b2ee0 2024-09-13T18:52:21.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc68ffff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69015e0c0 con 0x7fc690157040 2024-09-13T18:52:21.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc68ffff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69015d820 con 0x7fc69015d0f0 2024-09-13T18:52:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc68f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc6900b2ee0 0x7fc6900acf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52202/0 (socket says 172.21.15.7:52202) 2024-09-13T18:52:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc68f7fe700 1 -- 172.21.15.7:0/4099180635 learned_addr learned my addr 172.21.15.7:0/4099180635 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc6867fc700 1 --2- 172.21.15.7:0/4099180635 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc69015d0f0 0x7fc6900b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:21.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc68effd700 1 --2- 172.21.15.7:0/4099180635 >> v2:172.21.15.171:3300/0 conn(0x7fc690157040 0x7fc6900ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc68e7fc700 1 -- 172.21.15.7:0/4099180635 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3271023700 0 0) 0x7fc69015d550 con 0x7fc6900b2ee0 2024-09-13T18:52:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc68e7fc700 1 -- 172.21.15.7:0/4099180635 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc67c0035d0 con 0x7fc6900b2ee0 2024-09-13T18:52:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc6867fc700 1 -- 172.21.15.7:0/4099180635 >> v2:172.21.15.171:3300/0 conn(0x7fc690157040 msgr2=0x7fc6900ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc6867fc700 1 --2- 172.21.15.7:0/4099180635 >> v2:172.21.15.171:3300/0 conn(0x7fc690157040 0x7fc6900ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc6867fc700 1 -- 172.21.15.7:0/4099180635 >> v1:172.21.15.7:6789/0 conn(0x7fc6900b2ee0 legacy=0x7fc6900acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc6867fc700 1 -- 172.21.15.7:0/4099180635 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69015db00 con 0x7fc69015d0f0 2024-09-13T18:52:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc6867fc700 1 --2- 172.21.15.7:0/4099180635 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc69015d0f0 0x7fc6900b29a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc678009850 tx=0x7fc678009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae01199d5c6fbfb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.082+0000 7fc68e7fc700 1 -- 172.21.15.7:0/4099180635 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678011830 con 0x7fc69015d0f0 2024-09-13T18:52:21.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.086+0000 7fc68e7fc700 1 -- 172.21.15.7:0/4099180635 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc678011990 con 0x7fc69015d0f0 2024-09-13T18:52:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.086+0000 7fc68e7fc700 1 -- 172.21.15.7:0/4099180635 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678011d40 con 0x7fc69015d0f0 2024-09-13T18:52:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.086+0000 7fc68ffff700 1 -- 172.21.15.7:0/4099180635 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc69015d0f0 msgr2=0x7fc6900b29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.086+0000 7fc68ffff700 1 --2- 172.21.15.7:0/4099180635 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc69015d0f0 0x7fc6900b29a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc678009850 tx=0x7fc678009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.086+0000 7fc68ffff700 1 -- 172.21.15.7:0/4099180635 shutdown_connections 2024-09-13T18:52:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.086+0000 7fc68ffff700 1 --2- 172.21.15.7:0/4099180635 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc69015d0f0 0x7fc6900b29a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.086+0000 7fc68ffff700 1 --2- 172.21.15.7:0/4099180635 >> v2:172.21.15.171:3300/0 conn(0x7fc690157040 0x7fc6900ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.086+0000 7fc68ffff700 1 -- 172.21.15.7:0/4099180635 >> 172.21.15.7:0/4099180635 conn(0x7fc6900cf910 msgr2=0x7fc6900cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.086+0000 7fc68ffff700 1 -- 172.21.15.7:0/4099180635 shutdown_connections 2024-09-13T18:52:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.086+0000 7fc68ffff700 1 -- 172.21.15.7:0/4099180635 wait complete. 2024-09-13T18:52:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.086+0000 7fc68ffff700 1 Processor -- start 2024-09-13T18:52:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.086+0000 7fc68ffff700 1 -- start start 2024-09-13T18:52:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68ffff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6900b2ee0 0x7fc69006d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68ffff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc690157040 0x7fc69006ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc69015d0f0 0x7fc69015c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68ffff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69015ca50 con 0x7fc69015d0f0 2024-09-13T18:52:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68ffff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69015cd30 con 0x7fc6900b2ee0 2024-09-13T18:52:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68ffff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc690157720 con 0x7fc690157040 2024-09-13T18:52:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc69015d0f0 0x7fc69015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc69015d0f0 0x7fc69015c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55818/0 (socket says 172.21.15.7:55818) 2024-09-13T18:52:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68f7fe700 1 -- 172.21.15.7:0/3073784900 learned_addr learned my addr 172.21.15.7:0/3073784900 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68effd700 1 --2- 172.21.15.7:0/3073784900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6900b2ee0 0x7fc69006d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc6867fc700 1 --2- 172.21.15.7:0/3073784900 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc690157040 0x7fc69006ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68f7fe700 1 -- 172.21.15.7:0/3073784900 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc690157040 msgr2=0x7fc69006ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68f7fe700 1 --2- 172.21.15.7:0/3073784900 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc690157040 0x7fc69006ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68f7fe700 1 -- 172.21.15.7:0/3073784900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6900b2ee0 msgr2=0x7fc69006d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68f7fe700 1 --2- 172.21.15.7:0/3073784900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6900b2ee0 0x7fc69006d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68f7fe700 1 -- 172.21.15.7:0/3073784900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6780094b0 con 0x7fc69015d0f0 2024-09-13T18:52:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68effd700 1 --2- 172.21.15.7:0/3073784900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6900b2ee0 0x7fc69006d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68f7fe700 1 --2- 172.21.15.7:0/3073784900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc69015d0f0 0x7fc69015c220 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc68800cb40 tx=0x7fc68800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc687fff700 1 -- 172.21.15.7:0/3073784900 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68800a9e0 con 0x7fc69015d0f0 2024-09-13T18:52:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc687fff700 1 -- 172.21.15.7:0/3073784900 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68800ab40 con 0x7fc69015d0f0 2024-09-13T18:52:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68ffff700 1 -- 172.21.15.7:0/3073784900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc690157a00 con 0x7fc69015d0f0 2024-09-13T18:52:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc687fff700 1 -- 172.21.15.7:0/3073784900 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68800ce60 con 0x7fc69015d0f0 2024-09-13T18:52:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68ffff700 1 -- 172.21.15.7:0/3073784900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69015c760 con 0x7fc69015d0f0 2024-09-13T18:52:21.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.090+0000 7fc68ffff700 1 -- 172.21.15.7:0/3073784900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6900ae180 con 0x7fc69015d0f0 2024-09-13T18:52:21.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.094+0000 7fc687fff700 1 -- 172.21.15.7:0/3073784900 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc68801b3b0 con 0x7fc69015d0f0 2024-09-13T18:52:21.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.094+0000 7fc687fff700 1 --2- 172.21.15.7:0/3073784900 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc670066730 0x7fc670068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:21.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.094+0000 7fc687fff700 1 -- 172.21.15.7:0/3073784900 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc688083c60 con 0x7fc69015d0f0 2024-09-13T18:52:21.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.094+0000 7fc68effd700 1 --2- 172.21.15.7:0/3073784900 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc670066730 0x7fc670068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:21.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.094+0000 7fc68effd700 1 --2- 172.21.15.7:0/3073784900 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc670066730 0x7fc670068bf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc6900b0fc0 tx=0x7fc680008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:21.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.098+0000 7fc687fff700 1 -- 172.21.15.7:0/3073784900 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc688050fc0 con 0x7fc69015d0f0 2024-09-13T18:52:21.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.310+0000 7fc68ffff700 1 -- 172.21.15.7:0/3073784900 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6900d0120 con 0x7fc69015d0f0 2024-09-13T18:52:21.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.310+0000 7fc687fff700 1 -- 172.21.15.7:0/3073784900 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4725 (secure 0 0 0) 0x7fc688017090 con 0x7fc69015d0f0 2024-09-13T18:52:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:21.316 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1309419125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1309419125},{"type":"v1","addr":"172.21.15.7:6835","nonce":1309419125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/951828843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":951828843},{"type":"v1","addr":"172.21.15.7:6833","nonce":951828843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4226627580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4226627580},{"type":"v1","addr":"172.21.15.171:6835","nonce":4226627580}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:52:17.908937+0000","modified":"2024-09-13T18:52:20.930515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6837/3658296803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3658296803},{"type":"v1","addr":"172.21.15.171:6837","nonce":3658296803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.314+0000 7fc68ffff700 1 -- 172.21.15.7:0/3073784900 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc670066730 msgr2=0x7fc670068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.314+0000 7fc68ffff700 1 --2- 172.21.15.7:0/3073784900 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc670066730 0x7fc670068bf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc6900b0fc0 tx=0x7fc680008040 comp rx=0 tx=0).stop 2024-09-13T18:52:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.314+0000 7fc68ffff700 1 -- 172.21.15.7:0/3073784900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc69015d0f0 msgr2=0x7fc69015c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.314+0000 7fc68ffff700 1 --2- 172.21.15.7:0/3073784900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc69015d0f0 0x7fc69015c220 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc68800cb40 tx=0x7fc68800cc20 comp rx=0 tx=0).stop 2024-09-13T18:52:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.314+0000 7fc68ffff700 1 -- 172.21.15.7:0/3073784900 shutdown_connections 2024-09-13T18:52:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.314+0000 7fc68ffff700 1 --2- 172.21.15.7:0/3073784900 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc670066730 0x7fc670068bf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.314+0000 7fc68ffff700 1 --2- 172.21.15.7:0/3073784900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc69015d0f0 0x7fc69015c220 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.314+0000 7fc68ffff700 1 --2- 172.21.15.7:0/3073784900 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc690157040 0x7fc69006ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.314+0000 7fc68ffff700 1 --2- 172.21.15.7:0/3073784900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc6900b2ee0 0x7fc69006d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.314+0000 7fc68ffff700 1 -- 172.21.15.7:0/3073784900 >> 172.21.15.7:0/3073784900 conn(0x7fc6900cf910 msgr2=0x7fc69015f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:21.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.314+0000 7fc68ffff700 1 -- 172.21.15.7:0/3073784900 shutdown_connections 2024-09-13T18:52:21.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.314+0000 7fc68ffff700 1 -- 172.21.15.7:0/3073784900 wait complete. 2024-09-13T18:52:21.319 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-13T18:52:21.331 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:21.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.438+0000 7f5afb59e700 1 Processor -- start 2024-09-13T18:52:21.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5afb59e700 1 -- start start 2024-09-13T18:52:21.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5afb59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5af4033510 0x7f5af401db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:21.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5afb59e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af4033e90 con 0x7f5af4036d80 2024-09-13T18:52:21.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5afb59e700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af4033bb0 con 0x7f5af404ec10 2024-09-13T18:52:21.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5afb59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af401d080 con 0x7f5af4033510 2024-09-13T18:52:21.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5afad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5af4033510 0x7f5af401db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:21.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5af9d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5af4036d80 0x7f5af4031240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35692/0 (socket says 172.21.15.7:35692) 2024-09-13T18:52:21.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5af9d9b700 1 -- 172.21.15.7:0/3537929203 learned_addr learned my addr 172.21.15.7:0/3537929203 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:21.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5af959a700 1 -- 172.21.15.7:0/3537929203 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 833477462 0 0) 0x7f5af4033e90 con 0x7f5af4036d80 2024-09-13T18:52:21.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5af959a700 1 -- 172.21.15.7:0/3537929203 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ae40035d0 con 0x7f5af4036d80 2024-09-13T18:52:21.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5afad9d700 1 -- 172.21.15.7:0/3537929203 >> v1:172.21.15.171:6789/0 conn(0x7f5af404ec10 legacy=0x7f5af404eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5af959a700 1 -- 172.21.15.7:0/3537929203 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2510280693 0 0) 0x7f5af4033bb0 con 0x7f5af404ec10 2024-09-13T18:52:21.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5afad9d700 1 -- 172.21.15.7:0/3537929203 >> v1:172.21.15.7:6789/0 conn(0x7f5af4036d80 legacy=0x7f5af4031240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5afad9d700 1 -- 172.21.15.7:0/3537929203 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af401d360 con 0x7f5af4033510 2024-09-13T18:52:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5afad9d700 1 --2- 172.21.15.7:0/3537929203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5af4033510 0x7f5af401db60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5afc04f480 tx=0x7f5afc074840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb3c54e29ef47d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5af959a700 1 -- 172.21.15.7:0/3537929203 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc04f010 con 0x7f5af4033510 2024-09-13T18:52:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5af959a700 1 -- 172.21.15.7:0/3537929203 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5afc0730f0 con 0x7f5af4033510 2024-09-13T18:52:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5af959a700 1 -- 172.21.15.7:0/3537929203 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0734c0 con 0x7f5af4033510 2024-09-13T18:52:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5afb59e700 1 -- 172.21.15.7:0/3537929203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5af4033510 msgr2=0x7f5af401db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5afb59e700 1 --2- 172.21.15.7:0/3537929203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5af4033510 0x7f5af401db60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5afc04f480 tx=0x7f5afc074840 comp rx=0 tx=0).stop 2024-09-13T18:52:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5afb59e700 1 -- 172.21.15.7:0/3537929203 shutdown_connections 2024-09-13T18:52:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5afb59e700 1 --2- 172.21.15.7:0/3537929203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5af4033510 0x7f5af401db60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5afb59e700 1 -- 172.21.15.7:0/3537929203 >> 172.21.15.7:0/3537929203 conn(0x7f5af4036140 msgr2=0x7f5af4036550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5afb59e700 1 -- 172.21.15.7:0/3537929203 shutdown_connections 2024-09-13T18:52:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.442+0000 7f5afb59e700 1 -- 172.21.15.7:0/3537929203 wait complete. 2024-09-13T18:52:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afb59e700 1 Processor -- start 2024-09-13T18:52:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afb59e700 1 -- start start 2024-09-13T18:52:21.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5af4033510 0x7f5af40dde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:21.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afb59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5af4036d80 0x7f5af400aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:21.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afb59e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5af404ec10 0x7f5af400b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:21.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afb59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4039650 con 0x7f5af4033510 2024-09-13T18:52:21.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afb59e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af40382d0 con 0x7f5af404ec10 2024-09-13T18:52:21.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afb59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af417e000 con 0x7f5af4036d80 2024-09-13T18:52:21.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5af4033510 0x7f5af40dde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5af4033510 0x7f5af40dde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49178/0 (socket says 172.21.15.7:49178) 2024-09-13T18:52:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afa59c700 1 -- 172.21.15.7:0/148640559 learned_addr learned my addr 172.21.15.7:0/148640559 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afad9d700 1 --2- 172.21.15.7:0/148640559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5af404ec10 0x7f5af400b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5af9d9b700 1 --2- 172.21.15.7:0/148640559 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5af4036d80 0x7f5af400aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afa59c700 1 -- 172.21.15.7:0/148640559 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5af4036d80 msgr2=0x7f5af400aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afa59c700 1 --2- 172.21.15.7:0/148640559 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5af4036d80 0x7f5af400aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afa59c700 1 -- 172.21.15.7:0/148640559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5af404ec10 msgr2=0x7f5af400b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afa59c700 1 --2- 172.21.15.7:0/148640559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5af404ec10 0x7f5af400b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afa59c700 1 -- 172.21.15.7:0/148640559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc04f5c0 con 0x7f5af4033510 2024-09-13T18:52:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afad9d700 1 --2- 172.21.15.7:0/148640559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5af404ec10 0x7f5af400b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5af9d9b700 1 --2- 172.21.15.7:0/148640559 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5af4036d80 0x7f5af400aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afa59c700 1 --2- 172.21.15.7:0/148640559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5af4033510 0x7f5af40dde10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00c750 tx=0x7f5aec00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5aeaffd700 1 -- 172.21.15.7:0/148640559 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec0054e0 con 0x7f5af4033510 2024-09-13T18:52:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5aeaffd700 1 -- 172.21.15.7:0/148640559 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec0036f0 con 0x7f5af4033510 2024-09-13T18:52:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5aeaffd700 1 -- 172.21.15.7:0/148640559 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec003aa0 con 0x7f5af4033510 2024-09-13T18:52:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afb59e700 1 -- 172.21.15.7:0/148640559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af417e2e0 con 0x7f5af4033510 2024-09-13T18:52:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.446+0000 7f5afb59e700 1 -- 172.21.15.7:0/148640559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af40385b0 con 0x7f5af4033510 2024-09-13T18:52:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.450+0000 7f5afb59e700 1 -- 172.21.15.7:0/148640559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af4004800 con 0x7f5af4033510 2024-09-13T18:52:21.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.450+0000 7f5aeaffd700 1 -- 172.21.15.7:0/148640559 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5aec003c00 con 0x7f5af4033510 2024-09-13T18:52:21.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.450+0000 7f5aeaffd700 1 --2- 172.21.15.7:0/148640559 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5adc066730 0x7f5adc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:21.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.450+0000 7f5aeaffd700 1 -- 172.21.15.7:0/148640559 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5aec0837c0 con 0x7f5af4033510 2024-09-13T18:52:21.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.450+0000 7f5af9d9b700 1 --2- 172.21.15.7:0/148640559 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5adc066730 0x7f5adc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:21.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.454+0000 7f5af9d9b700 1 --2- 172.21.15.7:0/148640559 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5adc066730 0x7f5adc068bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5af40ea3e0 tx=0x7f5af0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.458+0000 7f5aeaffd700 1 -- 172.21.15.7:0/148640559 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5aec050a90 con 0x7f5af4033510 2024-09-13T18:52:21.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.630+0000 7f5afb59e700 1 -- 172.21.15.7:0/148640559 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5af417e800 con 0x7f5af4033510 2024-09-13T18:52:21.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.634+0000 7f5aeaffd700 1 -- 172.21.15.7:0/148640559 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f5aec017090 con 0x7f5af4033510 2024-09-13T18:52:21.639 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:21.639 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.638+0000 7f5afb59e700 1 -- 172.21.15.7:0/148640559 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5adc066730 msgr2=0x7f5adc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.638+0000 7f5afb59e700 1 --2- 172.21.15.7:0/148640559 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5adc066730 0x7f5adc068bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5af40ea3e0 tx=0x7f5af0009040 comp rx=0 tx=0).stop 2024-09-13T18:52:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.638+0000 7f5afb59e700 1 -- 172.21.15.7:0/148640559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5af4033510 msgr2=0x7f5af40dde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.638+0000 7f5afb59e700 1 --2- 172.21.15.7:0/148640559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5af4033510 0x7f5af40dde10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00c750 tx=0x7f5aec00cb10 comp rx=0 tx=0).stop 2024-09-13T18:52:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.638+0000 7f5afb59e700 1 -- 172.21.15.7:0/148640559 shutdown_connections 2024-09-13T18:52:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.638+0000 7f5afb59e700 1 --2- 172.21.15.7:0/148640559 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5adc066730 0x7f5adc068bf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.638+0000 7f5afb59e700 1 --2- 172.21.15.7:0/148640559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5af404ec10 0x7f5af400b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.638+0000 7f5afb59e700 1 --2- 172.21.15.7:0/148640559 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5af4036d80 0x7f5af400aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.638+0000 7f5afb59e700 1 --2- 172.21.15.7:0/148640559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5af4033510 0x7f5af40dde10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.638+0000 7f5afb59e700 1 -- 172.21.15.7:0/148640559 >> 172.21.15.7:0/148640559 conn(0x7f5af4036140 msgr2=0x7f5af40e4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:21.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.638+0000 7f5afb59e700 1 -- 172.21.15.7:0/148640559 shutdown_connections 2024-09-13T18:52:21.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.638+0000 7f5afb59e700 1 -- 172.21.15.7:0/148640559 wait complete. 2024-09-13T18:52:21.655 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T18:52:21.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:52:21.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb7525a3700 1 Processor -- start 2024-09-13T18:52:21.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb7525a3700 1 -- start start 2024-09-13T18:52:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb7525a3700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb74c0c5120 0x7fb74c0ce130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb7525a3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb74c0ce670 0x7fb74c0c9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb7525a3700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb74c0c40c0 con 0x7fb74c1612c0 2024-09-13T18:52:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb7525a3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb74c0ceb00 con 0x7fb74c0ce670 2024-09-13T18:52:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb7525a3700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c0c4360 con 0x7fb74c0c5120 2024-09-13T18:52:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb750b40700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb74c0ce670 0x7fb74c0c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb750b40700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb74c0ce670 0x7fb74c0c9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49194/0 (socket says 172.21.15.7:49194) 2024-09-13T18:52:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb750b40700 1 -- 172.21.15.7:0/814931595 learned_addr learned my addr 172.21.15.7:0/814931595 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb74b7fe700 1 --2- 172.21.15.7:0/814931595 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb74c0c5120 0x7fb74c0ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:21.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb750b40700 1 -- 172.21.15.7:0/814931595 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb74c0c5120 msgr2=0x7fb74c0ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:21.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb750b40700 1 --2- 172.21.15.7:0/814931595 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb74c0c5120 0x7fb74c0ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb750b40700 1 -- 172.21.15.7:0/814931595 >> v1:172.21.15.171:6789/0 conn(0x7fb74c1612c0 legacy=0x7fb74c071d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb750b40700 1 -- 172.21.15.7:0/814931595 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb74c0c4640 con 0x7fb74c0ce670 2024-09-13T18:52:21.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb750b40700 1 --2- 172.21.15.7:0/814931595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb74c0ce670 0x7fb74c0c9870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009910 tx=0x7fb73c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2682b7bd05cf7e6f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:21.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb74affd700 1 -- 172.21.15.7:0/814931595 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c011830 con 0x7fb74c0ce670 2024-09-13T18:52:21.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb74affd700 1 -- 172.21.15.7:0/814931595 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73c011990 con 0x7fb74c0ce670 2024-09-13T18:52:21.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb74affd700 1 -- 172.21.15.7:0/814931595 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c011d40 con 0x7fb74c0ce670 2024-09-13T18:52:21.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb7525a3700 1 -- 172.21.15.7:0/814931595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb74c0ce670 msgr2=0x7fb74c0c9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb7525a3700 1 --2- 172.21.15.7:0/814931595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb74c0ce670 0x7fb74c0c9870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009910 tx=0x7fb73c009c20 comp rx=0 tx=0).stop 2024-09-13T18:52:21.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb7525a3700 1 -- 172.21.15.7:0/814931595 shutdown_connections 2024-09-13T18:52:21.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb7525a3700 1 --2- 172.21.15.7:0/814931595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb74c0ce670 0x7fb74c0c9870 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb7525a3700 1 --2- 172.21.15.7:0/814931595 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb74c0c5120 0x7fb74c0ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb7525a3700 1 -- 172.21.15.7:0/814931595 >> 172.21.15.7:0/814931595 conn(0x7fb74c0cbca0 msgr2=0x7fb74c0cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:21.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb7525a3700 1 -- 172.21.15.7:0/814931595 shutdown_connections 2024-09-13T18:52:21.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.766+0000 7fb7525a3700 1 -- 172.21.15.7:0/814931595 wait complete. 2024-09-13T18:52:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb7525a3700 1 Processor -- start 2024-09-13T18:52:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb7525a3700 1 -- start start 2024-09-13T18:52:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb7525a3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb74c0c5120 0x7fb74c1f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb7525a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb74c0ce670 0x7fb74c164e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb7525a3700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb74c1612c0 0x7fb74c165350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb7525a3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c165890 con 0x7fb74c0ce670 2024-09-13T18:52:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb7525a3700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c0c92e0 con 0x7fb74c0c5120 2024-09-13T18:52:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb7525a3700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74c1619a0 con 0x7fb74c1612c0 2024-09-13T18:52:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb74b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb74c0ce670 0x7fb74c164e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb74b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb74c0ce670 0x7fb74c164e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49198/0 (socket says 172.21.15.7:49198) 2024-09-13T18:52:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb74b7fe700 1 -- 172.21.15.7:0/2303836561 learned_addr learned my addr 172.21.15.7:0/2303836561 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb750b40700 1 --2- 172.21.15.7:0/2303836561 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb74c1612c0 0x7fb74c165350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb74bfff700 1 --2- 172.21.15.7:0/2303836561 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb74c0c5120 0x7fb74c1f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb74b7fe700 1 -- 172.21.15.7:0/2303836561 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb74c1612c0 msgr2=0x7fb74c165350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb74b7fe700 1 --2- 172.21.15.7:0/2303836561 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb74c1612c0 0x7fb74c165350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb74b7fe700 1 -- 172.21.15.7:0/2303836561 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb74c0c5120 msgr2=0x7fb74c1f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb74b7fe700 1 --2- 172.21.15.7:0/2303836561 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb74c0c5120 0x7fb74c1f6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb74b7fe700 1 -- 172.21.15.7:0/2303836561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73c009570 con 0x7fb74c0ce670 2024-09-13T18:52:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb74b7fe700 1 --2- 172.21.15.7:0/2303836561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb74c0ce670 0x7fb74c164e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb74400efc0 tx=0x7fb74400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:21.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb748ff9700 1 -- 172.21.15.7:0/2303836561 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74400cb50 con 0x7fb74c0ce670 2024-09-13T18:52:21.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb748ff9700 1 -- 172.21.15.7:0/2303836561 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74400f040 con 0x7fb74c0ce670 2024-09-13T18:52:21.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb7525a3700 1 -- 172.21.15.7:0/2303836561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74c161c80 con 0x7fb74c0ce670 2024-09-13T18:52:21.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb748ff9700 1 -- 172.21.15.7:0/2303836561 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb744014480 con 0x7fb74c0ce670 2024-09-13T18:52:21.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.770+0000 7fb7525a3700 1 -- 172.21.15.7:0/2303836561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74c1f6970 con 0x7fb74c0ce670 2024-09-13T18:52:21.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.774+0000 7fb7525a3700 1 -- 172.21.15.7:0/2303836561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb74c0cc890 con 0x7fb74c0ce670 2024-09-13T18:52:21.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.774+0000 7fb748ff9700 1 -- 172.21.15.7:0/2303836561 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb7440145e0 con 0x7fb74c0ce670 2024-09-13T18:52:21.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.774+0000 7fb748ff9700 1 --2- 172.21.15.7:0/2303836561 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb738066730 0x7fb738068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:21.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.774+0000 7fb748ff9700 1 -- 172.21.15.7:0/2303836561 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb744084b00 con 0x7fb74c0ce670 2024-09-13T18:52:21.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.778+0000 7fb74bfff700 1 --2- 172.21.15.7:0/2303836561 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb738066730 0x7fb738068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:21.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.778+0000 7fb74bfff700 1 --2- 172.21.15.7:0/2303836561 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb738066730 0x7fb738068bf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb740000f60 tx=0x7fb740009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:21.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.782+0000 7fb748ff9700 1 -- 172.21.15.7:0/2303836561 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb744051f00 con 0x7fb74c0ce670 2024-09-13T18:52:21.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.950+0000 7fb7525a3700 1 -- 172.21.15.7:0/2303836561 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb74c0c4c00 con 0x7fb738066730 2024-09-13T18:52:21.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.962+0000 7fb748ff9700 1 -- 172.21.15.7:0/2303836561 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249854 (secure 0 0 0) 0x7fb7400015e0 con 0x7fb738066730 2024-09-13T18:52:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:21.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.966+0000 7fb7525a3700 1 -- 172.21.15.7:0/2303836561 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb738066730 msgr2=0x7fb738068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.966+0000 7fb7525a3700 1 --2- 172.21.15.7:0/2303836561 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb738066730 0x7fb738068bf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb740000f60 tx=0x7fb740009040 comp rx=0 tx=0).stop 2024-09-13T18:52:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.966+0000 7fb7525a3700 1 -- 172.21.15.7:0/2303836561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb74c0ce670 msgr2=0x7fb74c164e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.966+0000 7fb7525a3700 1 --2- 172.21.15.7:0/2303836561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb74c0ce670 0x7fb74c164e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb74400efc0 tx=0x7fb74400c0c0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.966+0000 7fb7525a3700 1 -- 172.21.15.7:0/2303836561 shutdown_connections 2024-09-13T18:52:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.966+0000 7fb7525a3700 1 --2- 172.21.15.7:0/2303836561 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb738066730 0x7fb738068bf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.966+0000 7fb7525a3700 1 --2- 172.21.15.7:0/2303836561 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb74c1612c0 0x7fb74c165350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.966+0000 7fb7525a3700 1 --2- 172.21.15.7:0/2303836561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb74c0ce670 0x7fb74c164e10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.966+0000 7fb7525a3700 1 --2- 172.21.15.7:0/2303836561 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb74c0c5120 0x7fb74c1f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.966+0000 7fb7525a3700 1 -- 172.21.15.7:0/2303836561 >> 172.21.15.7:0/2303836561 conn(0x7fb74c0cbca0 msgr2=0x7fb74c0b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.970+0000 7fb7525a3700 1 -- 172.21.15.7:0/2303836561 shutdown_connections 2024-09-13T18:52:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:21.970+0000 7fb7525a3700 1 -- 172.21.15.7:0/2303836561 wait complete. 2024-09-13T18:52:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-13T18:52:21.989 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-13T18:52:20.211197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":145,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":235944,"kb_used_data":2512,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738776,"statfs":{"total":773094113280,"available":772852506624,"internally_reserved":0,"allocated":2572288,"data_stored":1598866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.584241"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'105","reported_seq":145,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T18:52:16.009416+0000","last_change":"2024-09-13T18:52:15.080080+0000","last_active":"2024-09-13T18:52:16.009416+0000","last_peered":"2024-09-13T18:52:16.009416+0000","last_clean":"2024-09-13T18:52:16.009416+0000","last_became_active":"2024-09-13T18:52:15.078100+0000","last_became_peered":"2024-09-13T18:52:15.078100+0000","last_unstale":"2024-09-13T18:52:16.009416+0000","last_undegraded":"2024-09-13T18:52:16.009416+0000","last_fullsized":"2024-09-13T18:52:16.009416+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_clean_scrub_stamp":"2024-09-13T18:52:13.196473+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:29:59.452757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":145,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:17.509490+0000","last_change":"2024-09-13T18:52:17.509490+0000","last_active":"2024-09-13T18:52:17.509490+0000","last_peered":"2024-09-13T18:52:17.509490+0000","last_clean":"2024-09-13T18:52:17.509490+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:17.509490+0000","last_undegraded":"2024-09-13T18:52:17.509490+0000","last_fullsized":"2024-09-13T18:52:17.509490+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:52:16.505621+0000","last_change":"2024-09-13T18:52:16.505621+0000","last_active":"2024-09-13T18:52:16.505621+0000","last_peered":"2024-09-13T18:52:16.505621+0000","last_clean":"2024-09-13T18:52:16.505621+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:52:16.505621+0000","last_undegraded":"2024-09-13T18:52:16.505621+0000","last_fullsized":"2024-09-13T18:52:16.505621+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":145,"num_write_kb":2154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":770048,"data_stored":643991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7689999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4990000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27188,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":163840,"data_stored":48196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3660000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":163840,"data_stored":48196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5620000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27200,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":176128,"data_stored":53623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31288,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":176128,"data_stored":53623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.837}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.893}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.421}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9350000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31292,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340548,"statfs":{"total":96636764160,"available":96604721152,"internally_reserved":0,"allocated":176128,"data_stored":53623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5049999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31288,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":176128,"data_stored":53623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.554}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":770048,"data_stored":643991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.357}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4730000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5049999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:52:21.995 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:52:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08f59e700 1 Processor -- start 2024-09-13T18:52:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08f59e700 1 -- start start 2024-09-13T18:52:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08f59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff0900b3ae0 0x7ff09006a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08f59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff09006aaf0 0x7ff09015e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08f59e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff09013dcb0 con 0x7ff09006bbf0 2024-09-13T18:52:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08f59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09015be60 con 0x7ff09006aaf0 2024-09-13T18:52:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08f59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09013ffd0 con 0x7ff0900b3ae0 2024-09-13T18:52:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff09006aaf0 0x7ff09015e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08e59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff09006bbf0 0x7ff0900b33d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35698/0 (socket says 172.21.15.7:35698) 2024-09-13T18:52:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08e59c700 1 -- 172.21.15.7:0/2628188444 learned_addr learned my addr 172.21.15.7:0/2628188444 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08dd9b700 1 --2- 172.21.15.7:0/2628188444 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff0900b3ae0 0x7ff09006a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08ed9d700 1 -- 172.21.15.7:0/2628188444 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff0900b3ae0 msgr2=0x7ff09006a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08ed9d700 1 --2- 172.21.15.7:0/2628188444 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff0900b3ae0 0x7ff09006a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08ed9d700 1 -- 172.21.15.7:0/2628188444 >> v1:172.21.15.7:6789/0 conn(0x7ff09006bbf0 legacy=0x7ff0900b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08d59a700 1 -- 172.21.15.7:0/2628188444 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078440730 0 0) 0x7ff09013dcb0 con 0x7ff09006bbf0 2024-09-13T18:52:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08ed9d700 1 -- 172.21.15.7:0/2628188444 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff090140220 con 0x7ff09006aaf0 2024-09-13T18:52:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08ed9d700 1 --2- 172.21.15.7:0/2628188444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff09006aaf0 0x7ff09015e440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff088009d90 tx=0x7ff08800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df31a90cedfaa5a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08d59a700 1 -- 172.21.15.7:0/2628188444 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08800e070 con 0x7ff09006aaf0 2024-09-13T18:52:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.106+0000 7ff08d59a700 1 -- 172.21.15.7:0/2628188444 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08800bb20 con 0x7ff09006aaf0 2024-09-13T18:52:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08d59a700 1 -- 172.21.15.7:0/2628188444 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088012430 con 0x7ff09006aaf0 2024-09-13T18:52:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08f59e700 1 -- 172.21.15.7:0/2628188444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff09006aaf0 msgr2=0x7ff09015e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08f59e700 1 --2- 172.21.15.7:0/2628188444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff09006aaf0 0x7ff09015e440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff088009d90 tx=0x7ff08800b330 comp rx=0 tx=0).stop 2024-09-13T18:52:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08f59e700 1 -- 172.21.15.7:0/2628188444 shutdown_connections 2024-09-13T18:52:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08f59e700 1 --2- 172.21.15.7:0/2628188444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff09006aaf0 0x7ff09015e440 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08f59e700 1 --2- 172.21.15.7:0/2628188444 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff0900b3ae0 0x7ff09006a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08f59e700 1 -- 172.21.15.7:0/2628188444 >> 172.21.15.7:0/2628188444 conn(0x7ff0900b0f00 msgr2=0x7ff0900b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08f59e700 1 -- 172.21.15.7:0/2628188444 shutdown_connections 2024-09-13T18:52:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08f59e700 1 -- 172.21.15.7:0/2628188444 wait complete. 2024-09-13T18:52:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08f59e700 1 Processor -- start 2024-09-13T18:52:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08f59e700 1 -- start start 2024-09-13T18:52:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08f59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff09006aaf0 0x7ff0900cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08f59e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff09006bbf0 0x7ff090152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0900b3ae0 0x7ff090141a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08f59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0901415d0 con 0x7ff0900b3ae0 2024-09-13T18:52:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08f59e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff090153340 con 0x7ff09006bbf0 2024-09-13T18:52:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08f59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff090140f80 con 0x7ff09006aaf0 2024-09-13T18:52:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0900b3ae0 0x7ff090141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0900b3ae0 0x7ff090141a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49210/0 (socket says 172.21.15.7:49210) 2024-09-13T18:52:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08ed9d700 1 -- 172.21.15.7:0/4285221491 learned_addr learned my addr 172.21.15.7:0/4285221491 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08e59c700 1 --2- 172.21.15.7:0/4285221491 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff09006aaf0 0x7ff0900cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.110+0000 7ff08dd9b700 1 --2- 172.21.15.7:0/4285221491 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff09006bbf0 0x7ff090152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.114+0000 7ff08ed9d700 1 -- 172.21.15.7:0/4285221491 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff09006aaf0 msgr2=0x7ff0900cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.114+0000 7ff08ed9d700 1 --2- 172.21.15.7:0/4285221491 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff09006aaf0 0x7ff0900cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.114+0000 7ff08ed9d700 1 -- 172.21.15.7:0/4285221491 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff09006bbf0 msgr2=0x7ff090152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.114+0000 7ff08ed9d700 1 --2- 172.21.15.7:0/4285221491 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff09006bbf0 0x7ff090152960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.114+0000 7ff08ed9d700 1 -- 172.21.15.7:0/4285221491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff088009a70 con 0x7ff0900b3ae0 2024-09-13T18:52:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.114+0000 7ff08ed9d700 1 --2- 172.21.15.7:0/4285221491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0900b3ae0 0x7ff090141a60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff09006b390 tx=0x7ff088004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.114+0000 7ff086ffd700 1 -- 172.21.15.7:0/4285221491 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088012430 con 0x7ff0900b3ae0 2024-09-13T18:52:22.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.114+0000 7ff086ffd700 1 -- 172.21.15.7:0/4285221491 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff088020070 con 0x7ff0900b3ae0 2024-09-13T18:52:22.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.114+0000 7ff086ffd700 1 -- 172.21.15.7:0/4285221491 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088004280 con 0x7ff0900b3ae0 2024-09-13T18:52:22.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.114+0000 7ff08f59e700 1 -- 172.21.15.7:0/4285221491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff090141260 con 0x7ff0900b3ae0 2024-09-13T18:52:22.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.114+0000 7ff08f59e700 1 -- 172.21.15.7:0/4285221491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff090153620 con 0x7ff0900b3ae0 2024-09-13T18:52:22.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.114+0000 7ff08f59e700 1 -- 172.21.15.7:0/4285221491 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff090141780 con 0x7ff0900b3ae0 2024-09-13T18:52:22.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.114+0000 7ff086ffd700 1 -- 172.21.15.7:0/4285221491 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff0880043e0 con 0x7ff0900b3ae0 2024-09-13T18:52:22.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.114+0000 7ff086ffd700 1 --2- 172.21.15.7:0/4285221491 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff074066730 0x7ff074068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.114+0000 7ff086ffd700 1 -- 172.21.15.7:0/4285221491 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff088022070 con 0x7ff0900b3ae0 2024-09-13T18:52:22.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.118+0000 7ff08e59c700 1 --2- 172.21.15.7:0/4285221491 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff074066730 0x7ff074068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.118+0000 7ff08e59c700 1 --2- 172.21.15.7:0/4285221491 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff074066730 0x7ff074068bf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff080000f60 tx=0x7ff080007e00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:22.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.122+0000 7ff086ffd700 1 -- 172.21.15.7:0/4285221491 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff090141780 con 0x7ff0900b3ae0 2024-09-13T18:52:22.212 INFO:tasks.ceph.mds.b.smithi171.stderr:2024-09-13T18:52:22.210+0000 7f9587f7c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T18:52:22.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.286+0000 7ff08f59e700 1 -- 172.21.15.7:0/4285221491 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff090140ac0 con 0x7ff074066730 2024-09-13T18:52:22.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.306+0000 7ff086ffd700 1 -- 172.21.15.7:0/4285221491 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264606 (secure 0 0 0) 0x7ff0800013f0 con 0x7ff074066730 2024-09-13T18:52:22.313 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.310+0000 7ff08f59e700 1 -- 172.21.15.7:0/4285221491 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff074066730 msgr2=0x7ff074068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.310+0000 7ff08f59e700 1 --2- 172.21.15.7:0/4285221491 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff074066730 0x7ff074068bf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff080000f60 tx=0x7ff080007e00 comp rx=0 tx=0).stop 2024-09-13T18:52:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.310+0000 7ff08f59e700 1 -- 172.21.15.7:0/4285221491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0900b3ae0 msgr2=0x7ff090141a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.310+0000 7ff08f59e700 1 --2- 172.21.15.7:0/4285221491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0900b3ae0 0x7ff090141a60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff09006b390 tx=0x7ff088004af0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.310+0000 7ff08f59e700 1 -- 172.21.15.7:0/4285221491 shutdown_connections 2024-09-13T18:52:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.310+0000 7ff08f59e700 1 --2- 172.21.15.7:0/4285221491 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff074066730 0x7ff074068bf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.310+0000 7ff08f59e700 1 --2- 172.21.15.7:0/4285221491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff0900b3ae0 0x7ff090141a60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.310+0000 7ff08f59e700 1 --2- 172.21.15.7:0/4285221491 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff09006bbf0 0x7ff090152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.310+0000 7ff08f59e700 1 --2- 172.21.15.7:0/4285221491 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff09006aaf0 0x7ff0900cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.310+0000 7ff08f59e700 1 -- 172.21.15.7:0/4285221491 >> 172.21.15.7:0/4285221491 conn(0x7ff0900b0f00 msgr2=0x7ff09015a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.310+0000 7ff08f59e700 1 -- 172.21.15.7:0/4285221491 shutdown_connections 2024-09-13T18:52:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.310+0000 7ff08f59e700 1 -- 172.21.15.7:0/4285221491 wait complete. 2024-09-13T18:52:22.318 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-13T18:52:22.332 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-13T18:52:22.211653+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235740,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738980,"statfs":{"total":773094113280,"available":772852715520,"internally_reserved":0,"allocated":3055616,"data_stored":1815780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.584696"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200870+0000","last_change":"2024-09-13T18:52:17.941085+0000","last_active":"2024-09-13T18:52:20.200870+0000","last_peered":"2024-09-13T18:52:20.200870+0000","last_clean":"2024-09-13T18:52:20.200870+0000","last_became_active":"2024-09-13T18:52:17.940741+0000","last_became_peered":"2024-09-13T18:52:17.940741+0000","last_unstale":"2024-09-13T18:52:20.200870+0000","last_undegraded":"2024-09-13T18:52:20.200870+0000","last_fullsized":"2024-09-13T18:52:20.200870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:50.609433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195285+0000","last_change":"2024-09-13T18:52:17.942350+0000","last_active":"2024-09-13T18:52:20.195285+0000","last_peered":"2024-09-13T18:52:20.195285+0000","last_clean":"2024-09-13T18:52:20.195285+0000","last_became_active":"2024-09-13T18:52:17.942062+0000","last_became_peered":"2024-09-13T18:52:17.942062+0000","last_unstale":"2024-09-13T18:52:20.195285+0000","last_undegraded":"2024-09-13T18:52:20.195285+0000","last_fullsized":"2024-09-13T18:52:20.195285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:45.768218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195540+0000","last_change":"2024-09-13T18:52:17.953055+0000","last_active":"2024-09-13T18:52:20.195540+0000","last_peered":"2024-09-13T18:52:20.195540+0000","last_clean":"2024-09-13T18:52:20.195540+0000","last_became_active":"2024-09-13T18:52:17.952806+0000","last_became_peered":"2024-09-13T18:52:17.952806+0000","last_unstale":"2024-09-13T18:52:20.195540+0000","last_undegraded":"2024-09-13T18:52:20.195540+0000","last_fullsized":"2024-09-13T18:52:20.195540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:18.149491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198959+0000","last_change":"2024-09-13T18:52:17.943472+0000","last_active":"2024-09-13T18:52:20.198959+0000","last_peered":"2024-09-13T18:52:20.198959+0000","last_clean":"2024-09-13T18:52:20.198959+0000","last_became_active":"2024-09-13T18:52:17.943070+0000","last_became_peered":"2024-09-13T18:52:17.943070+0000","last_unstale":"2024-09-13T18:52:20.198959+0000","last_undegraded":"2024-09-13T18:52:20.198959+0000","last_fullsized":"2024-09-13T18:52:20.198959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.990929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198942+0000","last_change":"2024-09-13T18:52:17.957081+0000","last_active":"2024-09-13T18:52:20.198942+0000","last_peered":"2024-09-13T18:52:20.198942+0000","last_clean":"2024-09-13T18:52:20.198942+0000","last_became_active":"2024-09-13T18:52:17.956787+0000","last_became_peered":"2024-09-13T18:52:17.956787+0000","last_unstale":"2024-09-13T18:52:20.198942+0000","last_undegraded":"2024-09-13T18:52:20.198942+0000","last_fullsized":"2024-09-13T18:52:20.198942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:31.026983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198843+0000","last_change":"2024-09-13T18:52:17.944754+0000","last_active":"2024-09-13T18:52:20.198843+0000","last_peered":"2024-09-13T18:52:20.198843+0000","last_clean":"2024-09-13T18:52:20.198843+0000","last_became_active":"2024-09-13T18:52:17.944495+0000","last_became_peered":"2024-09-13T18:52:17.944495+0000","last_unstale":"2024-09-13T18:52:20.198843+0000","last_undegraded":"2024-09-13T18:52:20.198843+0000","last_fullsized":"2024-09-13T18:52:20.198843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:40:09.595666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198289+0000","last_change":"2024-09-13T18:52:17.945805+0000","last_active":"2024-09-13T18:52:20.198289+0000","last_peered":"2024-09-13T18:52:20.198289+0000","last_clean":"2024-09-13T18:52:20.198289+0000","last_became_active":"2024-09-13T18:52:17.945364+0000","last_became_peered":"2024-09-13T18:52:17.945364+0000","last_unstale":"2024-09-13T18:52:20.198289+0000","last_undegraded":"2024-09-13T18:52:20.198289+0000","last_fullsized":"2024-09-13T18:52:20.198289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:43.043485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.923839+0000","last_change":"2024-09-13T18:52:17.939184+0000","last_active":"2024-09-13T18:52:18.923839+0000","last_peered":"2024-09-13T18:52:18.923839+0000","last_clean":"2024-09-13T18:52:18.923839+0000","last_became_active":"2024-09-13T18:52:17.938855+0000","last_became_peered":"2024-09-13T18:52:17.938855+0000","last_unstale":"2024-09-13T18:52:18.923839+0000","last_undegraded":"2024-09-13T18:52:18.923839+0000","last_fullsized":"2024-09-13T18:52:18.923839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:45.768218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200942+0000","last_change":"2024-09-13T18:52:17.940244+0000","last_active":"2024-09-13T18:52:20.200942+0000","last_peered":"2024-09-13T18:52:20.200942+0000","last_clean":"2024-09-13T18:52:20.200942+0000","last_became_active":"2024-09-13T18:52:17.939977+0000","last_became_peered":"2024-09-13T18:52:17.939977+0000","last_unstale":"2024-09-13T18:52:20.200942+0000","last_undegraded":"2024-09-13T18:52:20.200942+0000","last_fullsized":"2024-09-13T18:52:20.200942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:21.405599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.928272+0000","last_change":"2024-09-13T18:52:17.940027+0000","last_active":"2024-09-13T18:52:18.928272+0000","last_peered":"2024-09-13T18:52:18.928272+0000","last_clean":"2024-09-13T18:52:18.928272+0000","last_became_active":"2024-09-13T18:52:17.939743+0000","last_became_peered":"2024-09-13T18:52:17.939743+0000","last_unstale":"2024-09-13T18:52:18.928272+0000","last_undegraded":"2024-09-13T18:52:18.928272+0000","last_fullsized":"2024-09-13T18:52:18.928272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:26.228888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927956+0000","last_change":"2024-09-13T18:52:17.940006+0000","last_active":"2024-09-13T18:52:18.927956+0000","last_peered":"2024-09-13T18:52:18.927956+0000","last_clean":"2024-09-13T18:52:18.927956+0000","last_became_active":"2024-09-13T18:52:17.939474+0000","last_became_peered":"2024-09-13T18:52:17.939474+0000","last_unstale":"2024-09-13T18:52:18.927956+0000","last_undegraded":"2024-09-13T18:52:18.927956+0000","last_fullsized":"2024-09-13T18:52:18.927956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:24.025045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198153+0000","last_change":"2024-09-13T18:52:17.942513+0000","last_active":"2024-09-13T18:52:20.198153+0000","last_peered":"2024-09-13T18:52:20.198153+0000","last_clean":"2024-09-13T18:52:20.198153+0000","last_became_active":"2024-09-13T18:52:17.940486+0000","last_became_peered":"2024-09-13T18:52:17.940486+0000","last_unstale":"2024-09-13T18:52:20.198153+0000","last_undegraded":"2024-09-13T18:52:20.198153+0000","last_fullsized":"2024-09-13T18:52:20.198153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:08.839071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924421+0000","last_change":"2024-09-13T18:52:17.935439+0000","last_active":"2024-09-13T18:52:18.924421+0000","last_peered":"2024-09-13T18:52:18.924421+0000","last_clean":"2024-09-13T18:52:18.924421+0000","last_became_active":"2024-09-13T18:52:17.935114+0000","last_became_peered":"2024-09-13T18:52:17.935114+0000","last_unstale":"2024-09-13T18:52:18.924421+0000","last_undegraded":"2024-09-13T18:52:18.924421+0000","last_fullsized":"2024-09-13T18:52:18.924421+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:05:38.704891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195436+0000","last_change":"2024-09-13T18:52:17.942475+0000","last_active":"2024-09-13T18:52:20.195436+0000","last_peered":"2024-09-13T18:52:20.195436+0000","last_clean":"2024-09-13T18:52:20.195436+0000","last_became_active":"2024-09-13T18:52:17.942233+0000","last_became_peered":"2024-09-13T18:52:17.942233+0000","last_unstale":"2024-09-13T18:52:20.195436+0000","last_undegraded":"2024-09-13T18:52:20.195436+0000","last_fullsized":"2024-09-13T18:52:20.195436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:24.025045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.194709+0000","last_change":"2024-09-13T18:52:17.926146+0000","last_active":"2024-09-13T18:52:20.194709+0000","last_peered":"2024-09-13T18:52:20.194709+0000","last_clean":"2024-09-13T18:52:20.194709+0000","last_became_active":"2024-09-13T18:52:17.925837+0000","last_became_peered":"2024-09-13T18:52:17.925837+0000","last_unstale":"2024-09-13T18:52:20.194709+0000","last_undegraded":"2024-09-13T18:52:20.194709+0000","last_fullsized":"2024-09-13T18:52:20.194709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:30.871944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196715+0000","last_change":"2024-09-13T18:52:17.945764+0000","last_active":"2024-09-13T18:52:20.196715+0000","last_peered":"2024-09-13T18:52:20.196715+0000","last_clean":"2024-09-13T18:52:20.196715+0000","last_became_active":"2024-09-13T18:52:17.944484+0000","last_became_peered":"2024-09-13T18:52:17.944484+0000","last_unstale":"2024-09-13T18:52:20.196715+0000","last_undegraded":"2024-09-13T18:52:20.196715+0000","last_fullsized":"2024-09-13T18:52:20.196715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:45.768218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199740+0000","last_change":"2024-09-13T18:52:17.943317+0000","last_active":"2024-09-13T18:52:20.199740+0000","last_peered":"2024-09-13T18:52:20.199740+0000","last_clean":"2024-09-13T18:52:20.199740+0000","last_became_active":"2024-09-13T18:52:17.943015+0000","last_became_peered":"2024-09-13T18:52:17.943015+0000","last_unstale":"2024-09-13T18:52:20.199740+0000","last_undegraded":"2024-09-13T18:52:20.199740+0000","last_fullsized":"2024-09-13T18:52:20.199740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:50.609433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925962+0000","last_change":"2024-09-13T18:52:17.956474+0000","last_active":"2024-09-13T18:52:18.925962+0000","last_peered":"2024-09-13T18:52:18.925962+0000","last_clean":"2024-09-13T18:52:18.925962+0000","last_became_active":"2024-09-13T18:52:17.956200+0000","last_became_peered":"2024-09-13T18:52:17.956200+0000","last_unstale":"2024-09-13T18:52:18.925962+0000","last_undegraded":"2024-09-13T18:52:18.925962+0000","last_fullsized":"2024-09-13T18:52:18.925962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:17:56.476262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200865+0000","last_change":"2024-09-13T18:52:17.955849+0000","last_active":"2024-09-13T18:52:20.200865+0000","last_peered":"2024-09-13T18:52:20.200865+0000","last_clean":"2024-09-13T18:52:20.200865+0000","last_became_active":"2024-09-13T18:52:17.955606+0000","last_became_peered":"2024-09-13T18:52:17.955606+0000","last_unstale":"2024-09-13T18:52:20.200865+0000","last_undegraded":"2024-09-13T18:52:20.200865+0000","last_fullsized":"2024-09-13T18:52:20.200865+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:45.768218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927684+0000","last_change":"2024-09-13T18:52:17.947271+0000","last_active":"2024-09-13T18:52:18.927684+0000","last_peered":"2024-09-13T18:52:18.927684+0000","last_clean":"2024-09-13T18:52:18.927684+0000","last_became_active":"2024-09-13T18:52:17.947018+0000","last_became_peered":"2024-09-13T18:52:17.947018+0000","last_unstale":"2024-09-13T18:52:18.927684+0000","last_undegraded":"2024-09-13T18:52:18.927684+0000","last_fullsized":"2024-09-13T18:52:18.927684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:45.768218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195004+0000","last_change":"2024-09-13T18:52:17.949499+0000","last_active":"2024-09-13T18:52:20.195004+0000","last_peered":"2024-09-13T18:52:20.195004+0000","last_clean":"2024-09-13T18:52:20.195004+0000","last_became_active":"2024-09-13T18:52:17.949233+0000","last_became_peered":"2024-09-13T18:52:17.949233+0000","last_unstale":"2024-09-13T18:52:20.195004+0000","last_undegraded":"2024-09-13T18:52:20.195004+0000","last_fullsized":"2024-09-13T18:52:20.195004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:05:38.704891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197193+0000","last_change":"2024-09-13T18:52:17.941847+0000","last_active":"2024-09-13T18:52:20.197193+0000","last_peered":"2024-09-13T18:52:20.197193+0000","last_clean":"2024-09-13T18:52:20.197193+0000","last_became_active":"2024-09-13T18:52:17.941228+0000","last_became_peered":"2024-09-13T18:52:17.941228+0000","last_unstale":"2024-09-13T18:52:20.197193+0000","last_undegraded":"2024-09-13T18:52:20.197193+0000","last_fullsized":"2024-09-13T18:52:20.197193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:41.161193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924015+0000","last_change":"2024-09-13T18:52:17.924625+0000","last_active":"2024-09-13T18:52:18.924015+0000","last_peered":"2024-09-13T18:52:18.924015+0000","last_clean":"2024-09-13T18:52:18.924015+0000","last_became_active":"2024-09-13T18:52:17.923776+0000","last_became_peered":"2024-09-13T18:52:17.923776+0000","last_unstale":"2024-09-13T18:52:18.924015+0000","last_undegraded":"2024-09-13T18:52:18.924015+0000","last_fullsized":"2024-09-13T18:52:18.924015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:36.215528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196911+0000","last_change":"2024-09-13T18:52:17.938687+0000","last_active":"2024-09-13T18:52:20.196911+0000","last_peered":"2024-09-13T18:52:20.196911+0000","last_clean":"2024-09-13T18:52:20.196911+0000","last_became_active":"2024-09-13T18:52:17.938426+0000","last_became_peered":"2024-09-13T18:52:17.938426+0000","last_unstale":"2024-09-13T18:52:20.196911+0000","last_undegraded":"2024-09-13T18:52:20.196911+0000","last_fullsized":"2024-09-13T18:52:20.196911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:53.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924463+0000","last_change":"2024-09-13T18:52:17.924256+0000","last_active":"2024-09-13T18:52:18.924463+0000","last_peered":"2024-09-13T18:52:18.924463+0000","last_clean":"2024-09-13T18:52:18.924463+0000","last_became_active":"2024-09-13T18:52:17.923756+0000","last_became_peered":"2024-09-13T18:52:17.923756+0000","last_unstale":"2024-09-13T18:52:18.924463+0000","last_undegraded":"2024-09-13T18:52:18.924463+0000","last_fullsized":"2024-09-13T18:52:18.924463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:26.228888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.928375+0000","last_change":"2024-09-13T18:52:17.931734+0000","last_active":"2024-09-13T18:52:18.928375+0000","last_peered":"2024-09-13T18:52:18.928375+0000","last_clean":"2024-09-13T18:52:18.928375+0000","last_became_active":"2024-09-13T18:52:17.931034+0000","last_became_peered":"2024-09-13T18:52:17.931034+0000","last_unstale":"2024-09-13T18:52:18.928375+0000","last_undegraded":"2024-09-13T18:52:18.928375+0000","last_fullsized":"2024-09-13T18:52:18.928375+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:31.026983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.194568+0000","last_change":"2024-09-13T18:52:17.927130+0000","last_active":"2024-09-13T18:52:20.194568+0000","last_peered":"2024-09-13T18:52:20.194568+0000","last_clean":"2024-09-13T18:52:20.194568+0000","last_became_active":"2024-09-13T18:52:17.926099+0000","last_became_peered":"2024-09-13T18:52:17.926099+0000","last_unstale":"2024-09-13T18:52:20.194568+0000","last_undegraded":"2024-09-13T18:52:20.194568+0000","last_fullsized":"2024-09-13T18:52:20.194568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:05:38.704891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924857+0000","last_change":"2024-09-13T18:52:17.924220+0000","last_active":"2024-09-13T18:52:18.924857+0000","last_peered":"2024-09-13T18:52:18.924857+0000","last_clean":"2024-09-13T18:52:18.924857+0000","last_became_active":"2024-09-13T18:52:17.923830+0000","last_became_peered":"2024-09-13T18:52:17.923830+0000","last_unstale":"2024-09-13T18:52:18.924857+0000","last_undegraded":"2024-09-13T18:52:18.924857+0000","last_fullsized":"2024-09-13T18:52:18.924857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:24.025045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195245+0000","last_change":"2024-09-13T18:52:17.942480+0000","last_active":"2024-09-13T18:52:20.195245+0000","last_peered":"2024-09-13T18:52:20.195245+0000","last_clean":"2024-09-13T18:52:20.195245+0000","last_became_active":"2024-09-13T18:52:17.942156+0000","last_became_peered":"2024-09-13T18:52:17.942156+0000","last_unstale":"2024-09-13T18:52:20.195245+0000","last_undegraded":"2024-09-13T18:52:20.195245+0000","last_fullsized":"2024-09-13T18:52:20.195245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:30.871944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197425+0000","last_change":"2024-09-13T18:52:17.943608+0000","last_active":"2024-09-13T18:52:20.197425+0000","last_peered":"2024-09-13T18:52:20.197425+0000","last_clean":"2024-09-13T18:52:20.197425+0000","last_became_active":"2024-09-13T18:52:17.943314+0000","last_became_peered":"2024-09-13T18:52:17.943314+0000","last_unstale":"2024-09-13T18:52:20.197425+0000","last_undegraded":"2024-09-13T18:52:20.197425+0000","last_fullsized":"2024-09-13T18:52:20.197425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:43.043485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925985+0000","last_change":"2024-09-13T18:52:17.956342+0000","last_active":"2024-09-13T18:52:18.925985+0000","last_peered":"2024-09-13T18:52:18.925985+0000","last_clean":"2024-09-13T18:52:18.925985+0000","last_became_active":"2024-09-13T18:52:17.956066+0000","last_became_peered":"2024-09-13T18:52:17.956066+0000","last_unstale":"2024-09-13T18:52:18.925985+0000","last_undegraded":"2024-09-13T18:52:18.925985+0000","last_fullsized":"2024-09-13T18:52:18.925985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:24:03.326121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927056+0000","last_change":"2024-09-13T18:52:17.944931+0000","last_active":"2024-09-13T18:52:18.927056+0000","last_peered":"2024-09-13T18:52:18.927056+0000","last_clean":"2024-09-13T18:52:18.927056+0000","last_became_active":"2024-09-13T18:52:17.944522+0000","last_became_peered":"2024-09-13T18:52:17.944522+0000","last_unstale":"2024-09-13T18:52:18.927056+0000","last_undegraded":"2024-09-13T18:52:18.927056+0000","last_fullsized":"2024-09-13T18:52:18.927056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:36.215528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197266+0000","last_change":"2024-09-13T18:52:17.924921+0000","last_active":"2024-09-13T18:52:20.197266+0000","last_peered":"2024-09-13T18:52:20.197266+0000","last_clean":"2024-09-13T18:52:20.197266+0000","last_became_active":"2024-09-13T18:52:17.924708+0000","last_became_peered":"2024-09-13T18:52:17.924708+0000","last_unstale":"2024-09-13T18:52:20.197266+0000","last_undegraded":"2024-09-13T18:52:20.197266+0000","last_fullsized":"2024-09-13T18:52:20.197266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:25.786229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925450+0000","last_change":"2024-09-13T18:52:17.538391+0000","last_active":"2024-09-13T18:52:18.925450+0000","last_peered":"2024-09-13T18:52:18.925450+0000","last_clean":"2024-09-13T18:52:18.925450+0000","last_became_active":"2024-09-13T18:52:17.538114+0000","last_became_peered":"2024-09-13T18:52:17.538114+0000","last_unstale":"2024-09-13T18:52:18.925450+0000","last_undegraded":"2024-09-13T18:52:18.925450+0000","last_fullsized":"2024-09-13T18:52:18.925450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:04.695688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195477+0000","last_change":"2024-09-13T18:52:17.946924+0000","last_active":"2024-09-13T18:52:20.195477+0000","last_peered":"2024-09-13T18:52:20.195477+0000","last_clean":"2024-09-13T18:52:20.195477+0000","last_became_active":"2024-09-13T18:52:17.946599+0000","last_became_peered":"2024-09-13T18:52:17.946599+0000","last_unstale":"2024-09-13T18:52:20.195477+0000","last_undegraded":"2024-09-13T18:52:20.195477+0000","last_fullsized":"2024-09-13T18:52:20.195477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:26.228888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927670+0000","last_change":"2024-09-13T18:52:17.535479+0000","last_active":"2024-09-13T18:52:18.927670+0000","last_peered":"2024-09-13T18:52:18.927670+0000","last_clean":"2024-09-13T18:52:18.927670+0000","last_became_active":"2024-09-13T18:52:17.535185+0000","last_became_peered":"2024-09-13T18:52:17.535185+0000","last_unstale":"2024-09-13T18:52:18.927670+0000","last_undegraded":"2024-09-13T18:52:18.927670+0000","last_fullsized":"2024-09-13T18:52:18.927670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:47.942952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924218+0000","last_change":"2024-09-13T18:52:17.531093+0000","last_active":"2024-09-13T18:52:18.924218+0000","last_peered":"2024-09-13T18:52:18.924218+0000","last_clean":"2024-09-13T18:52:18.924218+0000","last_became_active":"2024-09-13T18:52:17.530499+0000","last_became_peered":"2024-09-13T18:52:17.530499+0000","last_unstale":"2024-09-13T18:52:18.924218+0000","last_undegraded":"2024-09-13T18:52:18.924218+0000","last_fullsized":"2024-09-13T18:52:18.924218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:33:25.953860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.923892+0000","last_change":"2024-09-13T18:52:17.940517+0000","last_active":"2024-09-13T18:52:18.923892+0000","last_peered":"2024-09-13T18:52:18.923892+0000","last_clean":"2024-09-13T18:52:18.923892+0000","last_became_active":"2024-09-13T18:52:17.940250+0000","last_became_peered":"2024-09-13T18:52:17.940250+0000","last_unstale":"2024-09-13T18:52:18.923892+0000","last_undegraded":"2024-09-13T18:52:18.923892+0000","last_fullsized":"2024-09-13T18:52:18.923892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.990929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196937+0000","last_change":"2024-09-13T18:52:17.539508+0000","last_active":"2024-09-13T18:52:20.196937+0000","last_peered":"2024-09-13T18:52:20.196937+0000","last_clean":"2024-09-13T18:52:20.196937+0000","last_became_active":"2024-09-13T18:52:17.539182+0000","last_became_peered":"2024-09-13T18:52:17.539182+0000","last_unstale":"2024-09-13T18:52:20.196937+0000","last_undegraded":"2024-09-13T18:52:20.196937+0000","last_fullsized":"2024-09-13T18:52:20.196937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:49.151192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.194851+0000","last_change":"2024-09-13T18:52:17.926798+0000","last_active":"2024-09-13T18:52:20.194851+0000","last_peered":"2024-09-13T18:52:20.194851+0000","last_clean":"2024-09-13T18:52:20.194851+0000","last_became_active":"2024-09-13T18:52:17.926182+0000","last_became_peered":"2024-09-13T18:52:17.926182+0000","last_unstale":"2024-09-13T18:52:20.194851+0000","last_undegraded":"2024-09-13T18:52:20.194851+0000","last_fullsized":"2024-09-13T18:52:20.194851+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:36.215528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.928162+0000","last_change":"2024-09-13T18:52:17.539633+0000","last_active":"2024-09-13T18:52:18.928162+0000","last_peered":"2024-09-13T18:52:18.928162+0000","last_clean":"2024-09-13T18:52:18.928162+0000","last_became_active":"2024-09-13T18:52:17.539340+0000","last_became_peered":"2024-09-13T18:52:17.539340+0000","last_unstale":"2024-09-13T18:52:18.928162+0000","last_undegraded":"2024-09-13T18:52:18.928162+0000","last_fullsized":"2024-09-13T18:52:18.928162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:06.694707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199408+0000","last_change":"2024-09-13T18:52:17.941631+0000","last_active":"2024-09-13T18:52:20.199408+0000","last_peered":"2024-09-13T18:52:20.199408+0000","last_clean":"2024-09-13T18:52:20.199408+0000","last_became_active":"2024-09-13T18:52:17.940875+0000","last_became_peered":"2024-09-13T18:52:17.940875+0000","last_unstale":"2024-09-13T18:52:20.199408+0000","last_undegraded":"2024-09-13T18:52:20.199408+0000","last_fullsized":"2024-09-13T18:52:20.199408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:42:22.626590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199507+0000","last_change":"2024-09-13T18:52:17.532300+0000","last_active":"2024-09-13T18:52:20.199507+0000","last_peered":"2024-09-13T18:52:20.199507+0000","last_clean":"2024-09-13T18:52:20.199507+0000","last_became_active":"2024-09-13T18:52:17.532019+0000","last_became_peered":"2024-09-13T18:52:17.532019+0000","last_unstale":"2024-09-13T18:52:20.199507+0000","last_undegraded":"2024-09-13T18:52:20.199507+0000","last_fullsized":"2024-09-13T18:52:20.199507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:05:28.094741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196062+0000","last_change":"2024-09-13T18:52:17.945629+0000","last_active":"2024-09-13T18:52:20.196062+0000","last_peered":"2024-09-13T18:52:20.196062+0000","last_clean":"2024-09-13T18:52:20.196062+0000","last_became_active":"2024-09-13T18:52:17.945372+0000","last_became_peered":"2024-09-13T18:52:17.945372+0000","last_unstale":"2024-09-13T18:52:20.196062+0000","last_undegraded":"2024-09-13T18:52:20.196062+0000","last_fullsized":"2024-09-13T18:52:20.196062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:36.215528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.194942+0000","last_change":"2024-09-13T18:52:17.538613+0000","last_active":"2024-09-13T18:52:20.194942+0000","last_peered":"2024-09-13T18:52:20.194942+0000","last_clean":"2024-09-13T18:52:20.194942+0000","last_became_active":"2024-09-13T18:52:17.538372+0000","last_became_peered":"2024-09-13T18:52:17.538372+0000","last_unstale":"2024-09-13T18:52:20.194942+0000","last_undegraded":"2024-09-13T18:52:20.194942+0000","last_fullsized":"2024-09-13T18:52:20.194942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:05:28.094741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198119+0000","last_change":"2024-09-13T18:52:17.941036+0000","last_active":"2024-09-13T18:52:20.198119+0000","last_peered":"2024-09-13T18:52:20.198119+0000","last_clean":"2024-09-13T18:52:20.198119+0000","last_became_active":"2024-09-13T18:52:17.940741+0000","last_became_peered":"2024-09-13T18:52:17.940741+0000","last_unstale":"2024-09-13T18:52:20.198119+0000","last_undegraded":"2024-09-13T18:52:20.198119+0000","last_fullsized":"2024-09-13T18:52:20.198119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:19:17.191755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927753+0000","last_change":"2024-09-13T18:52:17.536588+0000","last_active":"2024-09-13T18:52:18.927753+0000","last_peered":"2024-09-13T18:52:18.927753+0000","last_clean":"2024-09-13T18:52:18.927753+0000","last_became_active":"2024-09-13T18:52:17.536319+0000","last_became_peered":"2024-09-13T18:52:17.536319+0000","last_unstale":"2024-09-13T18:52:18.927753+0000","last_undegraded":"2024-09-13T18:52:18.927753+0000","last_fullsized":"2024-09-13T18:52:18.927753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:14:15.613948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199367+0000","last_change":"2024-09-13T18:52:17.935349+0000","last_active":"2024-09-13T18:52:20.199367+0000","last_peered":"2024-09-13T18:52:20.199367+0000","last_clean":"2024-09-13T18:52:20.199367+0000","last_became_active":"2024-09-13T18:52:17.935064+0000","last_became_peered":"2024-09-13T18:52:17.935064+0000","last_unstale":"2024-09-13T18:52:20.199367+0000","last_undegraded":"2024-09-13T18:52:20.199367+0000","last_fullsized":"2024-09-13T18:52:20.199367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:18.183823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195705+0000","last_change":"2024-09-13T18:52:17.538911+0000","last_active":"2024-09-13T18:52:20.195705+0000","last_peered":"2024-09-13T18:52:20.195705+0000","last_clean":"2024-09-13T18:52:20.195705+0000","last_became_active":"2024-09-13T18:52:17.538512+0000","last_became_peered":"2024-09-13T18:52:17.538512+0000","last_unstale":"2024-09-13T18:52:20.195705+0000","last_undegraded":"2024-09-13T18:52:20.195705+0000","last_fullsized":"2024-09-13T18:52:20.195705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:03:19.727396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195688+0000","last_change":"2024-09-13T18:52:17.949196+0000","last_active":"2024-09-13T18:52:20.195688+0000","last_peered":"2024-09-13T18:52:20.195688+0000","last_clean":"2024-09-13T18:52:20.195688+0000","last_became_active":"2024-09-13T18:52:17.948936+0000","last_became_peered":"2024-09-13T18:52:17.948936+0000","last_unstale":"2024-09-13T18:52:20.195688+0000","last_undegraded":"2024-09-13T18:52:20.195688+0000","last_fullsized":"2024-09-13T18:52:20.195688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:43:49.752608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197616+0000","last_change":"2024-09-13T18:52:17.534190+0000","last_active":"2024-09-13T18:52:20.197616+0000","last_peered":"2024-09-13T18:52:20.197616+0000","last_clean":"2024-09-13T18:52:20.197616+0000","last_became_active":"2024-09-13T18:52:17.532898+0000","last_became_peered":"2024-09-13T18:52:17.532898+0000","last_unstale":"2024-09-13T18:52:20.197616+0000","last_undegraded":"2024-09-13T18:52:20.197616+0000","last_fullsized":"2024-09-13T18:52:20.197616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:14:15.613948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200129+0000","last_change":"2024-09-13T18:52:17.947584+0000","last_active":"2024-09-13T18:52:20.200129+0000","last_peered":"2024-09-13T18:52:20.200129+0000","last_clean":"2024-09-13T18:52:20.200129+0000","last_became_active":"2024-09-13T18:52:17.947333+0000","last_became_peered":"2024-09-13T18:52:17.947333+0000","last_unstale":"2024-09-13T18:52:20.200129+0000","last_undegraded":"2024-09-13T18:52:20.200129+0000","last_fullsized":"2024-09-13T18:52:20.200129+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:53.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927484+0000","last_change":"2024-09-13T18:52:17.539689+0000","last_active":"2024-09-13T18:52:18.927484+0000","last_peered":"2024-09-13T18:52:18.927484+0000","last_clean":"2024-09-13T18:52:18.927484+0000","last_became_active":"2024-09-13T18:52:17.539435+0000","last_became_peered":"2024-09-13T18:52:17.539435+0000","last_unstale":"2024-09-13T18:52:18.927484+0000","last_undegraded":"2024-09-13T18:52:18.927484+0000","last_fullsized":"2024-09-13T18:52:18.927484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:22:26.864392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927431+0000","last_change":"2024-09-13T18:52:17.944767+0000","last_active":"2024-09-13T18:52:18.927431+0000","last_peered":"2024-09-13T18:52:18.927431+0000","last_clean":"2024-09-13T18:52:18.927431+0000","last_became_active":"2024-09-13T18:52:17.944163+0000","last_became_peered":"2024-09-13T18:52:17.944163+0000","last_unstale":"2024-09-13T18:52:18.927431+0000","last_undegraded":"2024-09-13T18:52:18.927431+0000","last_fullsized":"2024-09-13T18:52:18.927431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:30.871944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925915+0000","last_change":"2024-09-13T18:52:17.535230+0000","last_active":"2024-09-13T18:52:18.925915+0000","last_peered":"2024-09-13T18:52:18.925915+0000","last_clean":"2024-09-13T18:52:18.925915+0000","last_became_active":"2024-09-13T18:52:17.534923+0000","last_became_peered":"2024-09-13T18:52:17.534923+0000","last_unstale":"2024-09-13T18:52:18.925915+0000","last_undegraded":"2024-09-13T18:52:18.925915+0000","last_fullsized":"2024-09-13T18:52:18.925915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:47:07.580322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198532+0000","last_change":"2024-09-13T18:52:17.949849+0000","last_active":"2024-09-13T18:52:20.198532+0000","last_peered":"2024-09-13T18:52:20.198532+0000","last_clean":"2024-09-13T18:52:20.198532+0000","last_became_active":"2024-09-13T18:52:17.949503+0000","last_became_peered":"2024-09-13T18:52:17.949503+0000","last_unstale":"2024-09-13T18:52:20.198532+0000","last_undegraded":"2024-09-13T18:52:20.198532+0000","last_fullsized":"2024-09-13T18:52:20.198532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:30.871944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198430+0000","last_change":"2024-09-13T18:52:17.540059+0000","last_active":"2024-09-13T18:52:20.198430+0000","last_peered":"2024-09-13T18:52:20.198430+0000","last_clean":"2024-09-13T18:52:20.198430+0000","last_became_active":"2024-09-13T18:52:17.539768+0000","last_became_peered":"2024-09-13T18:52:17.539768+0000","last_unstale":"2024-09-13T18:52:20.198430+0000","last_undegraded":"2024-09-13T18:52:20.198430+0000","last_fullsized":"2024-09-13T18:52:20.198430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:33.879872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200908+0000","last_change":"2024-09-13T18:52:17.947339+0000","last_active":"2024-09-13T18:52:20.200908+0000","last_peered":"2024-09-13T18:52:20.200908+0000","last_clean":"2024-09-13T18:52:20.200908+0000","last_became_active":"2024-09-13T18:52:17.947073+0000","last_became_peered":"2024-09-13T18:52:17.947073+0000","last_unstale":"2024-09-13T18:52:20.200908+0000","last_undegraded":"2024-09-13T18:52:20.200908+0000","last_fullsized":"2024-09-13T18:52:20.200908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:36.215528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925944+0000","last_change":"2024-09-13T18:52:17.534989+0000","last_active":"2024-09-13T18:52:18.925944+0000","last_peered":"2024-09-13T18:52:18.925944+0000","last_clean":"2024-09-13T18:52:18.925944+0000","last_became_active":"2024-09-13T18:52:17.532634+0000","last_became_peered":"2024-09-13T18:52:17.532634+0000","last_unstale":"2024-09-13T18:52:18.925944+0000","last_undegraded":"2024-09-13T18:52:18.925944+0000","last_fullsized":"2024-09-13T18:52:18.925944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:10:12.228931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200958+0000","last_change":"2024-09-13T18:52:17.949744+0000","last_active":"2024-09-13T18:52:20.200958+0000","last_peered":"2024-09-13T18:52:20.200958+0000","last_clean":"2024-09-13T18:52:20.200958+0000","last_became_active":"2024-09-13T18:52:17.949469+0000","last_became_peered":"2024-09-13T18:52:17.949469+0000","last_unstale":"2024-09-13T18:52:20.200958+0000","last_undegraded":"2024-09-13T18:52:20.200958+0000","last_fullsized":"2024-09-13T18:52:20.200958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:42:22.626590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195041+0000","last_change":"2024-09-13T18:52:17.537724+0000","last_active":"2024-09-13T18:52:20.195041+0000","last_peered":"2024-09-13T18:52:20.195041+0000","last_clean":"2024-09-13T18:52:20.195041+0000","last_became_active":"2024-09-13T18:52:17.537474+0000","last_became_peered":"2024-09-13T18:52:17.537474+0000","last_unstale":"2024-09-13T18:52:20.195041+0000","last_undegraded":"2024-09-13T18:52:20.195041+0000","last_fullsized":"2024-09-13T18:52:20.195041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:06.694707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199207+0000","last_change":"2024-09-13T18:52:17.939592+0000","last_active":"2024-09-13T18:52:20.199207+0000","last_peered":"2024-09-13T18:52:20.199207+0000","last_clean":"2024-09-13T18:52:20.199207+0000","last_became_active":"2024-09-13T18:52:17.939331+0000","last_became_peered":"2024-09-13T18:52:17.939331+0000","last_unstale":"2024-09-13T18:52:20.199207+0000","last_undegraded":"2024-09-13T18:52:20.199207+0000","last_fullsized":"2024-09-13T18:52:20.199207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:31.026983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.926147+0000","last_change":"2024-09-13T18:52:17.536715+0000","last_active":"2024-09-13T18:52:18.926147+0000","last_peered":"2024-09-13T18:52:18.926147+0000","last_clean":"2024-09-13T18:52:18.926147+0000","last_became_active":"2024-09-13T18:52:17.536409+0000","last_became_peered":"2024-09-13T18:52:17.536409+0000","last_unstale":"2024-09-13T18:52:18.926147+0000","last_undegraded":"2024-09-13T18:52:18.926147+0000","last_fullsized":"2024-09-13T18:52:18.926147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:01.849550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200621+0000","last_change":"2024-09-13T18:52:17.940926+0000","last_active":"2024-09-13T18:52:20.200621+0000","last_peered":"2024-09-13T18:52:20.200621+0000","last_clean":"2024-09-13T18:52:20.200621+0000","last_became_active":"2024-09-13T18:52:17.940572+0000","last_became_peered":"2024-09-13T18:52:17.940572+0000","last_unstale":"2024-09-13T18:52:20.200621+0000","last_undegraded":"2024-09-13T18:52:20.200621+0000","last_fullsized":"2024-09-13T18:52:20.200621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.990929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T18:52:17.928742+0000","last_change":"2024-09-13T18:52:15.080080+0000","last_active":"2024-09-13T18:52:17.928742+0000","last_peered":"2024-09-13T18:52:17.928742+0000","last_clean":"2024-09-13T18:52:17.928742+0000","last_became_active":"2024-09-13T18:52:15.078100+0000","last_became_peered":"2024-09-13T18:52:15.078100+0000","last_unstale":"2024-09-13T18:52:17.928742+0000","last_undegraded":"2024-09-13T18:52:17.928742+0000","last_fullsized":"2024-09-13T18:52:17.928742+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_clean_scrub_stamp":"2024-09-13T18:52:13.196473+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:29:59.452757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197092+0000","last_change":"2024-09-13T18:52:17.531117+0000","last_active":"2024-09-13T18:52:20.197092+0000","last_peered":"2024-09-13T18:52:20.197092+0000","last_clean":"2024-09-13T18:52:20.197092+0000","last_became_active":"2024-09-13T18:52:17.530778+0000","last_became_peered":"2024-09-13T18:52:17.530778+0000","last_unstale":"2024-09-13T18:52:20.197092+0000","last_undegraded":"2024-09-13T18:52:20.197092+0000","last_fullsized":"2024-09-13T18:52:20.197092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:06.694707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924347+0000","last_change":"2024-09-13T18:52:17.943655+0000","last_active":"2024-09-13T18:52:18.924347+0000","last_peered":"2024-09-13T18:52:18.924347+0000","last_clean":"2024-09-13T18:52:18.924347+0000","last_became_active":"2024-09-13T18:52:17.943301+0000","last_became_peered":"2024-09-13T18:52:17.943301+0000","last_unstale":"2024-09-13T18:52:18.924347+0000","last_undegraded":"2024-09-13T18:52:18.924347+0000","last_fullsized":"2024-09-13T18:52:18.924347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:53.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197196+0000","last_change":"2024-09-13T18:52:17.548220+0000","last_active":"2024-09-13T18:52:20.197196+0000","last_peered":"2024-09-13T18:52:20.197196+0000","last_clean":"2024-09-13T18:52:20.197196+0000","last_became_active":"2024-09-13T18:52:17.547957+0000","last_became_peered":"2024-09-13T18:52:17.547957+0000","last_unstale":"2024-09-13T18:52:20.197196+0000","last_undegraded":"2024-09-13T18:52:20.197196+0000","last_fullsized":"2024-09-13T18:52:20.197196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:47.942952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.194587+0000","last_change":"2024-09-13T18:52:17.946587+0000","last_active":"2024-09-13T18:52:20.194587+0000","last_peered":"2024-09-13T18:52:20.194587+0000","last_clean":"2024-09-13T18:52:20.194587+0000","last_became_active":"2024-09-13T18:52:17.946352+0000","last_became_peered":"2024-09-13T18:52:17.946352+0000","last_unstale":"2024-09-13T18:52:20.194587+0000","last_undegraded":"2024-09-13T18:52:20.194587+0000","last_fullsized":"2024-09-13T18:52:20.194587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:37.404109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.926130+0000","last_change":"2024-09-13T18:52:17.532972+0000","last_active":"2024-09-13T18:52:18.926130+0000","last_peered":"2024-09-13T18:52:18.926130+0000","last_clean":"2024-09-13T18:52:18.926130+0000","last_became_active":"2024-09-13T18:52:17.532675+0000","last_became_peered":"2024-09-13T18:52:17.532675+0000","last_unstale":"2024-09-13T18:52:18.926130+0000","last_undegraded":"2024-09-13T18:52:18.926130+0000","last_fullsized":"2024-09-13T18:52:18.926130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:41.232168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.194709+0000","last_change":"2024-09-13T18:52:17.955502+0000","last_active":"2024-09-13T18:52:20.194709+0000","last_peered":"2024-09-13T18:52:20.194709+0000","last_clean":"2024-09-13T18:52:20.194709+0000","last_became_active":"2024-09-13T18:52:17.955061+0000","last_became_peered":"2024-09-13T18:52:17.955061+0000","last_unstale":"2024-09-13T18:52:20.194709+0000","last_undegraded":"2024-09-13T18:52:20.194709+0000","last_fullsized":"2024-09-13T18:52:20.194709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:24.025045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197805+0000","last_change":"2024-09-13T18:52:17.535698+0000","last_active":"2024-09-13T18:52:20.197805+0000","last_peered":"2024-09-13T18:52:20.197805+0000","last_clean":"2024-09-13T18:52:20.197805+0000","last_became_active":"2024-09-13T18:52:17.535439+0000","last_became_peered":"2024-09-13T18:52:17.535439+0000","last_unstale":"2024-09-13T18:52:20.197805+0000","last_undegraded":"2024-09-13T18:52:20.197805+0000","last_fullsized":"2024-09-13T18:52:20.197805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:47.942952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197950+0000","last_change":"2024-09-13T18:52:17.945038+0000","last_active":"2024-09-13T18:52:20.197950+0000","last_peered":"2024-09-13T18:52:20.197950+0000","last_clean":"2024-09-13T18:52:20.197950+0000","last_became_active":"2024-09-13T18:52:17.944770+0000","last_became_peered":"2024-09-13T18:52:17.944770+0000","last_unstale":"2024-09-13T18:52:20.197950+0000","last_undegraded":"2024-09-13T18:52:20.197950+0000","last_fullsized":"2024-09-13T18:52:20.197950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:24:03.326121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196209+0000","last_change":"2024-09-13T18:52:17.539013+0000","last_active":"2024-09-13T18:52:20.196209+0000","last_peered":"2024-09-13T18:52:20.196209+0000","last_clean":"2024-09-13T18:52:20.196209+0000","last_became_active":"2024-09-13T18:52:17.538720+0000","last_became_peered":"2024-09-13T18:52:17.538720+0000","last_unstale":"2024-09-13T18:52:20.196209+0000","last_undegraded":"2024-09-13T18:52:20.196209+0000","last_fullsized":"2024-09-13T18:52:20.196209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:49.151192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199583+0000","last_change":"2024-09-13T18:52:17.943875+0000","last_active":"2024-09-13T18:52:20.199583+0000","last_peered":"2024-09-13T18:52:20.199583+0000","last_clean":"2024-09-13T18:52:20.199583+0000","last_became_active":"2024-09-13T18:52:17.943557+0000","last_became_peered":"2024-09-13T18:52:17.943557+0000","last_unstale":"2024-09-13T18:52:20.199583+0000","last_undegraded":"2024-09-13T18:52:20.199583+0000","last_fullsized":"2024-09-13T18:52:20.199583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:18.183823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200835+0000","last_change":"2024-09-13T18:52:17.548571+0000","last_active":"2024-09-13T18:52:20.200835+0000","last_peered":"2024-09-13T18:52:20.200835+0000","last_clean":"2024-09-13T18:52:20.200835+0000","last_became_active":"2024-09-13T18:52:17.548306+0000","last_became_peered":"2024-09-13T18:52:17.548306+0000","last_unstale":"2024-09-13T18:52:20.200835+0000","last_undegraded":"2024-09-13T18:52:20.200835+0000","last_fullsized":"2024-09-13T18:52:20.200835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:22:26.864392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195209+0000","last_change":"2024-09-13T18:52:17.954517+0000","last_active":"2024-09-13T18:52:20.195209+0000","last_peered":"2024-09-13T18:52:20.195209+0000","last_clean":"2024-09-13T18:52:20.195209+0000","last_became_active":"2024-09-13T18:52:17.954264+0000","last_became_peered":"2024-09-13T18:52:17.954264+0000","last_unstale":"2024-09-13T18:52:20.195209+0000","last_undegraded":"2024-09-13T18:52:20.195209+0000","last_fullsized":"2024-09-13T18:52:20.195209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:57:51.000231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:21.214627+0000","last_change":"2024-09-13T18:52:17.538927+0000","last_active":"2024-09-13T18:52:21.214627+0000","last_peered":"2024-09-13T18:52:21.214627+0000","last_clean":"2024-09-13T18:52:21.214627+0000","last_became_active":"2024-09-13T18:52:17.538334+0000","last_became_peered":"2024-09-13T18:52:17.538334+0000","last_unstale":"2024-09-13T18:52:21.214627+0000","last_undegraded":"2024-09-13T18:52:21.214627+0000","last_fullsized":"2024-09-13T18:52:21.214627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:47.942952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924572+0000","last_change":"2024-09-13T18:52:17.929155+0000","last_active":"2024-09-13T18:52:18.924572+0000","last_peered":"2024-09-13T18:52:18.924572+0000","last_clean":"2024-09-13T18:52:18.924572+0000","last_became_active":"2024-09-13T18:52:17.928876+0000","last_became_peered":"2024-09-13T18:52:17.928876+0000","last_unstale":"2024-09-13T18:52:18.924572+0000","last_undegraded":"2024-09-13T18:52:18.924572+0000","last_fullsized":"2024-09-13T18:52:18.924572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:01:39.422836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.928355+0000","last_change":"2024-09-13T18:52:17.543500+0000","last_active":"2024-09-13T18:52:18.928355+0000","last_peered":"2024-09-13T18:52:18.928355+0000","last_clean":"2024-09-13T18:52:18.928355+0000","last_became_active":"2024-09-13T18:52:17.543184+0000","last_became_peered":"2024-09-13T18:52:17.543184+0000","last_unstale":"2024-09-13T18:52:18.928355+0000","last_undegraded":"2024-09-13T18:52:18.928355+0000","last_fullsized":"2024-09-13T18:52:18.928355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:05:28.094741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195839+0000","last_change":"2024-09-13T18:52:17.942277+0000","last_active":"2024-09-13T18:52:20.195839+0000","last_peered":"2024-09-13T18:52:20.195839+0000","last_clean":"2024-09-13T18:52:20.195839+0000","last_became_active":"2024-09-13T18:52:17.941984+0000","last_became_peered":"2024-09-13T18:52:17.941984+0000","last_unstale":"2024-09-13T18:52:20.195839+0000","last_undegraded":"2024-09-13T18:52:20.195839+0000","last_fullsized":"2024-09-13T18:52:20.195839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:20:42.480429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197001+0000","last_change":"2024-09-13T18:52:17.545742+0000","last_active":"2024-09-13T18:52:20.197001+0000","last_peered":"2024-09-13T18:52:20.197001+0000","last_clean":"2024-09-13T18:52:20.197001+0000","last_became_active":"2024-09-13T18:52:17.545448+0000","last_became_peered":"2024-09-13T18:52:17.545448+0000","last_unstale":"2024-09-13T18:52:20.197001+0000","last_undegraded":"2024-09-13T18:52:20.197001+0000","last_fullsized":"2024-09-13T18:52:20.197001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:49.151192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196848+0000","last_change":"2024-09-13T18:52:17.955942+0000","last_active":"2024-09-13T18:52:20.196848+0000","last_peered":"2024-09-13T18:52:20.196848+0000","last_clean":"2024-09-13T18:52:20.196848+0000","last_became_active":"2024-09-13T18:52:17.955598+0000","last_became_peered":"2024-09-13T18:52:17.955598+0000","last_unstale":"2024-09-13T18:52:20.196848+0000","last_undegraded":"2024-09-13T18:52:20.196848+0000","last_fullsized":"2024-09-13T18:52:20.196848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:42.236037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196212+0000","last_change":"2024-09-13T18:52:17.541045+0000","last_active":"2024-09-13T18:52:20.196212+0000","last_peered":"2024-09-13T18:52:20.196212+0000","last_clean":"2024-09-13T18:52:20.196212+0000","last_became_active":"2024-09-13T18:52:17.540627+0000","last_became_peered":"2024-09-13T18:52:17.540627+0000","last_unstale":"2024-09-13T18:52:20.196212+0000","last_undegraded":"2024-09-13T18:52:20.196212+0000","last_fullsized":"2024-09-13T18:52:20.196212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:14:15.613948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924137+0000","last_change":"2024-09-13T18:52:17.924064+0000","last_active":"2024-09-13T18:52:18.924137+0000","last_peered":"2024-09-13T18:52:18.924137+0000","last_clean":"2024-09-13T18:52:18.924137+0000","last_became_active":"2024-09-13T18:52:17.921690+0000","last_became_peered":"2024-09-13T18:52:17.921690+0000","last_unstale":"2024-09-13T18:52:18.924137+0000","last_undegraded":"2024-09-13T18:52:18.924137+0000","last_fullsized":"2024-09-13T18:52:18.924137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:30.871944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195867+0000","last_change":"2024-09-13T18:52:17.533845+0000","last_active":"2024-09-13T18:52:20.195867+0000","last_peered":"2024-09-13T18:52:20.195867+0000","last_clean":"2024-09-13T18:52:20.195867+0000","last_became_active":"2024-09-13T18:52:17.533579+0000","last_became_peered":"2024-09-13T18:52:17.533579+0000","last_unstale":"2024-09-13T18:52:20.195867+0000","last_undegraded":"2024-09-13T18:52:20.195867+0000","last_fullsized":"2024-09-13T18:52:20.195867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:04.695688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925441+0000","last_change":"2024-09-13T18:52:17.944835+0000","last_active":"2024-09-13T18:52:18.925441+0000","last_peered":"2024-09-13T18:52:18.925441+0000","last_clean":"2024-09-13T18:52:18.925441+0000","last_became_active":"2024-09-13T18:52:17.944469+0000","last_became_peered":"2024-09-13T18:52:17.944469+0000","last_unstale":"2024-09-13T18:52:18.925441+0000","last_undegraded":"2024-09-13T18:52:18.925441+0000","last_fullsized":"2024-09-13T18:52:18.925441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:31.026983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200263+0000","last_change":"2024-09-13T18:52:17.545810+0000","last_active":"2024-09-13T18:52:20.200263+0000","last_peered":"2024-09-13T18:52:20.200263+0000","last_clean":"2024-09-13T18:52:20.200263+0000","last_became_active":"2024-09-13T18:52:17.545523+0000","last_became_peered":"2024-09-13T18:52:17.545523+0000","last_unstale":"2024-09-13T18:52:20.200263+0000","last_undegraded":"2024-09-13T18:52:20.200263+0000","last_fullsized":"2024-09-13T18:52:20.200263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:05:28.094741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927471+0000","last_change":"2024-09-13T18:52:17.940857+0000","last_active":"2024-09-13T18:52:18.927471+0000","last_peered":"2024-09-13T18:52:18.927471+0000","last_clean":"2024-09-13T18:52:18.927471+0000","last_became_active":"2024-09-13T18:52:17.940675+0000","last_became_peered":"2024-09-13T18:52:17.940675+0000","last_unstale":"2024-09-13T18:52:18.927471+0000","last_undegraded":"2024-09-13T18:52:18.927471+0000","last_fullsized":"2024-09-13T18:52:18.927471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:53.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199367+0000","last_change":"2024-09-13T18:52:17.947933+0000","last_active":"2024-09-13T18:52:20.199367+0000","last_peered":"2024-09-13T18:52:20.199367+0000","last_clean":"2024-09-13T18:52:20.199367+0000","last_became_active":"2024-09-13T18:52:17.947665+0000","last_became_peered":"2024-09-13T18:52:17.947665+0000","last_unstale":"2024-09-13T18:52:20.199367+0000","last_undegraded":"2024-09-13T18:52:20.199367+0000","last_fullsized":"2024-09-13T18:52:20.199367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:21.405599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199849+0000","last_change":"2024-09-13T18:52:17.548211+0000","last_active":"2024-09-13T18:52:20.199849+0000","last_peered":"2024-09-13T18:52:20.199849+0000","last_clean":"2024-09-13T18:52:20.199849+0000","last_became_active":"2024-09-13T18:52:17.547997+0000","last_became_peered":"2024-09-13T18:52:17.547997+0000","last_unstale":"2024-09-13T18:52:20.199849+0000","last_undegraded":"2024-09-13T18:52:20.199849+0000","last_fullsized":"2024-09-13T18:52:20.199849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:06.694707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199118+0000","last_change":"2024-09-13T18:52:17.948133+0000","last_active":"2024-09-13T18:52:20.199118+0000","last_peered":"2024-09-13T18:52:20.199118+0000","last_clean":"2024-09-13T18:52:20.199118+0000","last_became_active":"2024-09-13T18:52:17.947793+0000","last_became_peered":"2024-09-13T18:52:17.947793+0000","last_unstale":"2024-09-13T18:52:20.199118+0000","last_undegraded":"2024-09-13T18:52:20.199118+0000","last_fullsized":"2024-09-13T18:52:20.199118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:20:43.355560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200592+0000","last_change":"2024-09-13T18:52:17.548922+0000","last_active":"2024-09-13T18:52:20.200592+0000","last_peered":"2024-09-13T18:52:20.200592+0000","last_clean":"2024-09-13T18:52:20.200592+0000","last_became_active":"2024-09-13T18:52:17.548775+0000","last_became_peered":"2024-09-13T18:52:17.548775+0000","last_unstale":"2024-09-13T18:52:20.200592+0000","last_undegraded":"2024-09-13T18:52:20.200592+0000","last_fullsized":"2024-09-13T18:52:20.200592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:14:15.613948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195999+0000","last_change":"2024-09-13T18:52:17.950416+0000","last_active":"2024-09-13T18:52:20.195999+0000","last_peered":"2024-09-13T18:52:20.195999+0000","last_clean":"2024-09-13T18:52:20.195999+0000","last_became_active":"2024-09-13T18:52:17.949988+0000","last_became_peered":"2024-09-13T18:52:17.949988+0000","last_unstale":"2024-09-13T18:52:20.195999+0000","last_undegraded":"2024-09-13T18:52:20.195999+0000","last_fullsized":"2024-09-13T18:52:20.195999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:45.768218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925140+0000","last_change":"2024-09-13T18:52:17.535536+0000","last_active":"2024-09-13T18:52:18.925140+0000","last_peered":"2024-09-13T18:52:18.925140+0000","last_clean":"2024-09-13T18:52:18.925140+0000","last_became_active":"2024-09-13T18:52:17.534924+0000","last_became_peered":"2024-09-13T18:52:17.534924+0000","last_unstale":"2024-09-13T18:52:18.925140+0000","last_undegraded":"2024-09-13T18:52:18.925140+0000","last_fullsized":"2024-09-13T18:52:18.925140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:55.776670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924931+0000","last_change":"2024-09-13T18:52:17.931585+0000","last_active":"2024-09-13T18:52:18.924931+0000","last_peered":"2024-09-13T18:52:18.924931+0000","last_clean":"2024-09-13T18:52:18.924931+0000","last_became_active":"2024-09-13T18:52:17.931280+0000","last_became_peered":"2024-09-13T18:52:17.931280+0000","last_unstale":"2024-09-13T18:52:18.924931+0000","last_undegraded":"2024-09-13T18:52:18.924931+0000","last_fullsized":"2024-09-13T18:52:18.924931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:20:42.480429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196202+0000","last_change":"2024-09-13T18:52:17.542275+0000","last_active":"2024-09-13T18:52:20.196202+0000","last_peered":"2024-09-13T18:52:20.196202+0000","last_clean":"2024-09-13T18:52:20.196202+0000","last_became_active":"2024-09-13T18:52:17.541964+0000","last_became_peered":"2024-09-13T18:52:17.541964+0000","last_unstale":"2024-09-13T18:52:20.196202+0000","last_undegraded":"2024-09-13T18:52:20.196202+0000","last_fullsized":"2024-09-13T18:52:20.196202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:55.776670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":823296,"data_stored":669740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]}]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":80354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.556}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2170000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3240000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":78880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5620000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]}]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":78916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]}]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":237568,"data_stored":80354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.373}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":225280,"data_stored":78790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":233472,"data_stored":78950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":831488,"data_stored":669796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:52:22.336 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T18:52:22.336 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T18:52:22.337 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T18:52:22.337 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T18:52:22.337 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T18:52:22.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.454+0000 7f9915000700 1 Processor -- start 2024-09-13T18:52:22.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.454+0000 7f9915000700 1 -- start start 2024-09-13T18:52:22.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.454+0000 7f9915000700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f991006bb90 0x7f99100cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.454+0000 7f9915000700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991006a780 con 0x7f991006fb30 2024-09-13T18:52:22.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.454+0000 7f9915000700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9910159470 con 0x7f991006d150 2024-09-13T18:52:22.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.454+0000 7f9915000700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991006cd30 con 0x7f991006bb90 2024-09-13T18:52:22.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.454+0000 7f990f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f991006fb30 0x7f99100adbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35708/0 (socket says 172.21.15.7:35708) 2024-09-13T18:52:22.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.454+0000 7f990f7fe700 1 -- 172.21.15.7:0/3132878191 learned_addr learned my addr 172.21.15.7:0/3132878191 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:22.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.454+0000 7f990effd700 1 --2- 172.21.15.7:0/3132878191 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f991006bb90 0x7f99100cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.454+0000 7f990e7fc700 1 -- 172.21.15.7:0/3132878191 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134894243 0 0) 0x7f991006a780 con 0x7f991006fb30 2024-09-13T18:52:22.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.454+0000 7f990e7fc700 1 -- 172.21.15.7:0/3132878191 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98f80035d0 con 0x7f991006fb30 2024-09-13T18:52:22.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.454+0000 7f990e7fc700 1 -- 172.21.15.7:0/3132878191 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 870146941 0 0) 0x7f9910159470 con 0x7f991006d150 2024-09-13T18:52:22.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.454+0000 7f990e7fc700 1 -- 172.21.15.7:0/3132878191 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f991006a780 con 0x7f991006d150 2024-09-13T18:52:22.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.454+0000 7f990e7fc700 1 -- 172.21.15.7:0/3132878191 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3167955575 0 0) 0x7f98f80035d0 con 0x7f991006fb30 2024-09-13T18:52:22.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990e7fc700 1 -- 172.21.15.7:0/3132878191 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f991006bb90 msgr2=0x7f99100cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:22.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990e7fc700 1 --2- 172.21.15.7:0/3132878191 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f991006bb90 0x7f99100cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990e7fc700 1 -- 172.21.15.7:0/3132878191 >> v1:172.21.15.171:6789/0 conn(0x7f991006d150 legacy=0x7f991006d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:22.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990e7fc700 1 -- 172.21.15.7:0/3132878191 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991006c0b0 con 0x7f991006fb30 2024-09-13T18:52:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990e7fc700 1 -- 172.21.15.7:0/3132878191 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7f9900001670 con 0x7f991006fb30 2024-09-13T18:52:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990e7fc700 1 -- 172.21.15.7:0/3132878191 >> v1:172.21.15.7:6789/0 conn(0x7f991006fb30 legacy=0x7f99100adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990e7fc700 1 --2- 172.21.15.7:0/3132878191 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f991006bb90 0x7f98f8003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990e7fc700 1 --2- 172.21.15.7:0/3132878191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98f8004140 0x7f98f8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990e7fc700 1 --2- 172.21.15.7:0/3132878191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98f8004db0 0x7f98f8005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990e7fc700 1 -- 172.21.15.7:0/3132878191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991006cd30 con 0x7f98f8004140 2024-09-13T18:52:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990e7fc700 1 -- 172.21.15.7:0/3132878191 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f800de50 con 0x7f98f8004db0 2024-09-13T18:52:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990e7fc700 1 -- 172.21.15.7:0/3132878191 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f800e010 con 0x7f991006bb90 2024-09-13T18:52:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990effd700 1 --2- 172.21.15.7:0/3132878191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98f8004140 0x7f98f8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990f7fe700 1 --2- 172.21.15.7:0/3132878191 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f991006bb90 0x7f98f8003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990ffff700 1 --2- 172.21.15.7:0/3132878191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98f8004db0 0x7f98f8005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990effd700 1 -- 172.21.15.7:0/3132878191 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f991006bb90 msgr2=0x7f98f8003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990effd700 1 --2- 172.21.15.7:0/3132878191 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f991006bb90 0x7f98f8003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990effd700 1 -- 172.21.15.7:0/3132878191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98f8004db0 msgr2=0x7f98f8005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990effd700 1 --2- 172.21.15.7:0/3132878191 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98f8004db0 0x7f98f8005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990effd700 1 -- 172.21.15.7:0/3132878191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f80035d0 con 0x7f98f8004140 2024-09-13T18:52:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990effd700 1 --2- 172.21.15.7:0/3132878191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98f8004140 0x7f98f8004870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9904009b40 tx=0x7f98f80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90453d012276b47 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990e7fc700 1 -- 172.21.15.7:0/3132878191 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990400e6c0 con 0x7f98f8004140 2024-09-13T18:52:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990e7fc700 1 -- 172.21.15.7:0/3132878191 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990400e820 con 0x7f98f8004140 2024-09-13T18:52:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990e7fc700 1 -- 172.21.15.7:0/3132878191 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990400eb90 con 0x7f98f8004140 2024-09-13T18:52:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f9915000700 1 -- 172.21.15.7:0/3132878191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98f8004140 msgr2=0x7f98f8004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f9915000700 1 --2- 172.21.15.7:0/3132878191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98f8004140 0x7f98f8004870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9904009b40 tx=0x7f98f80058a0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f990ffff700 1 -- 172.21.15.7:0/3132878191 reap_dead start 2024-09-13T18:52:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f9915000700 1 -- 172.21.15.7:0/3132878191 shutdown_connections 2024-09-13T18:52:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f9915000700 1 -- 172.21.15.7:0/3132878191 >> 172.21.15.7:0/3132878191 conn(0x7f99100b0f00 msgr2=0x7f99100b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f9915000700 1 -- 172.21.15.7:0/3132878191 shutdown_connections 2024-09-13T18:52:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.458+0000 7f9915000700 1 -- 172.21.15.7:0/3132878191 wait complete. 2024-09-13T18:52:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f9915000700 1 Processor -- start 2024-09-13T18:52:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f9915000700 1 -- start start 2024-09-13T18:52:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f9915000700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98f8004140 0x7f99100ab750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f9915000700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f991015b4e0 0x7f991014bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f9915000700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f991014de80 0x7f991014e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f9915000700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991014e840 con 0x7f991014de80 2024-09-13T18:52:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f9915000700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99100abc90 con 0x7f98f8004140 2024-09-13T18:52:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f9915000700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99100abee0 con 0x7f991015b4e0 2024-09-13T18:52:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f990ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f991014de80 0x7f991014e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f990ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f991014de80 0x7f991014e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49232/0 (socket says 172.21.15.7:49232) 2024-09-13T18:52:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f990ffff700 1 -- 172.21.15.7:0/3905548395 learned_addr learned my addr 172.21.15.7:0/3905548395 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f990f7fe700 1 --2- 172.21.15.7:0/3905548395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98f8004140 0x7f99100ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f990effd700 1 --2- 172.21.15.7:0/3905548395 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f991015b4e0 0x7f991014bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f990ffff700 1 -- 172.21.15.7:0/3905548395 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f991015b4e0 msgr2=0x7f991014bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f990ffff700 1 --2- 172.21.15.7:0/3905548395 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f991015b4e0 0x7f991014bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f990ffff700 1 -- 172.21.15.7:0/3905548395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98f8004140 msgr2=0x7f99100ab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f990ffff700 1 --2- 172.21.15.7:0/3905548395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98f8004140 0x7f99100ab750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f990ffff700 1 -- 172.21.15.7:0/3905548395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99100ac1c0 con 0x7f991014de80 2024-09-13T18:52:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f990effd700 1 --2- 172.21.15.7:0/3905548395 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f991015b4e0 0x7f991014bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f990ffff700 1 --2- 172.21.15.7:0/3905548395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f991014de80 0x7f991014e300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9910071ac0 tx=0x7f990800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f98f7fff700 1 -- 172.21.15.7:0/3905548395 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908003c30 con 0x7f991014de80 2024-09-13T18:52:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f98f7fff700 1 -- 172.21.15.7:0/3905548395 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9908003d90 con 0x7f991014de80 2024-09-13T18:52:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f9915000700 1 -- 172.21.15.7:0/3905548395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99100ac4a0 con 0x7f991014de80 2024-09-13T18:52:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f98f7fff700 1 -- 172.21.15.7:0/3905548395 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990800c950 con 0x7f991014de80 2024-09-13T18:52:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.462+0000 7f9915000700 1 -- 172.21.15.7:0/3905548395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99100aca60 con 0x7f991014de80 2024-09-13T18:52:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.466+0000 7f9915000700 1 -- 172.21.15.7:0/3905548395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99100b3710 con 0x7f991014de80 2024-09-13T18:52:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.466+0000 7f98f7fff700 1 -- 172.21.15.7:0/3905548395 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f990800cab0 con 0x7f991014de80 2024-09-13T18:52:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.466+0000 7f98f7fff700 1 --2- 172.21.15.7:0/3905548395 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f98fc066730 0x7f98fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.466+0000 7f98f7fff700 1 -- 172.21.15.7:0/3905548395 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99080841d0 con 0x7f991014de80 2024-09-13T18:52:22.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.466+0000 7f990f7fe700 1 --2- 172.21.15.7:0/3905548395 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f98fc066730 0x7f98fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.466+0000 7f990f7fe700 1 --2- 172.21.15.7:0/3905548395 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f98fc066730 0x7f98fc068bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99101f13f0 tx=0x7f9900009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:22.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.474+0000 7f98f7fff700 1 -- 172.21.15.7:0/3905548395 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9908051540 con 0x7f991014de80 2024-09-13T18:52:22.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.686+0000 7f9915000700 1 -- 172.21.15.7:0/3905548395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f9910067b70 con 0x7f991014de80 2024-09-13T18:52:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.694+0000 7f98f7fff700 1 -- 172.21.15.7:0/3905548395 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164905 (secure 0 0 0) 0x7f9908054b60 con 0x7f991014de80 2024-09-13T18:52:22.704 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:22.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.698+0000 7f9915000700 1 -- 172.21.15.7:0/3905548395 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f98fc066730 msgr2=0x7f98fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:22.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.698+0000 7f9915000700 1 --2- 172.21.15.7:0/3905548395 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f98fc066730 0x7f98fc068bf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f99101f13f0 tx=0x7f9900009040 comp rx=0 tx=0).stop 2024-09-13T18:52:22.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.698+0000 7f9915000700 1 -- 172.21.15.7:0/3905548395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f991014de80 msgr2=0x7f991014e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:22.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.698+0000 7f9915000700 1 --2- 172.21.15.7:0/3905548395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f991014de80 0x7f991014e300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9910071ac0 tx=0x7f990800ef20 comp rx=0 tx=0).stop 2024-09-13T18:52:22.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.698+0000 7f9915000700 1 -- 172.21.15.7:0/3905548395 shutdown_connections 2024-09-13T18:52:22.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.698+0000 7f9915000700 1 --2- 172.21.15.7:0/3905548395 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f98fc066730 0x7f98fc068bf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.698+0000 7f9915000700 1 --2- 172.21.15.7:0/3905548395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f991014de80 0x7f991014e300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.698+0000 7f9915000700 1 --2- 172.21.15.7:0/3905548395 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f991015b4e0 0x7f991014bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.698+0000 7f9915000700 1 --2- 172.21.15.7:0/3905548395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f98f8004140 0x7f99100ab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.698+0000 7f9915000700 1 -- 172.21.15.7:0/3905548395 >> 172.21.15.7:0/3905548395 conn(0x7f99100b0f00 msgr2=0x7f9910070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.698+0000 7f9915000700 1 -- 172.21.15.7:0/3905548395 shutdown_connections 2024-09-13T18:52:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.698+0000 7f9915000700 1 -- 172.21.15.7:0/3905548395 wait complete. 2024-09-13T18:52:22.717 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":2829153556},{"type":"v1","addr":"172.21.15.171:6833","nonce":2829153556}]},"active_addr":"172.21.15.171:6833/2829153556","active_change":"2024-09-13T18:52:06.193440+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.171:0","nonce":2214102650}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.171:0","nonce":2648341040}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.171:0","nonce":66234119}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.171:0","nonce":2939409575}]}]} 2024-09-13T18:52:22.722 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T18:52:22.722 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T18:52:22.723 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:22.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.834+0000 7f7ed15bc700 1 Processor -- start 2024-09-13T18:52:22.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.834+0000 7f7ed15bc700 1 -- start start 2024-09-13T18:52:22.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed15bc700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f7ecc159720 0x7f7ecc06cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed15bc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ecc0b3ae0 0x7f7ecc0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed15bc700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ecc06af00 con 0x7f7ecc06aa90 2024-09-13T18:52:22.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed15bc700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ecc06d690 con 0x7f7ecc0b3ae0 2024-09-13T18:52:22.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed15bc700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ecc0cbfe0 con 0x7f7ecc159720 2024-09-13T18:52:22.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed0dbb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ecc0b3ae0 0x7f7ecc0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed0dbb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ecc0b3ae0 0x7f7ecc0ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49238/0 (socket says 172.21.15.7:49238) 2024-09-13T18:52:22.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ecb7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7ecc06aa90 0x7f7ecc0b33d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35714/0 (socket says 172.21.15.7:35714) 2024-09-13T18:52:22.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed0dbb700 1 -- 172.21.15.7:0/1528214187 learned_addr learned my addr 172.21.15.7:0/1528214187 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:22.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ecbfff700 1 --2- 172.21.15.7:0/1528214187 >> v2:172.21.15.171:3300/0 conn(0x7f7ecc159720 0x7f7ecc06cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed0dbb700 1 -- 172.21.15.7:0/1528214187 >> v2:172.21.15.171:3300/0 conn(0x7f7ecc159720 msgr2=0x7f7ecc06cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:22.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed0dbb700 1 --2- 172.21.15.7:0/1528214187 >> v2:172.21.15.171:3300/0 conn(0x7f7ecc159720 0x7f7ecc06cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed0dbb700 1 -- 172.21.15.7:0/1528214187 >> v1:172.21.15.7:6789/0 conn(0x7f7ecc06aa90 legacy=0x7f7ecc0b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:22.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed0dbb700 1 -- 172.21.15.7:0/1528214187 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc0cc2c0 con 0x7f7ecc0b3ae0 2024-09-13T18:52:22.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ecbfff700 1 --2- 172.21.15.7:0/1528214187 >> v2:172.21.15.171:3300/0 conn(0x7f7ecc159720 0x7f7ecc06cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:22.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed0dbb700 1 --2- 172.21.15.7:0/1528214187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc0b3ae0 0x7f7ecc0ccac0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc071cb0 tx=0x7f7ec400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9469a2da74e14670 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:22.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ecaffd700 1 -- 172.21.15.7:0/1528214187 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec400e070 con 0x7f7ecc0b3ae0 2024-09-13T18:52:22.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ecaffd700 1 -- 172.21.15.7:0/1528214187 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec400b9c0 con 0x7f7ecc0b3ae0 2024-09-13T18:52:22.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ecaffd700 1 -- 172.21.15.7:0/1528214187 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec400bd70 con 0x7f7ecc0b3ae0 2024-09-13T18:52:22.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed15bc700 1 -- 172.21.15.7:0/1528214187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc0b3ae0 msgr2=0x7f7ecc0ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:22.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed15bc700 1 --2- 172.21.15.7:0/1528214187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc0b3ae0 0x7f7ecc0ccac0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc071cb0 tx=0x7f7ec400b330 comp rx=0 tx=0).stop 2024-09-13T18:52:22.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed15bc700 1 -- 172.21.15.7:0/1528214187 shutdown_connections 2024-09-13T18:52:22.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed15bc700 1 --2- 172.21.15.7:0/1528214187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc0b3ae0 0x7f7ecc0ccac0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed15bc700 1 --2- 172.21.15.7:0/1528214187 >> v2:172.21.15.171:3300/0 conn(0x7f7ecc159720 0x7f7ecc06cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed15bc700 1 -- 172.21.15.7:0/1528214187 >> 172.21.15.7:0/1528214187 conn(0x7f7ecc0b0f00 msgr2=0x7f7ecc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:22.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed15bc700 1 -- 172.21.15.7:0/1528214187 shutdown_connections 2024-09-13T18:52:22.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.838+0000 7f7ed15bc700 1 -- 172.21.15.7:0/1528214187 wait complete. 2024-09-13T18:52:22.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ed15bc700 1 Processor -- start 2024-09-13T18:52:22.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ed15bc700 1 -- start start 2024-09-13T18:52:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ed15bc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc06aa90 0x7f7ecc139920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ed15bc700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ecc0b3ae0 0x7f7ecc14c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ed15bc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ecc159720 0x7f7ecc1514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ed15bc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc13c990 con 0x7f7ecc06aa90 2024-09-13T18:52:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ed15bc700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc150e70 con 0x7f7ecc159720 2024-09-13T18:52:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ed15bc700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc15ad80 con 0x7f7ecc0b3ae0 2024-09-13T18:52:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ecbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc06aa90 0x7f7ecc139920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ecbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc06aa90 0x7f7ecc139920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49252/0 (socket says 172.21.15.7:49252) 2024-09-13T18:52:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ecbfff700 1 -- 172.21.15.7:0/3256744200 learned_addr learned my addr 172.21.15.7:0/3256744200 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ed0dbb700 1 --2- 172.21.15.7:0/3256744200 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ecc159720 0x7f7ecc1514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ecb7fe700 1 --2- 172.21.15.7:0/3256744200 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ecc0b3ae0 0x7f7ecc14c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ecbfff700 1 -- 172.21.15.7:0/3256744200 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ecc0b3ae0 msgr2=0x7f7ecc14c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ecbfff700 1 --2- 172.21.15.7:0/3256744200 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ecc0b3ae0 0x7f7ecc14c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ecbfff700 1 -- 172.21.15.7:0/3256744200 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ecc159720 msgr2=0x7f7ecc1514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ecbfff700 1 --2- 172.21.15.7:0/3256744200 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ecc159720 0x7f7ecc1514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ecbfff700 1 -- 172.21.15.7:0/3256744200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec4009b30 con 0x7f7ecc06aa90 2024-09-13T18:52:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ecbfff700 1 --2- 172.21.15.7:0/3256744200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc06aa90 0x7f7ecc139920 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000e810 tx=0x7f7ec000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:22.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ec8ff9700 1 -- 172.21.15.7:0/3256744200 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0004230 con 0x7f7ecc06aa90 2024-09-13T18:52:22.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ec8ff9700 1 -- 172.21.15.7:0/3256744200 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec0004390 con 0x7f7ecc06aa90 2024-09-13T18:52:22.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ed15bc700 1 -- 172.21.15.7:0/3256744200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ecc15b040 con 0x7f7ecc06aa90 2024-09-13T18:52:22.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.842+0000 7f7ec8ff9700 1 -- 172.21.15.7:0/3256744200 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec00036c0 con 0x7f7ecc06aa90 2024-09-13T18:52:22.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.846+0000 7f7ed15bc700 1 -- 172.21.15.7:0/3256744200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ecc15b600 con 0x7f7ecc06aa90 2024-09-13T18:52:22.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.846+0000 7f7ed15bc700 1 -- 172.21.15.7:0/3256744200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ecc0ccac0 con 0x7f7ecc06aa90 2024-09-13T18:52:22.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.846+0000 7f7ec8ff9700 1 -- 172.21.15.7:0/3256744200 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7ec0003840 con 0x7f7ecc06aa90 2024-09-13T18:52:22.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.846+0000 7f7ec8ff9700 1 --2- 172.21.15.7:0/3256744200 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7eb8066730 0x7f7eb8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:22.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.846+0000 7f7ec8ff9700 1 -- 172.21.15.7:0/3256744200 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ec0083b70 con 0x7f7ecc06aa90 2024-09-13T18:52:22.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.846+0000 7f7ecb7fe700 1 --2- 172.21.15.7:0/3256744200 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7eb8066730 0x7f7eb8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:22.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.850+0000 7f7ecb7fe700 1 --2- 172.21.15.7:0/3256744200 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7eb8066730 0x7f7eb8068bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc1f5ec0 tx=0x7f7ebc007f30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:22.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:22.854+0000 7f7ec8ff9700 1 -- 172.21.15.7:0/3256744200 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7ec0050dc0 con 0x7f7ecc06aa90 2024-09-13T18:52:23.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.030+0000 7f7ed15bc700 1 -- 172.21.15.7:0/3256744200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ecc0cb550 con 0x7f7ecc06aa90 2024-09-13T18:52:23.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.034+0000 7f7ec8ff9700 1 -- 172.21.15.7:0/3256744200 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f7ec000c450 con 0x7f7ecc06aa90 2024-09-13T18:52:23.040 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:23.040 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:23.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.038+0000 7f7ed15bc700 1 -- 172.21.15.7:0/3256744200 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7eb8066730 msgr2=0x7f7eb8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.038+0000 7f7ed15bc700 1 --2- 172.21.15.7:0/3256744200 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7eb8066730 0x7f7eb8068bf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc1f5ec0 tx=0x7f7ebc007f30 comp rx=0 tx=0).stop 2024-09-13T18:52:23.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.038+0000 7f7ed15bc700 1 -- 172.21.15.7:0/3256744200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc06aa90 msgr2=0x7f7ecc139920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.038+0000 7f7ed15bc700 1 --2- 172.21.15.7:0/3256744200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc06aa90 0x7f7ecc139920 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000e810 tx=0x7f7ec000eb20 comp rx=0 tx=0).stop 2024-09-13T18:52:23.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.038+0000 7f7ed15bc700 1 -- 172.21.15.7:0/3256744200 shutdown_connections 2024-09-13T18:52:23.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.038+0000 7f7ed15bc700 1 --2- 172.21.15.7:0/3256744200 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7eb8066730 0x7f7eb8068bf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.038+0000 7f7ed15bc700 1 --2- 172.21.15.7:0/3256744200 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ecc159720 0x7f7ecc1514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.038+0000 7f7ed15bc700 1 --2- 172.21.15.7:0/3256744200 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ecc0b3ae0 0x7f7ecc14c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.038+0000 7f7ed15bc700 1 --2- 172.21.15.7:0/3256744200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ecc06aa90 0x7f7ecc139920 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.038+0000 7f7ed15bc700 1 -- 172.21.15.7:0/3256744200 >> 172.21.15.7:0/3256744200 conn(0x7f7ecc0b0f00 msgr2=0x7f7ecc0afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.038+0000 7f7ed15bc700 1 -- 172.21.15.7:0/3256744200 shutdown_connections 2024-09-13T18:52:23.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.038+0000 7f7ed15bc700 1 -- 172.21.15.7:0/3256744200 wait complete. 2024-09-13T18:52:23.055 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T18:52:23.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:23.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a9759e700 1 Processor -- start 2024-09-13T18:52:23.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a9759e700 1 -- start start 2024-09-13T18:52:23.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a9759e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4a980b3ae0 0x7f4a98165590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a9759e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a98071f30 con 0x7f4a98160eb0 2024-09-13T18:52:23.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a9759e700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a981622e0 con 0x7f4a98161290 2024-09-13T18:52:23.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a9759e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a98162000 con 0x7f4a980b3ae0 2024-09-13T18:52:23.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a9659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4a98160eb0 0x7f4a980b33d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35724/0 (socket says 172.21.15.7:35724) 2024-09-13T18:52:23.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a9659c700 1 -- 172.21.15.7:0/2035500028 learned_addr learned my addr 172.21.15.7:0/2035500028 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:23.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a8dd9b700 1 --2- 172.21.15.7:0/2035500028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4a980b3ae0 0x7f4a98165590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 -- 172.21.15.7:0/2035500028 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3654597575 0 0) 0x7f4a98071f30 con 0x7f4a98160eb0 2024-09-13T18:52:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 -- 172.21.15.7:0/2035500028 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a840035d0 con 0x7f4a98160eb0 2024-09-13T18:52:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 -- 172.21.15.7:0/2035500028 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980608655 0 0) 0x7f4a981622e0 con 0x7f4a98161290 2024-09-13T18:52:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 -- 172.21.15.7:0/2035500028 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a98071f30 con 0x7f4a98161290 2024-09-13T18:52:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 -- 172.21.15.7:0/2035500028 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1281436101 0 0) 0x7f4a840035d0 con 0x7f4a98160eb0 2024-09-13T18:52:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 -- 172.21.15.7:0/2035500028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4a980b3ae0 msgr2=0x7f4a98165590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 --2- 172.21.15.7:0/2035500028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4a980b3ae0 0x7f4a98165590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 -- 172.21.15.7:0/2035500028 >> v1:172.21.15.171:6789/0 conn(0x7f4a98161290 legacy=0x7f4a98161740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 -- 172.21.15.7:0/2035500028 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a98164ab0 con 0x7f4a98160eb0 2024-09-13T18:52:23.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 -- 172.21.15.7:0/2035500028 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7f4a88001670 con 0x7f4a98160eb0 2024-09-13T18:52:23.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 -- 172.21.15.7:0/2035500028 >> v1:172.21.15.7:6789/0 conn(0x7f4a98160eb0 legacy=0x7f4a980b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 --2- 172.21.15.7:0/2035500028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4a980b3ae0 0x7f4a84003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 --2- 172.21.15.7:0/2035500028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a84004140 0x7f4a84004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 --2- 172.21.15.7:0/2035500028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a84004db0 0x7f4a84005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 -- 172.21.15.7:0/2035500028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a98162000 con 0x7f4a84004db0 2024-09-13T18:52:23.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 -- 172.21.15.7:0/2035500028 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8400bb10 con 0x7f4a84004140 2024-09-13T18:52:23.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 -- 172.21.15.7:0/2035500028 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8400bcd0 con 0x7f4a980b3ae0 2024-09-13T18:52:23.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a96d9d700 1 --2- 172.21.15.7:0/2035500028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a84004db0 0x7f4a84005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a8dd9b700 1 --2- 172.21.15.7:0/2035500028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a84004140 0x7f4a84004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a9659c700 1 --2- 172.21.15.7:0/2035500028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4a980b3ae0 0x7f4a84003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a96d9d700 1 -- 172.21.15.7:0/2035500028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4a980b3ae0 msgr2=0x7f4a84003be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:23.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a96d9d700 1 --2- 172.21.15.7:0/2035500028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4a980b3ae0 0x7f4a84003be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a96d9d700 1 -- 172.21.15.7:0/2035500028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a84004140 msgr2=0x7f4a84004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:23.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a96d9d700 1 --2- 172.21.15.7:0/2035500028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a84004140 0x7f4a84004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a96d9d700 1 -- 172.21.15.7:0/2035500028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a840035d0 con 0x7f4a84004db0 2024-09-13T18:52:23.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a8dd9b700 1 --2- 172.21.15.7:0/2035500028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a84004140 0x7f4a84004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:23.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a96d9d700 1 --2- 172.21.15.7:0/2035500028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a84004db0 0x7f4a84005230 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000c2f0 tx=0x7f4a840058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f23f51846d5932 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 -- 172.21.15.7:0/2035500028 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90001f30 con 0x7f4a84004db0 2024-09-13T18:52:23.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 -- 172.21.15.7:0/2035500028 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9000ce90 con 0x7f4a84004db0 2024-09-13T18:52:23.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.162+0000 7f4a95d9b700 1 -- 172.21.15.7:0/2035500028 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90003c60 con 0x7f4a84004db0 2024-09-13T18:52:23.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.166+0000 7f4a9759e700 1 -- 172.21.15.7:0/2035500028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a84004db0 msgr2=0x7f4a84005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.166+0000 7f4a9759e700 1 --2- 172.21.15.7:0/2035500028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a84004db0 0x7f4a84005230 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000c2f0 tx=0x7f4a840058a0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.166+0000 7f4a96d9d700 1 -- 172.21.15.7:0/2035500028 reap_dead start 2024-09-13T18:52:23.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.166+0000 7f4a9759e700 1 -- 172.21.15.7:0/2035500028 shutdown_connections 2024-09-13T18:52:23.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.166+0000 7f4a9759e700 1 -- 172.21.15.7:0/2035500028 >> 172.21.15.7:0/2035500028 conn(0x7f4a980b0f00 msgr2=0x7f4a980b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.166+0000 7f4a9759e700 1 -- 172.21.15.7:0/2035500028 shutdown_connections 2024-09-13T18:52:23.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.166+0000 7f4a9759e700 1 -- 172.21.15.7:0/2035500028 wait complete. 2024-09-13T18:52:23.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.166+0000 7f4a9759e700 1 Processor -- start 2024-09-13T18:52:23.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.166+0000 7f4a9759e700 1 -- start start 2024-09-13T18:52:23.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.166+0000 7f4a9759e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a84004db0 0x7f4a98160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.166+0000 7f4a9759e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4a98156400 0x7f4a9814b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.166+0000 7f4a9759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a980af5e0 0x7f4a980afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.166+0000 7f4a9759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a980affa0 con 0x7f4a980af5e0 2024-09-13T18:52:23.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.166+0000 7f4a9759e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9815f230 con 0x7f4a84004db0 2024-09-13T18:52:23.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.166+0000 7f4a9759e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9815e620 con 0x7f4a98156400 2024-09-13T18:52:23.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.170+0000 7f4a96d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a980af5e0 0x7f4a980afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.170+0000 7f4a96d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a980af5e0 0x7f4a980afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49270/0 (socket says 172.21.15.7:49270) 2024-09-13T18:52:23.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.170+0000 7f4a96d9d700 1 -- 172.21.15.7:0/4238549704 learned_addr learned my addr 172.21.15.7:0/4238549704 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.170+0000 7f4a8dd9b700 1 --2- 172.21.15.7:0/4238549704 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4a98156400 0x7f4a9814b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.170+0000 7f4a9659c700 1 --2- 172.21.15.7:0/4238549704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a84004db0 0x7f4a98160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.170+0000 7f4a96d9d700 1 -- 172.21.15.7:0/4238549704 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4a98156400 msgr2=0x7f4a9814b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.170+0000 7f4a96d9d700 1 --2- 172.21.15.7:0/4238549704 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4a98156400 0x7f4a9814b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.170+0000 7f4a96d9d700 1 -- 172.21.15.7:0/4238549704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a84004db0 msgr2=0x7f4a98160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.170+0000 7f4a96d9d700 1 --2- 172.21.15.7:0/4238549704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a84004db0 0x7f4a98160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.170+0000 7f4a96d9d700 1 -- 172.21.15.7:0/4238549704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9815e900 con 0x7f4a980af5e0 2024-09-13T18:52:23.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.170+0000 7f4a8dd9b700 1 --2- 172.21.15.7:0/4238549704 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4a98156400 0x7f4a9814b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:23.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.170+0000 7f4a96d9d700 1 --2- 172.21.15.7:0/4238549704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a980af5e0 0x7f4a980afa60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4a90004000 tx=0x7f4a90001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.170+0000 7f4a8f7fe700 1 -- 172.21.15.7:0/4238549704 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9000cc90 con 0x7f4a980af5e0 2024-09-13T18:52:23.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.170+0000 7f4a8f7fe700 1 -- 172.21.15.7:0/4238549704 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a90003e00 con 0x7f4a980af5e0 2024-09-13T18:52:23.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.170+0000 7f4a8f7fe700 1 -- 172.21.15.7:0/4238549704 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a90005520 con 0x7f4a980af5e0 2024-09-13T18:52:23.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.170+0000 7f4a9759e700 1 -- 172.21.15.7:0/4238549704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9815ebe0 con 0x7f4a980af5e0 2024-09-13T18:52:23.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.170+0000 7f4a9759e700 1 -- 172.21.15.7:0/4238549704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9815f6c0 con 0x7f4a980af5e0 2024-09-13T18:52:23.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.170+0000 7f4a9759e700 1 -- 172.21.15.7:0/4238549704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a980b33d0 con 0x7f4a980af5e0 2024-09-13T18:52:23.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.174+0000 7f4a8f7fe700 1 -- 172.21.15.7:0/4238549704 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4a90019070 con 0x7f4a980af5e0 2024-09-13T18:52:23.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.174+0000 7f4a8f7fe700 1 --2- 172.21.15.7:0/4238549704 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4a78066730 0x7f4a78068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.174+0000 7f4a8f7fe700 1 -- 172.21.15.7:0/4238549704 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a90084e70 con 0x7f4a980af5e0 2024-09-13T18:52:23.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.174+0000 7f4a9659c700 1 --2- 172.21.15.7:0/4238549704 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4a78066730 0x7f4a78068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.174+0000 7f4a9659c700 1 --2- 172.21.15.7:0/4238549704 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4a78066730 0x7f4a78068bf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4a88005b20 tx=0x7f4a88009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.182+0000 7f4a8f7fe700 1 -- 172.21.15.7:0/4238549704 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4a90052270 con 0x7f4a980af5e0 2024-09-13T18:52:23.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.366+0000 7f4a9759e700 1 -- 172.21.15.7:0/4238549704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a980b2e40 con 0x7f4a980af5e0 2024-09-13T18:52:23.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.366+0000 7f4a8f7fe700 1 -- 172.21.15.7:0/4238549704 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f4a900011c0 con 0x7f4a980af5e0 2024-09-13T18:52:23.373 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:23.373 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:23.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.370+0000 7f4a9759e700 1 -- 172.21.15.7:0/4238549704 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4a78066730 msgr2=0x7f4a78068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.370+0000 7f4a9759e700 1 --2- 172.21.15.7:0/4238549704 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4a78066730 0x7f4a78068bf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4a88005b20 tx=0x7f4a88009040 comp rx=0 tx=0).stop 2024-09-13T18:52:23.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.370+0000 7f4a9759e700 1 -- 172.21.15.7:0/4238549704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a980af5e0 msgr2=0x7f4a980afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.370+0000 7f4a9759e700 1 --2- 172.21.15.7:0/4238549704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a980af5e0 0x7f4a980afa60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4a90004000 tx=0x7f4a90001650 comp rx=0 tx=0).stop 2024-09-13T18:52:23.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.370+0000 7f4a9759e700 1 -- 172.21.15.7:0/4238549704 shutdown_connections 2024-09-13T18:52:23.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.370+0000 7f4a9759e700 1 --2- 172.21.15.7:0/4238549704 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4a78066730 0x7f4a78068bf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.370+0000 7f4a9759e700 1 --2- 172.21.15.7:0/4238549704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a980af5e0 0x7f4a980afa60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.370+0000 7f4a9759e700 1 --2- 172.21.15.7:0/4238549704 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4a98156400 0x7f4a9814b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.370+0000 7f4a9759e700 1 --2- 172.21.15.7:0/4238549704 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a84004db0 0x7f4a98160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.370+0000 7f4a9759e700 1 -- 172.21.15.7:0/4238549704 >> 172.21.15.7:0/4238549704 conn(0x7f4a980b0f00 msgr2=0x7f4a981639e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.370+0000 7f4a9759e700 1 -- 172.21.15.7:0/4238549704 shutdown_connections 2024-09-13T18:52:23.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.370+0000 7f4a9759e700 1 -- 172.21.15.7:0/4238549704 wait complete. 2024-09-13T18:52:23.390 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T18:52:23.390 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T18:52:23.390 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T18:52:23.391 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T18:52:23.391 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T18:52:23.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T18:52:23.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T18:52:23.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T18:52:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fc18f59e700 1 Processor -- start 2024-09-13T18:52:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fc18f59e700 1 -- start start 2024-09-13T18:52:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fc18f59e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fc190165160 0x7fc1900704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fc18f59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc190070a00 0x7fc190163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fc18f59e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc190162570 con 0x7fc19006d8c0 2024-09-13T18:52:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fc18f59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc190162e60 con 0x7fc190070a00 2024-09-13T18:52:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fc18f59e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc190071240 con 0x7fc190165160 2024-09-13T18:52:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fb50b59e700 1 Processor -- start 2024-09-13T18:52:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fb50b59e700 1 -- start start 2024-09-13T18:52:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fb50b59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb50c1422b0 0x7fb50c0b3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fb50b59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb50c15ac20 0x7fb50c156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fb50b59e700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb50c0b3c10 con 0x7fb50c0b37b0 2024-09-13T18:52:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fc18e59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc19006d8c0 0x7fc190164a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35736/0 (socket says 172.21.15.7:35736) 2024-09-13T18:52:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fc18e59c700 1 -- 172.21.15.7:0/4260422756 learned_addr learned my addr 172.21.15.7:0/4260422756 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fc18ed9d700 1 --2- 172.21.15.7:0/4260422756 >> v2:172.21.15.7:3300/0 conn(0x7fc190070a00 0x7fc190163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fb50b59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50c15be50 con 0x7fb50c15ac20 2024-09-13T18:52:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fb50b59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c15b240 con 0x7fb50c1422b0 2024-09-13T18:52:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7f525a764700 1 Processor -- start 2024-09-13T18:52:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7f525a764700 1 -- start start 2024-09-13T18:52:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fc18dd9b700 1 --2- 172.21.15.7:0/4260422756 >> v2:172.21.15.171:3300/0 conn(0x7fc190165160 0x7fc1900704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fb50ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb50c15ac20 0x7fb50c156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fc18d59a700 1 -- 172.21.15.7:0/4260422756 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025575521 0 0) 0x7fc190162570 con 0x7fc19006d8c0 2024-09-13T18:52:23.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fc18d59a700 1 -- 172.21.15.7:0/4260422756 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1780035d0 con 0x7fc19006d8c0 2024-09-13T18:52:23.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fb50ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb50c15ac20 0x7fb50c156cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49286/0 (socket says 172.21.15.7:49286) 2024-09-13T18:52:23.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fb50ad9d700 1 -- 172.21.15.7:0/1550413360 learned_addr learned my addr 172.21.15.7:0/1550413360 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fc18ed9d700 1 -- 172.21.15.7:0/4260422756 >> v2:172.21.15.171:3300/0 conn(0x7fc190165160 msgr2=0x7fc1900704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:23.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fc18ed9d700 1 --2- 172.21.15.7:0/4260422756 >> v2:172.21.15.171:3300/0 conn(0x7fc190165160 0x7fc1900704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fc18ed9d700 1 -- 172.21.15.7:0/4260422756 >> v1:172.21.15.7:6789/0 conn(0x7fc19006d8c0 legacy=0x7fc190164a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fc18ed9d700 1 -- 172.21.15.7:0/4260422756 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1901627a0 con 0x7fc190070a00 2024-09-13T18:52:23.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fc18dd9b700 1 --2- 172.21.15.7:0/4260422756 >> v2:172.21.15.171:3300/0 conn(0x7fc190165160 0x7fc1900704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:23.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fb50ad9d700 1 -- 172.21.15.7:0/1550413360 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb50c1422b0 msgr2=0x7fb50c0b3270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:52:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fc18ed9d700 1 --2- 172.21.15.7:0/4260422756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc190070a00 0x7fc190163280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc190158550 tx=0x7fc184009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d591aa7947a904 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fb50ad9d700 1 --2- 172.21.15.7:0/1550413360 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb50c1422b0 0x7fb50c0b3270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fb50ad9d700 1 -- 172.21.15.7:0/1550413360 >> v1:172.21.15.171:6789/0 conn(0x7fb50c0b37b0 legacy=0x7fb50c15a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f525a764700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52540d0880 0x7f52540b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.538+0000 7fb50ad9d700 1 -- 172.21.15.7:0/1550413360 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c15b520 con 0x7fb50c15ac20 2024-09-13T18:52:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fb50ad9d700 1 --2- 172.21.15.7:0/1550413360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb50c15ac20 0x7fb50c156cf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb504010520 tx=0x7fb504010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6820515022f5242d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f525a764700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f52540ce260 0x7f52540ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fc18d59a700 1 -- 172.21.15.7:0/4260422756 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184004230 con 0x7fc190070a00 2024-09-13T18:52:23.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fc18d59a700 1 -- 172.21.15.7:0/4260422756 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc184004390 con 0x7fc190070a00 2024-09-13T18:52:23.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fc18d59a700 1 -- 172.21.15.7:0/4260422756 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1840036c0 con 0x7fc190070a00 2024-09-13T18:52:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f525a764700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52540cebf0 0x7f52540d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f525a764700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52540d2330 con 0x7f52540d0880 2024-09-13T18:52:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f525a764700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5254164920 con 0x7f52540ce260 2024-09-13T18:52:23.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f525a764700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525406ca00 con 0x7f52540cebf0 2024-09-13T18:52:23.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f5259762700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52540d0880 0x7f52540b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f5259762700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52540d0880 0x7f52540b5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49292/0 (socket says 172.21.15.7:49292) 2024-09-13T18:52:23.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f5259762700 1 -- 172.21.15.7:0/1140944868 learned_addr learned my addr 172.21.15.7:0/1140944868 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f5258f61700 1 --2- 172.21.15.7:0/1140944868 >> v2:172.21.15.171:3300/0 conn(0x7f52540ce260 0x7f52540ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f5259762700 1 -- 172.21.15.7:0/1140944868 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52540cebf0 msgr2=0x7f52540d0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:23.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f5259762700 1 --2- 172.21.15.7:0/1140944868 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52540cebf0 0x7f52540d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f5259762700 1 -- 172.21.15.7:0/1140944868 >> v2:172.21.15.171:3300/0 conn(0x7f52540ce260 msgr2=0x7f52540ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:23.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f5259762700 1 --2- 172.21.15.7:0/1140944868 >> v2:172.21.15.171:3300/0 conn(0x7f52540ce260 0x7f52540ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f5259762700 1 -- 172.21.15.7:0/1140944868 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52540cd4a0 con 0x7f52540d0880 2024-09-13T18:52:23.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f5258f61700 1 --2- 172.21.15.7:0/1140944868 >> v2:172.21.15.171:3300/0 conn(0x7f52540ce260 0x7f52540ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:23.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f5259762700 1 --2- 172.21.15.7:0/1140944868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52540d0880 0x7f52540b5b00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f525406c540 tx=0x7f5250009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa3fe97feb2c049 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fb50959a700 1 -- 172.21.15.7:0/1550413360 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50400a2e0 con 0x7fb50c15ac20 2024-09-13T18:52:23.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fb50959a700 1 -- 172.21.15.7:0/1550413360 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50400a440 con 0x7fb50c15ac20 2024-09-13T18:52:23.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fb50959a700 1 -- 172.21.15.7:0/1550413360 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50400a7f0 con 0x7fb50c15ac20 2024-09-13T18:52:23.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fc18f59e700 1 -- 172.21.15.7:0/4260422756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc190070a00 msgr2=0x7fc190163280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fc18f59e700 1 --2- 172.21.15.7:0/4260422756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc190070a00 0x7fc190163280 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc190158550 tx=0x7fc184009b30 comp rx=0 tx=0).stop 2024-09-13T18:52:23.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fc18f59e700 1 -- 172.21.15.7:0/4260422756 shutdown_connections 2024-09-13T18:52:23.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fc18f59e700 1 --2- 172.21.15.7:0/4260422756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc190070a00 0x7fc190163280 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fc18f59e700 1 --2- 172.21.15.7:0/4260422756 >> v2:172.21.15.171:3300/0 conn(0x7fc190165160 0x7fc1900704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fc18f59e700 1 -- 172.21.15.7:0/4260422756 >> 172.21.15.7:0/4260422756 conn(0x7fc1900b0e10 msgr2=0x7fc1900b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f524bfff700 1 -- 172.21.15.7:0/1140944868 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525000b8a0 con 0x7f52540d0880 2024-09-13T18:52:23.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f524bfff700 1 -- 172.21.15.7:0/1140944868 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525000ba00 con 0x7f52540d0880 2024-09-13T18:52:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fc18f59e700 1 -- 172.21.15.7:0/4260422756 shutdown_connections 2024-09-13T18:52:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fc18f59e700 1 -- 172.21.15.7:0/4260422756 wait complete. 2024-09-13T18:52:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fb50b59e700 1 -- 172.21.15.7:0/1550413360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb50c15ac20 msgr2=0x7fb50c156cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fb50b59e700 1 --2- 172.21.15.7:0/1550413360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb50c15ac20 0x7fb50c156cf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb504010520 tx=0x7fb504010830 comp rx=0 tx=0).stop 2024-09-13T18:52:23.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f525a764700 1 -- 172.21.15.7:0/1140944868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52540d0880 msgr2=0x7f52540b5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7f525a764700 1 --2- 172.21.15.7:0/1140944868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52540d0880 0x7f52540b5b00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f525406c540 tx=0x7f5250009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fc18f59e700 1 Processor -- start 2024-09-13T18:52:23.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.542+0000 7fc18f59e700 1 -- start start 2024-09-13T18:52:23.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7f525a764700 1 -- 172.21.15.7:0/1140944868 shutdown_connections 2024-09-13T18:52:23.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7f525a764700 1 --2- 172.21.15.7:0/1140944868 >> v2:172.21.15.171:3300/0 conn(0x7f52540ce260 0x7f52540ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7f525a764700 1 --2- 172.21.15.7:0/1140944868 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52540cebf0 0x7f52540d0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7f525a764700 1 --2- 172.21.15.7:0/1140944868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52540d0880 0x7f52540b5b00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7f525a764700 1 -- 172.21.15.7:0/1140944868 >> 172.21.15.7:0/1140944868 conn(0x7f52540d2cb0 msgr2=0x7f52540d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7f525a764700 1 -- 172.21.15.7:0/1140944868 shutdown_connections 2024-09-13T18:52:23.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fb50b59e700 1 -- 172.21.15.7:0/1550413360 shutdown_connections 2024-09-13T18:52:23.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fb50b59e700 1 --2- 172.21.15.7:0/1550413360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb50c15ac20 0x7fb50c156cf0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fb50b59e700 1 --2- 172.21.15.7:0/1550413360 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb50c1422b0 0x7fb50c0b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fb50b59e700 1 -- 172.21.15.7:0/1550413360 >> 172.21.15.7:0/1550413360 conn(0x7fb50c0b0e10 msgr2=0x7fb50c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fb50b59e700 1 -- 172.21.15.7:0/1550413360 shutdown_connections 2024-09-13T18:52:23.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fb50b59e700 1 -- 172.21.15.7:0/1550413360 wait complete. 2024-09-13T18:52:23.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc18f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc19006d8c0 0x7fc190149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc18f59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc190070a00 0x7fc19013ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc18e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc19006d8c0 0x7fc190149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc18e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc19006d8c0 0x7fc190149d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49294/0 (socket says 172.21.15.7:49294) 2024-09-13T18:52:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc18e59c700 1 -- 172.21.15.7:0/4080973988 learned_addr learned my addr 172.21.15.7:0/4080973988 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc18f59e700 1 --2- 172.21.15.7:0/4080973988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc190165160 0x7fc19015d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc18f59e700 1 -- 172.21.15.7:0/4080973988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19014a260 con 0x7fc19006d8c0 2024-09-13T18:52:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc18f59e700 1 -- 172.21.15.7:0/4080973988 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19013f250 con 0x7fc190165160 2024-09-13T18:52:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc18f59e700 1 -- 172.21.15.7:0/4080973988 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19014a4a0 con 0x7fc190070a00 2024-09-13T18:52:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc18dd9b700 1 --2- 172.21.15.7:0/4080973988 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc190070a00 0x7fc19013ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc18e59c700 1 -- 172.21.15.7:0/4080973988 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc190070a00 msgr2=0x7fc19013ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc18e59c700 1 --2- 172.21.15.7:0/4080973988 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc190070a00 0x7fc19013ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc18e59c700 1 -- 172.21.15.7:0/4080973988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc190165160 msgr2=0x7fc19015d070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:52:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc18e59c700 1 --2- 172.21.15.7:0/4080973988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc190165160 0x7fc19015d070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc18e59c700 1 -- 172.21.15.7:0/4080973988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc188009a70 con 0x7fc19006d8c0 2024-09-13T18:52:23.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fb50b59e700 1 Processor -- start 2024-09-13T18:52:23.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc18e59c700 1 --2- 172.21.15.7:0/4080973988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc19006d8c0 0x7fc190149d20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc18000c580 tx=0x7fc18000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fa27a155700 1 Processor -- start 2024-09-13T18:52:23.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fa27a155700 1 -- start start 2024-09-13T18:52:23.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fbb5759e700 1 Processor -- start 2024-09-13T18:52:23.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fbb5759e700 1 -- start start 2024-09-13T18:52:23.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc17effd700 1 -- 172.21.15.7:0/4080973988 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180004ca0 con 0x7fc19006d8c0 2024-09-13T18:52:23.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc17effd700 1 -- 172.21.15.7:0/4080973988 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc180004e20 con 0x7fc19006d8c0 2024-09-13T18:52:23.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc17effd700 1 -- 172.21.15.7:0/4080973988 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180007310 con 0x7fc19006d8c0 2024-09-13T18:52:23.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc18f59e700 1 -- 172.21.15.7:0/4080973988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1840094b0 con 0x7fc19006d8c0 2024-09-13T18:52:23.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7f818bfff700 1 Processor -- start 2024-09-13T18:52:23.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fc18f59e700 1 -- 172.21.15.7:0/4080973988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc190071010 con 0x7fc19006d8c0 2024-09-13T18:52:23.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7f818bfff700 1 -- start start 2024-09-13T18:52:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fbb5759e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb5806e080 0x7fbb5806b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fbb5759e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fbb580700e0 0x7fbb58070550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fbb5759e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5806bfd0 con 0x7fbb5806fd00 2024-09-13T18:52:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fbb5759e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb58169b90 con 0x7fbb580700e0 2024-09-13T18:52:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fbb5759e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5806c2b0 con 0x7fbb5806e080 2024-09-13T18:52:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fbb5659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbb5806fd00 0x7fbb580ce0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35744/0 (socket says 172.21.15.7:35744) 2024-09-13T18:52:23.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fbb5659c700 1 -- 172.21.15.7:0/1624153471 learned_addr learned my addr 172.21.15.7:0/1624153471 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:23.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fa27a155700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa2741464d0 0x7fa274070540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fa27a155700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2740cc0e0 0x7fa2740cc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fa27a155700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2740cca80 con 0x7fa2741606e0 2024-09-13T18:52:23.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fbb55d9b700 1 --2- 172.21.15.7:0/1624153471 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb5806e080 0x7fbb5806b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fbb56d9d700 1 --2- 172.21.15.7:0/1624153471 >> v2:172.21.15.171:3300/0 conn(0x7fbb580700e0 0x7fbb58070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fa27a155700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa274071fd0 con 0x7fa2741464d0 2024-09-13T18:52:23.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fa27a155700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa274070f70 con 0x7fa2740cc0e0 2024-09-13T18:52:23.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fb50b59e700 1 -- start start 2024-09-13T18:52:23.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7fa279153700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa2741464d0 0x7fa274070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa279153700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa2741464d0 0x7fa274070540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:40718/0 (socket says 172.21.15.7:40718) 2024-09-13T18:52:23.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa279153700 1 -- 172.21.15.7:0/2574855222 learned_addr learned my addr 172.21.15.7:0/2574855222 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa278952700 1 --2- 172.21.15.7:0/2574855222 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2740cc0e0 0x7fa2740cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818bfff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f81840d96f0 0x7f818404f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818bfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f818401cf40 0x7f81840df550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f81840dfa90 0x7f818404fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818bfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81840d9ad0 con 0x7f81840dfa90 2024-09-13T18:52:23.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818bfff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81840e04f0 con 0x7f81840d96f0 2024-09-13T18:52:23.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818bfff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818404f0b0 con 0x7f818401cf40 2024-09-13T18:52:23.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fc18f59e700 1 -- 172.21.15.7:0/4080973988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc190070e90 con 0x7fc19006d8c0 2024-09-13T18:52:23.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fbb56d9d700 1 -- 172.21.15.7:0/1624153471 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb5806e080 msgr2=0x7fbb5806b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:23.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fbb56d9d700 1 --2- 172.21.15.7:0/1624153471 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb5806e080 0x7fbb5806b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fbb56d9d700 1 -- 172.21.15.7:0/1624153471 >> v1:172.21.15.7:6789/0 conn(0x7fbb5806fd00 legacy=0x7fbb580ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fbb56d9d700 1 -- 172.21.15.7:0/1624153471 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5806c590 con 0x7fbb580700e0 2024-09-13T18:52:23.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818affd700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f81840d96f0 0x7f818404f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818a7fc700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f818401cf40 0x7f81840df550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f81840dfa90 0x7f818404fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fc17effd700 1 -- 172.21.15.7:0/4080973988 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc180007470 con 0x7fc19006d8c0 2024-09-13T18:52:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f81840dfa90 0x7f818404fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49304/0 (socket says 172.21.15.7:49304) 2024-09-13T18:52:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818affd700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f81840d96f0 0x7f818404f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:40724/0 (socket says 172.21.15.7:40724) 2024-09-13T18:52:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818b7fe700 1 -- 172.21.15.7:0/1701148430 learned_addr learned my addr 172.21.15.7:0/1701148430 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa278952700 1 -- 172.21.15.7:0/2574855222 >> v2:172.21.15.171:3300/0 conn(0x7fa2741464d0 msgr2=0x7fa274070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa278952700 1 --2- 172.21.15.7:0/2574855222 >> v2:172.21.15.171:3300/0 conn(0x7fa2741464d0 0x7fa274070540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa278952700 1 -- 172.21.15.7:0/2574855222 >> v1:172.21.15.7:6789/0 conn(0x7fa2741606e0 legacy=0x7fa274160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:23.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa278952700 1 -- 172.21.15.7:0/2574855222 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa274071210 con 0x7fa2740cc0e0 2024-09-13T18:52:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818b7fe700 1 -- 172.21.15.7:0/1701148430 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f818401cf40 msgr2=0x7f81840df550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fbb56d9d700 1 --2- 172.21.15.7:0/1624153471 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb580700e0 0x7fbb58070550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbb400062a0 tx=0x7fbb40007b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a1babb9b4e134cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818b7fe700 1 --2- 172.21.15.7:0/1701148430 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f818401cf40 0x7f81840df550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818b7fe700 1 -- 172.21.15.7:0/1701148430 >> v2:172.21.15.171:3300/0 conn(0x7f81840d96f0 msgr2=0x7f818404f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818b7fe700 1 --2- 172.21.15.7:0/1701148430 >> v2:172.21.15.171:3300/0 conn(0x7f81840d96f0 0x7f818404f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818b7fe700 1 -- 172.21.15.7:0/1701148430 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81840dff30 con 0x7f81840dfa90 2024-09-13T18:52:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fc17effd700 1 --2- 172.21.15.7:0/4080973988 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc170066730 0x7fc170068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa278952700 1 --2- 172.21.15.7:0/2574855222 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2740cc0e0 0x7fa2740cc540 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa264009850 tx=0x7fa264009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f7417883f9a6b58 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fb50b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb50c0b37b0 0x7fb50c13b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fb50b59e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb50c1422b0 0x7fb50c156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fb50b59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb50c15ac20 0x7fb50c13a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fb50b59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c1ef290 con 0x7fb50c0b37b0 2024-09-13T18:52:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fb50b59e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c1ee710 con 0x7fb50c1422b0 2024-09-13T18:52:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fb50b59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c1ee9f0 con 0x7fb50c15ac20 2024-09-13T18:52:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fc17effd700 1 -- 172.21.15.7:0/4080973988 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc180083fb0 con 0x7fc19006d8c0 2024-09-13T18:52:23.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fbb5559a700 1 -- 172.21.15.7:0/1624153471 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40010070 con 0x7fbb580700e0 2024-09-13T18:52:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fc17effd700 1 --2- 172.21.15.7:0/4080973988 >> [v2:172.21.15.171:6817/2886154603,v1:172.21.15.171:6819/2886154603] conn(0x7fc17006e8d0 0x7fc170070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fc17effd700 1 -- 172.21.15.7:0/4080973988 --> [v2:172.21.15.171:6817/2886154603,v1:172.21.15.171:6819/2886154603] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1700713a0 con 0x7fc17006e8d0 2024-09-13T18:52:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fc17effd700 1 -- 172.21.15.7:0/4080973988 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc180086610 con 0x7fc19006d8c0 2024-09-13T18:52:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818b7fe700 1 --2- 172.21.15.7:0/1701148430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840dfa90 0x7f818404fec0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f81840080c0 tx=0x7f817c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfe0546b4214a4d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fb509d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb50c1422b0 0x7fb50c156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa26bfff700 1 -- 172.21.15.7:0/2574855222 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa264004230 con 0x7fa2740cc0e0 2024-09-13T18:52:23.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fc18dd9b700 1 --2- 172.21.15.7:0/4080973988 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc170066730 0x7fc170068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fb50ad9d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb50c15ac20 0x7fb50c13a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa26bfff700 1 -- 172.21.15.7:0/2574855222 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa264004390 con 0x7fa2740cc0e0 2024-09-13T18:52:23.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa26bfff700 1 -- 172.21.15.7:0/2574855222 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2640036c0 con 0x7fa2740cc0e0 2024-09-13T18:52:23.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fbb5559a700 1 -- 172.21.15.7:0/1624153471 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4000b2e0 con 0x7fbb580700e0 2024-09-13T18:52:23.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fbb5559a700 1 -- 172.21.15.7:0/1624153471 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4000b940 con 0x7fbb580700e0 2024-09-13T18:52:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fb50ad9d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb50c15ac20 0x7fb50c13a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:52930/0 (socket says 172.21.15.7:52930) 2024-09-13T18:52:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fb50ad9d700 1 -- 172.21.15.7:0/1422945066 learned_addr learned my addr 172.21.15.7:0/1422945066 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fb50a59c700 1 --2- 172.21.15.7:0/1422945066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb50c0b37b0 0x7fb50c13b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fb50a59c700 1 -- 172.21.15.7:0/1422945066 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb50c15ac20 msgr2=0x7fb50c13a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fb50a59c700 1 --2- 172.21.15.7:0/1422945066 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb50c15ac20 0x7fb50c13a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fb50a59c700 1 -- 172.21.15.7:0/1422945066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb50c1422b0 msgr2=0x7fb50c156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fb50a59c700 1 --2- 172.21.15.7:0/1422945066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb50c1422b0 0x7fb50c156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fb50a59c700 1 -- 172.21.15.7:0/1422945066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb504007790 con 0x7fb50c0b37b0 2024-09-13T18:52:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fb50a59c700 1 --2- 172.21.15.7:0/1422945066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb50c0b37b0 0x7fb50c13b9b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb50000b470 tx=0x7fb50000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fc18dd9b700 1 --2- 172.21.15.7:0/4080973988 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc170066730 0x7fc170068bf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc1901646f0 tx=0x7fc18800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fbb5759e700 1 -- 172.21.15.7:0/1624153471 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb580700e0 msgr2=0x7fbb58070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fbb5759e700 1 --2- 172.21.15.7:0/1624153471 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb580700e0 0x7fbb58070550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbb400062a0 tx=0x7fbb40007b10 comp rx=0 tx=0).stop 2024-09-13T18:52:23.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fbb5759e700 1 -- 172.21.15.7:0/1624153471 shutdown_connections 2024-09-13T18:52:23.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fbb5759e700 1 --2- 172.21.15.7:0/1624153471 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb580700e0 0x7fbb58070550 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fbb5759e700 1 --2- 172.21.15.7:0/1624153471 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb5806e080 0x7fbb5806b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fbb5759e700 1 -- 172.21.15.7:0/1624153471 >> 172.21.15.7:0/1624153471 conn(0x7fbb580ae590 msgr2=0x7fbb580ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7f525a764700 1 -- 172.21.15.7:0/1140944868 wait complete. 2024-09-13T18:52:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.546+0000 7f525a764700 1 Processor -- start 2024-09-13T18:52:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f525a764700 1 -- start start 2024-09-13T18:52:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f525a764700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52540ce260 0x7f5254154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f525a764700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52540cebf0 0x7f52540b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f525a764700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52540d0880 0x7f52540b6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f525a764700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52540c9810 con 0x7f52540d0880 2024-09-13T18:52:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f525a764700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52540c8170 con 0x7f52540cebf0 2024-09-13T18:52:23.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f525a764700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52540c8980 con 0x7f52540ce260 2024-09-13T18:52:23.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa27a155700 1 -- 172.21.15.7:0/2574855222 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2740cc0e0 msgr2=0x7fa2740cc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa27a155700 1 --2- 172.21.15.7:0/2574855222 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2740cc0e0 0x7fa2740cc540 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa264009850 tx=0x7fa264009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:23.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f5259f63700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52540d0880 0x7f52540b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f5259f63700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52540d0880 0x7f52540b6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49324/0 (socket says 172.21.15.7:49324) 2024-09-13T18:52:23.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f5259f63700 1 -- 172.21.15.7:0/173299813 learned_addr learned my addr 172.21.15.7:0/173299813 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f5259762700 1 --2- 172.21.15.7:0/173299813 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52540ce260 0x7f5254154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f5258f61700 1 --2- 172.21.15.7:0/173299813 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52540cebf0 0x7f52540b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fbb5759e700 1 -- 172.21.15.7:0/1624153471 shutdown_connections 2024-09-13T18:52:23.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f8189ffb700 1 -- 172.21.15.7:0/1701148430 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c00b810 con 0x7f81840dfa90 2024-09-13T18:52:23.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f8189ffb700 1 -- 172.21.15.7:0/1701148430 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c00b970 con 0x7f81840dfa90 2024-09-13T18:52:23.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f8189ffb700 1 -- 172.21.15.7:0/1701148430 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c00bce0 con 0x7f81840dfa90 2024-09-13T18:52:23.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818bfff700 1 -- 172.21.15.7:0/1701148430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840dfa90 msgr2=0x7f818404fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f818bfff700 1 --2- 172.21.15.7:0/1701148430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840dfa90 0x7f818404fec0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f81840080c0 tx=0x7f817c009c20 comp rx=0 tx=0).stop 2024-09-13T18:52:23.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fbb5759e700 1 -- 172.21.15.7:0/1624153471 wait complete. 2024-09-13T18:52:23.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fc18ed9d700 1 --2- 172.21.15.7:0/4080973988 >> [v2:172.21.15.171:6817/2886154603,v1:172.21.15.171:6819/2886154603] conn(0x7fc17006e8d0 0x7fc170070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa27a155700 1 -- 172.21.15.7:0/2574855222 shutdown_connections 2024-09-13T18:52:23.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa27a155700 1 --2- 172.21.15.7:0/2574855222 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2740cc0e0 0x7fa2740cc540 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa27a155700 1 --2- 172.21.15.7:0/2574855222 >> v2:172.21.15.171:3300/0 conn(0x7fa2741464d0 0x7fa274070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa27a155700 1 -- 172.21.15.7:0/2574855222 >> 172.21.15.7:0/2574855222 conn(0x7fa2740ae590 msgr2=0x7fa2740ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa27a155700 1 -- 172.21.15.7:0/2574855222 shutdown_connections 2024-09-13T18:52:23.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7fa27a155700 1 -- 172.21.15.7:0/2574855222 wait complete. 2024-09-13T18:52:23.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fb4faffd700 1 -- 172.21.15.7:0/1422945066 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb500011830 con 0x7fb50c0b37b0 2024-09-13T18:52:23.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fb4faffd700 1 -- 172.21.15.7:0/1422945066 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5000119b0 con 0x7fb50c0b37b0 2024-09-13T18:52:23.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fb4faffd700 1 -- 172.21.15.7:0/1422945066 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb500011d60 con 0x7fb50c0b37b0 2024-09-13T18:52:23.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fb50b59e700 1 -- 172.21.15.7:0/1422945066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50c1eecd0 con 0x7fb50c0b37b0 2024-09-13T18:52:23.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fc18ed9d700 1 --2- 172.21.15.7:0/4080973988 >> [v2:172.21.15.171:6817/2886154603,v1:172.21.15.171:6819/2886154603] conn(0x7fc17006e8d0 0x7fc170070cf0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fbb5759e700 1 Processor -- start 2024-09-13T18:52:23.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f5259f63700 1 -- 172.21.15.7:0/173299813 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52540ce260 msgr2=0x7f5254154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f5259f63700 1 --2- 172.21.15.7:0/173299813 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f52540ce260 0x7f5254154080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f5259f63700 1 -- 172.21.15.7:0/173299813 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52540cebf0 msgr2=0x7f52540b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f5259f63700 1 --2- 172.21.15.7:0/173299813 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52540cebf0 0x7f52540b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.550+0000 7f5259f63700 1 -- 172.21.15.7:0/173299813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5250009570 con 0x7f52540d0880 2024-09-13T18:52:23.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7f5259f63700 1 --2- 172.21.15.7:0/173299813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52540d0880 0x7f52540b6640 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f524c00b970 tx=0x7f524c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fb50b59e700 1 -- 172.21.15.7:0/1422945066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50c13bef0 con 0x7fb50c0b37b0 2024-09-13T18:52:23.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7f5249ffb700 1 -- 172.21.15.7:0/173299813 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c00c9b0 con 0x7f52540d0880 2024-09-13T18:52:23.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fbb5759e700 1 -- start start 2024-09-13T18:52:23.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7f5d6aa65700 1 Processor -- start 2024-09-13T18:52:23.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fa27a155700 1 Processor -- start 2024-09-13T18:52:23.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7f5249ffb700 1 -- 172.21.15.7:0/173299813 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c00cb30 con 0x7f52540d0880 2024-09-13T18:52:23.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7f5249ffb700 1 -- 172.21.15.7:0/173299813 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c014440 con 0x7f52540d0880 2024-09-13T18:52:23.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7f525a764700 1 -- 172.21.15.7:0/173299813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5254157560 con 0x7f52540d0880 2024-09-13T18:52:23.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7f525a764700 1 -- 172.21.15.7:0/173299813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52540c86a0 con 0x7f52540d0880 2024-09-13T18:52:23.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fc17effd700 1 -- 172.21.15.7:0/4080973988 <== osd.7 v2:172.21.15.171:6817/2886154603 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fc1700713a0 con 0x7fc17006e8d0 2024-09-13T18:52:23.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7f525a764700 1 -- 172.21.15.7:0/173299813 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f52540cdd40 con 0x7f52540d0880 2024-09-13T18:52:23.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7f5d6aa65700 1 -- start start 2024-09-13T18:52:23.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fbb5759e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb5806e080 0x7fbb5816c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fbb5759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5806fd00 0x7fbb5820fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fbb5759e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb580700e0 0x7fbb58213840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fbb5759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5816cbf0 con 0x7fbb5806fd00 2024-09-13T18:52:23.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fbb5759e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb582100f0 con 0x7fbb580700e0 2024-09-13T18:52:23.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fbb5759e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb58089b60 con 0x7fbb5806e080 2024-09-13T18:52:23.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fa27a155700 1 -- start start 2024-09-13T18:52:23.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fa27a155700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2740cc0e0 0x7fa2742021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fa27a155700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2741464d0 0x7fa27406c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fa27a155700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2741606e0 0x7fa27406ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fa27a155700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa274202700 con 0x7fa2740cc0e0 2024-09-13T18:52:23.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fa27a155700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa274142370 con 0x7fa2741606e0 2024-09-13T18:52:23.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fa27a155700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa274142650 con 0x7fa2741464d0 2024-09-13T18:52:23.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fbb5659c700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb5806e080 0x7fbb5816c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fbb55d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5806fd00 0x7fbb5820fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fbb5659c700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb5806e080 0x7fbb5816c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:52956/0 (socket says 172.21.15.7:52956) 2024-09-13T18:52:23.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fbb5659c700 1 -- 172.21.15.7:0/2562126954 learned_addr learned my addr 172.21.15.7:0/2562126954 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fa279153700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2740cc0e0 0x7fa2742021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fa279153700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2740cc0e0 0x7fa2742021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49330/0 (socket says 172.21.15.7:49330) 2024-09-13T18:52:23.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fa279153700 1 -- 172.21.15.7:0/4226314050 learned_addr learned my addr 172.21.15.7:0/4226314050 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fb4faffd700 1 -- 172.21.15.7:0/1422945066 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb50001a400 con 0x7fb50c0b37b0 2024-09-13T18:52:23.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fb50b59e700 1 -- 172.21.15.7:0/1422945066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb50c1f1890 con 0x7fb50c0b37b0 2024-09-13T18:52:23.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fb4faffd700 1 --2- 172.21.15.7:0/1422945066 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb4ec066780 0x7fb4ec068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fb4faffd700 1 -- 172.21.15.7:0/1422945066 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb500084000 con 0x7fb50c0b37b0 2024-09-13T18:52:23.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fb4faffd700 1 --2- 172.21.15.7:0/1422945066 >> [v2:172.21.15.7:6806/4238805012,v1:172.21.15.7:6810/4238805012] conn(0x7fb4ec06e920 0x7fb4ec070d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fb4faffd700 1 -- 172.21.15.7:0/1422945066 --> [v2:172.21.15.7:6806/4238805012,v1:172.21.15.7:6810/4238805012] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb4ec0713f0 con 0x7fb4ec06e920 2024-09-13T18:52:23.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7f818bfff700 1 -- 172.21.15.7:0/1701148430 shutdown_connections 2024-09-13T18:52:23.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7f818bfff700 1 --2- 172.21.15.7:0/1701148430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840dfa90 0x7f818404fec0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7f818bfff700 1 --2- 172.21.15.7:0/1701148430 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f818401cf40 0x7f81840df550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7f818bfff700 1 --2- 172.21.15.7:0/1701148430 >> v2:172.21.15.171:3300/0 conn(0x7f81840d96f0 0x7f818404f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7f818bfff700 1 -- 172.21.15.7:0/1701148430 >> 172.21.15.7:0/1701148430 conn(0x7f8184036140 msgr2=0x7f8184036550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7f818bfff700 1 -- 172.21.15.7:0/1701148430 shutdown_connections 2024-09-13T18:52:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fa278952700 1 --2- 172.21.15.7:0/4226314050 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2741464d0 0x7fa27406c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fb509d9b700 1 --2- 172.21.15.7:0/1422945066 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb4ec066780 0x7fb4ec068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fb50ad9d700 1 --2- 172.21.15.7:0/1422945066 >> [v2:172.21.15.7:6806/4238805012,v1:172.21.15.7:6810/4238805012] conn(0x7fb4ec06e920 0x7fb4ec070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fa279153700 1 -- 172.21.15.7:0/4226314050 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2741464d0 msgr2=0x7fa27406c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fa279153700 1 --2- 172.21.15.7:0/4226314050 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2741464d0 0x7fa27406c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fa279153700 1 -- 172.21.15.7:0/4226314050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2741606e0 msgr2=0x7fa27406ca20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:52:23.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fa279153700 1 --2- 172.21.15.7:0/4226314050 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2741606e0 0x7fa27406ca20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fa279153700 1 -- 172.21.15.7:0/4226314050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2640094b0 con 0x7fa2740cc0e0 2024-09-13T18:52:23.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fa278952700 1 --2- 172.21.15.7:0/4226314050 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2741464d0 0x7fa27406c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:23.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fbb5659c700 1 -- 172.21.15.7:0/2562126954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb580700e0 msgr2=0x7fbb58213840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:52:23.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fbb5659c700 1 --2- 172.21.15.7:0/2562126954 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb580700e0 0x7fbb58213840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fbb5659c700 1 -- 172.21.15.7:0/2562126954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5806fd00 msgr2=0x7fbb5820fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fbb5659c700 1 --2- 172.21.15.7:0/2562126954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb5806fd00 0x7fbb5820fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7fbb5659c700 1 -- 172.21.15.7:0/2562126954 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb40007510 con 0x7fbb5806e080 2024-09-13T18:52:23.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7fb50ad9d700 1 --2- 172.21.15.7:0/1422945066 >> [v2:172.21.15.7:6806/4238805012,v1:172.21.15.7:6810/4238805012] conn(0x7fb4ec06e920 0x7fb4ec070d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7fbb5659c700 1 --2- 172.21.15.7:0/2562126954 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb5806e080 0x7fbb5816c610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800c500 tx=0x7fbb4800c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f818bfff700 1 -- 172.21.15.7:0/1701148430 wait complete. 2024-09-13T18:52:23.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7fa279153700 1 --2- 172.21.15.7:0/4226314050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2740cc0e0 0x7fa2742021c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa27000e820 tx=0x7fa27000ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7fb4faffd700 1 -- 172.21.15.7:0/1422945066 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5000092c0 con 0x7fb50c0b37b0 2024-09-13T18:52:23.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7fbb4effd700 1 -- 172.21.15.7:0/2562126954 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48005d90 con 0x7fbb5806e080 2024-09-13T18:52:23.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7fbb4effd700 1 -- 172.21.15.7:0/2562126954 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb480070d0 con 0x7fbb5806e080 2024-09-13T18:52:23.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7fbb5759e700 1 -- 172.21.15.7:0/2562126954 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb580a8ff0 con 0x7fbb5806e080 2024-09-13T18:52:23.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7fbb5759e700 1 -- 172.21.15.7:0/2562126954 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb580a95b0 con 0x7fbb5806e080 2024-09-13T18:52:23.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7fbb4effd700 1 -- 172.21.15.7:0/2562126954 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48007440 con 0x7fbb5806e080 2024-09-13T18:52:23.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7fa269ffb700 1 -- 172.21.15.7:0/4226314050 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27000c690 con 0x7fa2740cc0e0 2024-09-13T18:52:23.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7fa27a155700 1 -- 172.21.15.7:0/4226314050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa274142930 con 0x7fa2740cc0e0 2024-09-13T18:52:23.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7fb509d9b700 1 --2- 172.21.15.7:0/1422945066 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb4ec066780 0x7fb4ec068c40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb50c0b3290 tx=0x7fb4fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7fa27a155700 1 -- 172.21.15.7:0/4226314050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa274142ef0 con 0x7fa2740cc0e0 2024-09-13T18:52:23.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f2db2e6d700 1 Processor -- start 2024-09-13T18:52:23.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f2db2e6d700 1 -- start start 2024-09-13T18:52:23.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7fa269ffb700 1 -- 172.21.15.7:0/4226314050 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27000c810 con 0x7fa2740cc0e0 2024-09-13T18:52:23.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7fa269ffb700 1 -- 172.21.15.7:0/4226314050 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27000cbc0 con 0x7fa2740cc0e0 2024-09-13T18:52:23.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5249ffb700 1 -- 172.21.15.7:0/173299813 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f524c01d3b0 con 0x7f52540d0880 2024-09-13T18:52:23.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5249ffb700 1 --2- 172.21.15.7:0/173299813 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5238066800 0x7f5238068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5249ffb700 1 -- 172.21.15.7:0/173299813 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f524c084440 con 0x7f52540d0880 2024-09-13T18:52:23.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5249ffb700 1 --2- 172.21.15.7:0/173299813 >> [v2:172.21.15.171:6800/1011659457,v1:172.21.15.171:6801/1011659457] conn(0x7f523806ea20 0x7f5238070e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5249ffb700 1 -- 172.21.15.7:0/173299813 --> [v2:172.21.15.171:6800/1011659457,v1:172.21.15.171:6801/1011659457] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52380714f0 con 0x7f523806ea20 2024-09-13T18:52:23.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5249ffb700 1 -- 172.21.15.7:0/173299813 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f524c0847f0 con 0x7f52540d0880 2024-09-13T18:52:23.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7fb4faffd700 1 -- 172.21.15.7:0/1422945066 <== osd.0 v2:172.21.15.7:6806/4238805012 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb4ec0713f0 con 0x7fb4ec06e920 2024-09-13T18:52:23.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f818bfff700 1 Processor -- start 2024-09-13T18:52:23.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5258f61700 1 --2- 172.21.15.7:0/173299813 >> [v2:172.21.15.171:6800/1011659457,v1:172.21.15.171:6801/1011659457] conn(0x7f523806ea20 0x7f5238070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f2db2e6d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2dac0cc810 0x7f2dac1677c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f2db2e6d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dac070e80 con 0x7f2dac146480 2024-09-13T18:52:23.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f2db2e6d700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dac071160 con 0x7f2dac15dd90 2024-09-13T18:52:23.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f2db2e6d700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac071440 con 0x7f2dac0cc810 2024-09-13T18:52:23.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f818bfff700 1 -- start start 2024-09-13T18:52:23.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f2db166a700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f2dac15dd90 0x7f2dac0cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:51570/0 (socket says 172.21.15.7:51570) 2024-09-13T18:52:23.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f2db166a700 1 -- 172.21.15.7:0/775352042 learned_addr learned my addr 172.21.15.7:0/775352042 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:23.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f2db266c700 1 --2- 172.21.15.7:0/775352042 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2dac0cc810 0x7f2dac1677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7fa27a155700 1 -- 172.21.15.7:0/4226314050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa274071ab0 con 0x7fa2740cc0e0 2024-09-13T18:52:23.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5259762700 1 --2- 172.21.15.7:0/173299813 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5238066800 0x7f5238068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f2db0e69700 1 -- 172.21.15.7:0/775352042 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2504807609 0 0) 0x7f2dac071160 con 0x7f2dac15dd90 2024-09-13T18:52:23.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f2db0e69700 1 -- 172.21.15.7:0/775352042 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d980035d0 con 0x7f2dac15dd90 2024-09-13T18:52:23.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f2db266c700 1 -- 172.21.15.7:0/775352042 >> v1:172.21.15.171:6789/0 conn(0x7f2dac15dd90 legacy=0x7f2dac0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f2db266c700 1 -- 172.21.15.7:0/775352042 >> v1:172.21.15.7:6789/0 conn(0x7f2dac146480 legacy=0x7f2dac0704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.554+0000 7f5d6aa65700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f5d641573e0 0x7f5d64159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7fbb4effd700 1 -- 172.21.15.7:0/2562126954 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbb480075a0 con 0x7fbb5806e080 2024-09-13T18:52:23.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5d6aa65700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d6415ac50 con 0x7f5d6406d3f0 2024-09-13T18:52:23.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5d6aa65700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d6415e100 con 0x7f5d6415a460 2024-09-13T18:52:23.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5d6aa65700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d640cc1b0 con 0x7f5d641573e0 2024-09-13T18:52:23.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5d69a63700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f5d641573e0 0x7f5d64159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5d69262700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f5d6415a460 0x7f5d6415d2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:51554/0 (socket says 172.21.15.7:51554) 2024-09-13T18:52:23.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5d69262700 1 -- 172.21.15.7:0/2294369080 learned_addr learned my addr 172.21.15.7:0/2294369080 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:23.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5d69a63700 1 -- 172.21.15.7:0/2294369080 >> v1:172.21.15.171:6789/0 conn(0x7f5d6415a460 legacy=0x7f5d6415d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5d69a63700 1 -- 172.21.15.7:0/2294369080 >> v1:172.21.15.7:6789/0 conn(0x7f5d6406d3f0 legacy=0x7f5d6415d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5d69a63700 1 -- 172.21.15.7:0/2294369080 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d64160430 con 0x7f5d641573e0 2024-09-13T18:52:23.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5d69a63700 1 --2- 172.21.15.7:0/2294369080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d641573e0 0x7f5d64159f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009850 tx=0x7f5d60009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e852ce2c0773dd29 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5d68a61700 1 -- 172.21.15.7:0/2294369080 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60011630 con 0x7f5d641573e0 2024-09-13T18:52:23.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.558+0000 7f5d68a61700 1 -- 172.21.15.7:0/2294369080 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d60011790 con 0x7f5d641573e0 2024-09-13T18:52:23.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f5d68a61700 1 -- 172.21.15.7:0/2294369080 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60011b20 con 0x7f5d641573e0 2024-09-13T18:52:23.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f2db266c700 1 -- 172.21.15.7:0/775352042 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dac071720 con 0x7f2dac0cc810 2024-09-13T18:52:23.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f2db266c700 1 --2- 172.21.15.7:0/775352042 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2dac0cc810 0x7f2dac1677c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c009880 tx=0x7f2d9c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1addfdf1dd1e9b2a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7fbb4effd700 1 --2- 172.21.15.7:0/2562126954 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbb38066730 0x7fbb38068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f2db0e69700 1 -- 172.21.15.7:0/775352042 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c011830 con 0x7f2dac0cc810 2024-09-13T18:52:23.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7fbb4effd700 1 -- 172.21.15.7:0/2562126954 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb480834d0 con 0x7fbb5806e080 2024-09-13T18:52:23.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f2db0e69700 1 -- 172.21.15.7:0/775352042 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c011990 con 0x7f2dac0cc810 2024-09-13T18:52:23.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f2db0e69700 1 -- 172.21.15.7:0/775352042 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c011d00 con 0x7f2dac0cc810 2024-09-13T18:52:23.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7fbb55d9b700 1 --2- 172.21.15.7:0/2562126954 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbb38066730 0x7fbb38068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f5d6aa65700 1 -- 172.21.15.7:0/2294369080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d641573e0 msgr2=0x7f5d64159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f5d6aa65700 1 --2- 172.21.15.7:0/2294369080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d641573e0 0x7f5d64159f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5d60009850 tx=0x7f5d60009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:23.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f5d6aa65700 1 -- 172.21.15.7:0/2294369080 shutdown_connections 2024-09-13T18:52:23.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f5d6aa65700 1 --2- 172.21.15.7:0/2294369080 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d641573e0 0x7f5d64159f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f5d6aa65700 1 -- 172.21.15.7:0/2294369080 >> 172.21.15.7:0/2294369080 conn(0x7f5d640b0e10 msgr2=0x7f5d640b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f5d6aa65700 1 -- 172.21.15.7:0/2294369080 shutdown_connections 2024-09-13T18:52:23.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f5d6aa65700 1 -- 172.21.15.7:0/2294369080 wait complete. 2024-09-13T18:52:23.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f5259762700 1 --2- 172.21.15.7:0/173299813 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5238066800 0x7f5238068cc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f525000c010 tx=0x7f5250008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f2db2e6d700 1 -- 172.21.15.7:0/775352042 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2dac0cc810 msgr2=0x7f2dac1677c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f2db2e6d700 1 --2- 172.21.15.7:0/775352042 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2dac0cc810 0x7f2dac1677c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c009880 tx=0x7f2d9c009b90 comp rx=0 tx=0).stop 2024-09-13T18:52:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f2db2e6d700 1 -- 172.21.15.7:0/775352042 shutdown_connections 2024-09-13T18:52:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f2db2e6d700 1 --2- 172.21.15.7:0/775352042 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2dac0cc810 0x7f2dac1677c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f2db2e6d700 1 -- 172.21.15.7:0/775352042 >> 172.21.15.7:0/775352042 conn(0x7f2dac0ae590 msgr2=0x7f2dac0ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f5258f61700 1 --2- 172.21.15.7:0/173299813 >> [v2:172.21.15.171:6800/1011659457,v1:172.21.15.171:6801/1011659457] conn(0x7f523806ea20 0x7f5238070e40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7fbb5759e700 1 --2- 172.21.15.7:0/2562126954 >> [v2:172.21.15.7:6808/2837570651,v1:172.21.15.7:6811/2837570651] conn(0x7fbb3c0015e0 0x7fbb3c003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7fbb5759e700 1 -- 172.21.15.7:0/2562126954 --> [v2:172.21.15.7:6808/2837570651,v1:172.21.15.7:6811/2837570651] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb3c006bf0 con 0x7fbb3c0015e0 2024-09-13T18:52:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f2db2e6d700 1 -- 172.21.15.7:0/775352042 shutdown_connections 2024-09-13T18:52:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f818401cf40 0x7f81840390b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f2db2e6d700 1 -- 172.21.15.7:0/775352042 wait complete. 2024-09-13T18:52:23.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7fbb56d9d700 1 --2- 172.21.15.7:0/2562126954 >> [v2:172.21.15.7:6808/2837570651,v1:172.21.15.7:6811/2837570651] conn(0x7fbb3c0015e0 0x7fbb3c003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818bfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f81840d96f0 0x7f81840d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840dfa90 0x7f818404a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818bfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81840395f0 con 0x7f81840dfa90 2024-09-13T18:52:23.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818bfff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81840481e0 con 0x7f818401cf40 2024-09-13T18:52:23.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818bfff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81840484c0 con 0x7f81840d96f0 2024-09-13T18:52:23.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818affd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f818401cf40 0x7f81840390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818affd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f818401cf40 0x7f81840390b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:40772/0 (socket says 172.21.15.7:40772) 2024-09-13T18:52:23.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818affd700 1 -- 172.21.15.7:0/2711171634 learned_addr learned my addr 172.21.15.7:0/2711171634 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818a7fc700 1 --2- 172.21.15.7:0/2711171634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f81840d96f0 0x7f81840d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818b7fe700 1 --2- 172.21.15.7:0/2711171634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840dfa90 0x7f818404a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7fbb56d9d700 1 --2- 172.21.15.7:0/2562126954 >> [v2:172.21.15.7:6808/2837570651,v1:172.21.15.7:6811/2837570651] conn(0x7fbb3c0015e0 0x7fbb3c003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818b7fe700 1 -- 172.21.15.7:0/2711171634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f81840d96f0 msgr2=0x7f81840d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818b7fe700 1 --2- 172.21.15.7:0/2711171634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f81840d96f0 0x7f81840d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818b7fe700 1 -- 172.21.15.7:0/2711171634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f818401cf40 msgr2=0x7f81840390b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818b7fe700 1 --2- 172.21.15.7:0/2711171634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f818401cf40 0x7f81840390b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818b7fe700 1 -- 172.21.15.7:0/2711171634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817c009570 con 0x7f81840dfa90 2024-09-13T18:52:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f5d6aa65700 1 Processor -- start 2024-09-13T18:52:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818b7fe700 1 --2- 172.21.15.7:0/2711171634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840dfa90 0x7f818404a000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f817c016000 tx=0x7f817c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f817bfff700 1 -- 172.21.15.7:0/2711171634 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c004510 con 0x7f81840dfa90 2024-09-13T18:52:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818bfff700 1 -- 172.21.15.7:0/2711171634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81840487a0 con 0x7f81840dfa90 2024-09-13T18:52:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f817bfff700 1 -- 172.21.15.7:0/2711171634 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c01e070 con 0x7f81840dfa90 2024-09-13T18:52:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f817bfff700 1 -- 172.21.15.7:0/2711171634 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c003900 con 0x7f81840dfa90 2024-09-13T18:52:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7fbb55d9b700 1 --2- 172.21.15.7:0/2562126954 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbb38066730 0x7fbb38068bf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbb5808a0a0 tx=0x7fbb5000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f818bfff700 1 -- 172.21.15.7:0/2711171634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81840398c0 con 0x7f81840dfa90 2024-09-13T18:52:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f5249ffb700 1 -- 172.21.15.7:0/173299813 <== osd.5 v2:172.21.15.171:6800/1011659457 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f52380714f0 con 0x7f523806ea20 2024-09-13T18:52:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7fbb4effd700 1 -- 172.21.15.7:0/2562126954 <== osd.1 v2:172.21.15.7:6808/2837570651 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fbb3c006bf0 con 0x7fbb3c0015e0 2024-09-13T18:52:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f2db2e6d700 1 Processor -- start 2024-09-13T18:52:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.562+0000 7f2db2e6d700 1 -- start start 2024-09-13T18:52:23.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7fa269ffb700 1 -- 172.21.15.7:0/4226314050 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa27000cd20 con 0x7fa2740cc0e0 2024-09-13T18:52:23.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7fa269ffb700 1 --2- 172.21.15.7:0/4226314050 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa258066730 0x7fa258068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7fa269ffb700 1 -- 172.21.15.7:0/4226314050 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa270084170 con 0x7fa2740cc0e0 2024-09-13T18:52:23.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f8179ffb700 1 -- 172.21.15.7:0/2711171634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f818404ed40 con 0x7f81840dfa90 2024-09-13T18:52:23.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7fa269ffb700 1 --2- 172.21.15.7:0/4226314050 >> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] conn(0x7fa25806e950 0x7fa258070d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7fa269ffb700 1 -- 172.21.15.7:0/4226314050 --> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa258071420 con 0x7fa25806e950 2024-09-13T18:52:23.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7fa269ffb700 1 -- 172.21.15.7:0/4226314050 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa270086790 con 0x7fa2740cc0e0 2024-09-13T18:52:23.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db2e6d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2dac0cc810 0x7f2dac1379d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db2e6d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2dac146480 0x7f2dac136df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db2e6d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dac15dd90 0x7f2dac1399b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db2e6d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac136070 con 0x7f2dac15dd90 2024-09-13T18:52:23.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db2e6d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac136310 con 0x7f2dac146480 2024-09-13T18:52:23.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db2e6d700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dac1365f0 con 0x7f2dac0cc810 2024-09-13T18:52:23.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7fa279954700 1 --2- 172.21.15.7:0/4226314050 >> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] conn(0x7fa25806e950 0x7fa258070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7fa278952700 1 --2- 172.21.15.7:0/4226314050 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa258066730 0x7fa258068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db1e6b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2dac0cc810 0x7f2dac1379d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db1e6b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2dac0cc810 0x7f2dac1379d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:52994/0 (socket says 172.21.15.7:52994) 2024-09-13T18:52:23.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db1e6b700 1 -- 172.21.15.7:0/2048557056 learned_addr learned my addr 172.21.15.7:0/2048557056 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db166a700 1 --2- 172.21.15.7:0/2048557056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2dac146480 0x7f2dac136df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db266c700 1 --2- 172.21.15.7:0/2048557056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dac15dd90 0x7f2dac1399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7fa279954700 1 --2- 172.21.15.7:0/4226314050 >> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] conn(0x7fa25806e950 0x7fa258070d70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7fa278952700 1 --2- 172.21.15.7:0/4226314050 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa258066730 0x7fa258068bf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa264009850 tx=0x7fa264008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db266c700 1 -- 172.21.15.7:0/2048557056 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2dac0cc810 msgr2=0x7f2dac1379d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db266c700 1 --2- 172.21.15.7:0/2048557056 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2dac0cc810 0x7f2dac1379d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db266c700 1 -- 172.21.15.7:0/2048557056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2dac146480 msgr2=0x7f2dac136df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db266c700 1 --2- 172.21.15.7:0/2048557056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2dac146480 0x7f2dac136df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db266c700 1 -- 172.21.15.7:0/2048557056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c0094b0 con 0x7f2dac15dd90 2024-09-13T18:52:23.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db166a700 1 --2- 172.21.15.7:0/2048557056 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2dac146480 0x7f2dac136df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:23.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f817bfff700 1 -- 172.21.15.7:0/2711171634 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f817c003a60 con 0x7f81840dfa90 2024-09-13T18:52:23.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f817bfff700 1 --2- 172.21.15.7:0/2711171634 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8170066760 0x7f8170068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f817bfff700 1 -- 172.21.15.7:0/2711171634 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f817c020070 con 0x7f81840dfa90 2024-09-13T18:52:23.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f817bfff700 1 --2- 172.21.15.7:0/2711171634 >> [v2:172.21.15.7:6800/2775757779,v1:172.21.15.7:6801/2775757779] conn(0x7f817006e900 0x7f8170070d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f817bfff700 1 -- 172.21.15.7:0/2711171634 --> [v2:172.21.15.7:6800/2775757779,v1:172.21.15.7:6801/2775757779] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f81700713d0 con 0x7f817006e900 2024-09-13T18:52:23.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f817bfff700 1 -- 172.21.15.7:0/2711171634 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f817c087890 con 0x7f81840dfa90 2024-09-13T18:52:23.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f818affd700 1 --2- 172.21.15.7:0/2711171634 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8170066760 0x7f8170068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f818a7fc700 1 --2- 172.21.15.7:0/2711171634 >> [v2:172.21.15.7:6800/2775757779,v1:172.21.15.7:6801/2775757779] conn(0x7f817006e900 0x7f8170070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db266c700 1 --2- 172.21.15.7:0/2048557056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dac15dd90 0x7f2dac1399b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c015000 tx=0x7f2d9c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2da27fc700 1 -- 172.21.15.7:0/2048557056 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c006ea0 con 0x7f2dac15dd90 2024-09-13T18:52:23.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db2e6d700 1 -- 172.21.15.7:0/2048557056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dac1368d0 con 0x7f2dac15dd90 2024-09-13T18:52:23.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f818affd700 1 --2- 172.21.15.7:0/2711171634 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8170066760 0x7f8170068c20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f818c04fc20 tx=0x7f818c08c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.570+0000 7f818a7fc700 1 --2- 172.21.15.7:0/2711171634 >> [v2:172.21.15.7:6800/2775757779,v1:172.21.15.7:6801/2775757779] conn(0x7f817006e900 0x7f8170070d20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.570+0000 7fa269ffb700 1 -- 172.21.15.7:0/4226314050 <== osd.4 v2:172.21.15.171:6816/1731498321 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fa258071420 con 0x7fa25806e950 2024-09-13T18:52:23.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2db2e6d700 1 -- 172.21.15.7:0/2048557056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dac15c6c0 con 0x7f2dac15dd90 2024-09-13T18:52:23.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.566+0000 7f2da27fc700 1 -- 172.21.15.7:0/2048557056 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c0036c0 con 0x7f2dac15dd90 2024-09-13T18:52:23.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.570+0000 7f2da27fc700 1 -- 172.21.15.7:0/2048557056 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c003a70 con 0x7f2dac15dd90 2024-09-13T18:52:23.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.570+0000 7f2da27fc700 1 -- 172.21.15.7:0/2048557056 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2d9c003c90 con 0x7f2dac15dd90 2024-09-13T18:52:23.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.570+0000 7f817bfff700 1 -- 172.21.15.7:0/2711171634 <== osd.3 v2:172.21.15.7:6800/2775757779 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f81700713d0 con 0x7f817006e900 2024-09-13T18:52:23.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.570+0000 7f2da27fc700 1 --2- 172.21.15.7:0/2048557056 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2d98066be0 0x7f2d980690a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.570+0000 7f2da27fc700 1 -- 172.21.15.7:0/2048557056 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d9c085590 con 0x7f2dac15dd90 2024-09-13T18:52:23.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.570+0000 7f2db1e6b700 1 --2- 172.21.15.7:0/2048557056 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2d98066be0 0x7f2d980690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.570+0000 7f2db1e6b700 1 --2- 172.21.15.7:0/2048557056 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2d98066be0 0x7f2d980690a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0aed30 tx=0x7f2da8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d6aa65700 1 -- start start 2024-09-13T18:52:23.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d6aa65700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6406d3f0 0x7f5d6415f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d6aa65700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5d641573e0 0x7f5d640cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d6aa65700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d6415a460 0x7f5d640b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d6aa65700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6415f6f0 con 0x7f5d6406d3f0 2024-09-13T18:52:23.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d6aa65700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6415fef0 con 0x7f5d6415a460 2024-09-13T18:52:23.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d6aa65700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d640ce140 con 0x7f5d641573e0 2024-09-13T18:52:23.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d69a63700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6406d3f0 0x7f5d6415f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d69a63700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6406d3f0 0x7f5d6415f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49368/0 (socket says 172.21.15.7:49368) 2024-09-13T18:52:23.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d69a63700 1 -- 172.21.15.7:0/1322986134 learned_addr learned my addr 172.21.15.7:0/1322986134 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d69262700 1 --2- 172.21.15.7:0/1322986134 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5d641573e0 0x7f5d640cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d6a264700 1 --2- 172.21.15.7:0/1322986134 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d6415a460 0x7f5d640b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d69a63700 1 -- 172.21.15.7:0/1322986134 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5d641573e0 msgr2=0x7f5d640cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d69a63700 1 --2- 172.21.15.7:0/1322986134 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5d641573e0 0x7f5d640cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d69a63700 1 -- 172.21.15.7:0/1322986134 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d6415a460 msgr2=0x7f5d640b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d69a63700 1 --2- 172.21.15.7:0/1322986134 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d6415a460 0x7f5d640b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d69a63700 1 -- 172.21.15.7:0/1322986134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d600094b0 con 0x7f5d6406d3f0 2024-09-13T18:52:23.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d69a63700 1 --2- 172.21.15.7:0/1322986134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6406d3f0 0x7f5d6415f1b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5d64157850 tx=0x7f5d60004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d5a7fc700 1 -- 172.21.15.7:0/1322986134 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60011d10 con 0x7f5d6406d3f0 2024-09-13T18:52:23.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d5a7fc700 1 -- 172.21.15.7:0/1322986134 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d60011e70 con 0x7f5d6406d3f0 2024-09-13T18:52:23.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d5a7fc700 1 -- 172.21.15.7:0/1322986134 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d60003900 con 0x7f5d6406d3f0 2024-09-13T18:52:23.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d6aa65700 1 -- 172.21.15.7:0/1322986134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d640ce370 con 0x7f5d6406d3f0 2024-09-13T18:52:23.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.574+0000 7f5d6aa65700 1 -- 172.21.15.7:0/1322986134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6415e6d0 con 0x7f5d6406d3f0 2024-09-13T18:52:23.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.578+0000 7f5d4ffff700 1 -- 172.21.15.7:0/1322986134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5d48000fc0 con 0x7f5d6406d3f0 2024-09-13T18:52:23.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.578+0000 7f5d5a7fc700 1 -- 172.21.15.7:0/1322986134 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5d60003a60 con 0x7f5d6406d3f0 2024-09-13T18:52:23.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.578+0000 7f5d5a7fc700 1 --2- 172.21.15.7:0/1322986134 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5d50066910 0x7f5d50068dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.578+0000 7f5d5a7fc700 1 -- 172.21.15.7:0/1322986134 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d60096c30 con 0x7f5d6406d3f0 2024-09-13T18:52:23.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.578+0000 7f5d5a7fc700 1 --2- 172.21.15.7:0/1322986134 >> [v2:172.21.15.7:6824/303159811,v1:172.21.15.7:6825/303159811] conn(0x7f5d5006eb30 0x7f5d50070f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.578+0000 7f5d5a7fc700 1 -- 172.21.15.7:0/1322986134 --> [v2:172.21.15.7:6824/303159811,v1:172.21.15.7:6825/303159811] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d50071600 con 0x7f5d5006eb30 2024-09-13T18:52:23.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.578+0000 7f5d5a7fc700 1 -- 172.21.15.7:0/1322986134 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d600970d0 con 0x7f5d6406d3f0 2024-09-13T18:52:23.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.578+0000 7f5d69262700 1 --2- 172.21.15.7:0/1322986134 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5d50066910 0x7f5d50068dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.578+0000 7f5d6a264700 1 --2- 172.21.15.7:0/1322986134 >> [v2:172.21.15.7:6824/303159811,v1:172.21.15.7:6825/303159811] conn(0x7f5d5006eb30 0x7f5d50070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.578+0000 7f5d6a264700 1 --2- 172.21.15.7:0/1322986134 >> [v2:172.21.15.7:6824/303159811,v1:172.21.15.7:6825/303159811] conn(0x7f5d5006eb30 0x7f5d50070f50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.578+0000 7f5d69262700 1 --2- 172.21.15.7:0/1322986134 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5d50066910 0x7f5d50068dd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5d64071500 tx=0x7f5d54008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.582+0000 7f5d5a7fc700 1 -- 172.21.15.7:0/1322986134 <== osd.2 v2:172.21.15.7:6824/303159811 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f5d50071600 con 0x7f5d5006eb30 2024-09-13T18:52:23.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.582+0000 7f2db2e6d700 1 --2- 172.21.15.7:0/2048557056 >> [v2:172.21.15.171:6808/2277626206,v1:172.21.15.171:6809/2277626206] conn(0x7f2d940015e0 0x7f2d94003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.582+0000 7f2db2e6d700 1 -- 172.21.15.7:0/2048557056 --> [v2:172.21.15.171:6808/2277626206,v1:172.21.15.171:6809/2277626206] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2d94006bf0 con 0x7f2d940015e0 2024-09-13T18:52:23.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.582+0000 7f2db166a700 1 --2- 172.21.15.7:0/2048557056 >> [v2:172.21.15.171:6808/2277626206,v1:172.21.15.171:6809/2277626206] conn(0x7f2d940015e0 0x7f2d94003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.582+0000 7f2db166a700 1 --2- 172.21.15.7:0/2048557056 >> [v2:172.21.15.171:6808/2277626206,v1:172.21.15.171:6809/2277626206] conn(0x7f2d940015e0 0x7f2d94003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.582+0000 7f2da27fc700 1 -- 172.21.15.7:0/2048557056 <== osd.6 v2:172.21.15.171:6808/2277626206 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f2d94006bf0 con 0x7f2d940015e0 2024-09-13T18:52:23.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.590+0000 7fb50b59e700 1 -- 172.21.15.7:0/1422945066 --> [v2:172.21.15.7:6806/4238805012,v1:172.21.15.7:6810/4238805012] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb50c142690 con 0x7fb4ec06e920 2024-09-13T18:52:23.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.590+0000 7fb4faffd700 1 -- 172.21.15.7:0/1422945066 <== osd.0 v2:172.21.15.7:6806/4238805012 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb504007c00 con 0x7fb4ec06e920 2024-09-13T18:52:23.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.590+0000 7f525a764700 1 -- 172.21.15.7:0/173299813 --> [v2:172.21.15.171:6800/1011659457,v1:172.21.15.171:6801/1011659457] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f52540cda60 con 0x7f523806ea20 2024-09-13T18:52:23.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.590+0000 7fb50b59e700 1 -- 172.21.15.7:0/1422945066 >> [v2:172.21.15.7:6806/4238805012,v1:172.21.15.7:6810/4238805012] conn(0x7fb4ec06e920 msgr2=0x7fb4ec070d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.590+0000 7fb50b59e700 1 --2- 172.21.15.7:0/1422945066 >> [v2:172.21.15.7:6806/4238805012,v1:172.21.15.7:6810/4238805012] conn(0x7fb4ec06e920 0x7fb4ec070d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.590+0000 7fb50b59e700 1 -- 172.21.15.7:0/1422945066 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb4ec066780 msgr2=0x7fb4ec068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.590+0000 7fb50b59e700 1 --2- 172.21.15.7:0/1422945066 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb4ec066780 0x7fb4ec068c40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb50c0b3290 tx=0x7fb4fc009040 comp rx=0 tx=0).stop 2024-09-13T18:52:23.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.590+0000 7fb50b59e700 1 -- 172.21.15.7:0/1422945066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb50c0b37b0 msgr2=0x7fb50c13b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.590+0000 7fb50b59e700 1 --2- 172.21.15.7:0/1422945066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb50c0b37b0 0x7fb50c13b9b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb50000b470 tx=0x7fb50000b780 comp rx=0 tx=0).stop 2024-09-13T18:52:23.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.590+0000 7fb50ad9d700 1 -- 172.21.15.7:0/1422945066 reap_dead start 2024-09-13T18:52:23.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.590+0000 7f5249ffb700 1 -- 172.21.15.7:0/173299813 <== osd.5 v2:172.21.15.171:6800/1011659457 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f524400c040 con 0x7f523806ea20 2024-09-13T18:52:23.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.590+0000 7fb50b59e700 1 -- 172.21.15.7:0/1422945066 shutdown_connections 2024-09-13T18:52:23.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.590+0000 7fb50b59e700 1 -- 172.21.15.7:0/1422945066 >> 172.21.15.7:0/1422945066 conn(0x7fb50c0b0e10 msgr2=0x7fb50c0b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.590+0000 7fb50b59e700 1 -- 172.21.15.7:0/1422945066 shutdown_connections 2024-09-13T18:52:23.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.590+0000 7fb50b59e700 1 -- 172.21.15.7:0/1422945066 wait complete. 2024-09-13T18:52:23.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.594+0000 7f525a764700 1 -- 172.21.15.7:0/173299813 >> [v2:172.21.15.171:6800/1011659457,v1:172.21.15.171:6801/1011659457] conn(0x7f523806ea20 msgr2=0x7f5238070e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.594+0000 7f525a764700 1 --2- 172.21.15.7:0/173299813 >> [v2:172.21.15.171:6800/1011659457,v1:172.21.15.171:6801/1011659457] conn(0x7f523806ea20 0x7f5238070e40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.594+0000 7f525a764700 1 -- 172.21.15.7:0/173299813 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5238066800 msgr2=0x7f5238068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.594+0000 7f525a764700 1 --2- 172.21.15.7:0/173299813 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5238066800 0x7f5238068cc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f525000c010 tx=0x7f5250008030 comp rx=0 tx=0).stop 2024-09-13T18:52:23.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.594+0000 7f525a764700 1 -- 172.21.15.7:0/173299813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52540d0880 msgr2=0x7f52540b6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.594+0000 7f525a764700 1 --2- 172.21.15.7:0/173299813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52540d0880 0x7f52540b6640 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f524c00b970 tx=0x7f524c00bd30 comp rx=0 tx=0).stop 2024-09-13T18:52:23.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.594+0000 7f5259f63700 1 -- 172.21.15.7:0/173299813 reap_dead start 2024-09-13T18:52:23.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.594+0000 7f525a764700 1 -- 172.21.15.7:0/173299813 shutdown_connections 2024-09-13T18:52:23.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.594+0000 7f525a764700 1 -- 172.21.15.7:0/173299813 >> 172.21.15.7:0/173299813 conn(0x7f52540d2cb0 msgr2=0x7f52540b7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.594+0000 7f525a764700 1 -- 172.21.15.7:0/173299813 shutdown_connections 2024-09-13T18:52:23.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.594+0000 7f525a764700 1 -- 172.21.15.7:0/173299813 wait complete. 2024-09-13T18:52:23.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.594+0000 7fc18f59e700 1 -- 172.21.15.7:0/4080973988 --> [v2:172.21.15.171:6817/2886154603,v1:172.21.15.171:6819/2886154603] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc19006dca0 con 0x7fc17006e8d0 2024-09-13T18:52:23.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.594+0000 7fbb5759e700 1 -- 172.21.15.7:0/2562126954 --> [v2:172.21.15.7:6808/2837570651,v1:172.21.15.7:6811/2837570651] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbb3c005c70 con 0x7fbb3c0015e0 2024-09-13T18:52:23.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.594+0000 7fbb4effd700 1 -- 172.21.15.7:0/2562126954 <== osd.1 v2:172.21.15.7:6808/2837570651 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbb3c005c70 con 0x7fbb3c0015e0 2024-09-13T18:52:23.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7fbb4cff9700 1 -- 172.21.15.7:0/2562126954 >> [v2:172.21.15.7:6808/2837570651,v1:172.21.15.7:6811/2837570651] conn(0x7fbb3c0015e0 msgr2=0x7fbb3c003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7fc17effd700 1 -- 172.21.15.7:0/4080973988 <== osd.7 v2:172.21.15.171:6817/2886154603 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc184004500 con 0x7fc17006e8d0 2024-09-13T18:52:23.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7fbb4cff9700 1 --2- 172.21.15.7:0/2562126954 >> [v2:172.21.15.7:6808/2837570651,v1:172.21.15.7:6811/2837570651] conn(0x7fbb3c0015e0 0x7fbb3c003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7fbb4cff9700 1 -- 172.21.15.7:0/2562126954 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbb38066730 msgr2=0x7fbb38068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7fbb4cff9700 1 --2- 172.21.15.7:0/2562126954 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbb38066730 0x7fbb38068bf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbb5808a0a0 tx=0x7fbb5000b040 comp rx=0 tx=0).stop 2024-09-13T18:52:23.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7fbb4cff9700 1 -- 172.21.15.7:0/2562126954 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb5806e080 msgr2=0x7fbb5816c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7fbb4cff9700 1 --2- 172.21.15.7:0/2562126954 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb5806e080 0x7fbb5816c610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800c500 tx=0x7fbb4800c810 comp rx=0 tx=0).stop 2024-09-13T18:52:23.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7fbb56d9d700 1 -- 172.21.15.7:0/2562126954 reap_dead start 2024-09-13T18:52:23.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7fbb4cff9700 1 -- 172.21.15.7:0/2562126954 shutdown_connections 2024-09-13T18:52:23.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7fbb4cff9700 1 -- 172.21.15.7:0/2562126954 >> 172.21.15.7:0/2562126954 conn(0x7fbb580ae590 msgr2=0x7fbb580cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7f8179ffb700 1 -- 172.21.15.7:0/2711171634 --> [v2:172.21.15.7:6800/2775757779,v1:172.21.15.7:6801/2775757779] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8184048cc0 con 0x7f817006e900 2024-09-13T18:52:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7f817bfff700 1 -- 172.21.15.7:0/2711171634 <== osd.3 v2:172.21.15.7:6800/2775757779 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8184048cc0 con 0x7f817006e900 2024-09-13T18:52:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7f8179ffb700 1 -- 172.21.15.7:0/2711171634 >> [v2:172.21.15.7:6800/2775757779,v1:172.21.15.7:6801/2775757779] conn(0x7f817006e900 msgr2=0x7f8170070d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7f8179ffb700 1 --2- 172.21.15.7:0/2711171634 >> [v2:172.21.15.7:6800/2775757779,v1:172.21.15.7:6801/2775757779] conn(0x7f817006e900 0x7f8170070d20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7f8179ffb700 1 -- 172.21.15.7:0/2711171634 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8170066760 msgr2=0x7f8170068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7f8179ffb700 1 --2- 172.21.15.7:0/2711171634 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8170066760 0x7f8170068c20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f818c04fc20 tx=0x7f818c08c040 comp rx=0 tx=0).stop 2024-09-13T18:52:23.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7f8179ffb700 1 -- 172.21.15.7:0/2711171634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840dfa90 msgr2=0x7f818404a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7f8179ffb700 1 --2- 172.21.15.7:0/2711171634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81840dfa90 0x7f818404a000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f817c016000 tx=0x7f817c004770 comp rx=0 tx=0).stop 2024-09-13T18:52:23.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7f818b7fe700 1 -- 172.21.15.7:0/2711171634 reap_dead start 2024-09-13T18:52:23.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7fa27a155700 1 -- 172.21.15.7:0/4226314050 --> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa2740717d0 con 0x7fa25806e950 2024-09-13T18:52:23.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7f8179ffb700 1 -- 172.21.15.7:0/2711171634 shutdown_connections 2024-09-13T18:52:23.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7f8179ffb700 1 -- 172.21.15.7:0/2711171634 >> 172.21.15.7:0/2711171634 conn(0x7f8184036140 msgr2=0x7f8184036520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7fbb4cff9700 1 -- 172.21.15.7:0/2562126954 shutdown_connections 2024-09-13T18:52:23.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7fbb4cff9700 1 -- 172.21.15.7:0/2562126954 wait complete. 2024-09-13T18:52:23.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7f8179ffb700 1 -- 172.21.15.7:0/2711171634 shutdown_connections 2024-09-13T18:52:23.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.598+0000 7f8179ffb700 1 -- 172.21.15.7:0/2711171634 wait complete. 2024-09-13T18:52:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fa269ffb700 1 -- 172.21.15.7:0/4226314050 <== osd.4 v2:172.21.15.171:6816/1731498321 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa26c00a700 con 0x7fa25806e950 2024-09-13T18:52:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fc18f59e700 1 -- 172.21.15.7:0/4080973988 >> [v2:172.21.15.171:6817/2886154603,v1:172.21.15.171:6819/2886154603] conn(0x7fc17006e8d0 msgr2=0x7fc170070cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fc18f59e700 1 --2- 172.21.15.7:0/4080973988 >> [v2:172.21.15.171:6817/2886154603,v1:172.21.15.171:6819/2886154603] conn(0x7fc17006e8d0 0x7fc170070cf0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fa27a155700 1 -- 172.21.15.7:0/4226314050 >> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] conn(0x7fa25806e950 msgr2=0x7fa258070d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fa27a155700 1 --2- 172.21.15.7:0/4226314050 >> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] conn(0x7fa25806e950 0x7fa258070d70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fa27a155700 1 -- 172.21.15.7:0/4226314050 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa258066730 msgr2=0x7fa258068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fa27a155700 1 --2- 172.21.15.7:0/4226314050 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa258066730 0x7fa258068bf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa264009850 tx=0x7fa264008030 comp rx=0 tx=0).stop 2024-09-13T18:52:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fa27a155700 1 -- 172.21.15.7:0/4226314050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2740cc0e0 msgr2=0x7fa2742021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fa27a155700 1 --2- 172.21.15.7:0/4226314050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2740cc0e0 0x7fa2742021c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa27000e820 tx=0x7fa27000ebe0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fa279954700 1 -- 172.21.15.7:0/4226314050 reap_dead start 2024-09-13T18:52:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fa27a155700 1 -- 172.21.15.7:0/4226314050 shutdown_connections 2024-09-13T18:52:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fa27a155700 1 -- 172.21.15.7:0/4226314050 >> 172.21.15.7:0/4226314050 conn(0x7fa2740ae590 msgr2=0x7fa2740b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fa27a155700 1 -- 172.21.15.7:0/4226314050 shutdown_connections 2024-09-13T18:52:23.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fa27a155700 1 -- 172.21.15.7:0/4226314050 wait complete. 2024-09-13T18:52:23.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fc18f59e700 1 -- 172.21.15.7:0/4080973988 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc170066730 msgr2=0x7fc170068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fc18f59e700 1 --2- 172.21.15.7:0/4080973988 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc170066730 0x7fc170068bf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc1901646f0 tx=0x7fc18800b040 comp rx=0 tx=0).stop 2024-09-13T18:52:23.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fc18f59e700 1 -- 172.21.15.7:0/4080973988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc19006d8c0 msgr2=0x7fc190149d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fc18f59e700 1 --2- 172.21.15.7:0/4080973988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc19006d8c0 0x7fc190149d20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc18000c580 tx=0x7fc18000c940 comp rx=0 tx=0).stop 2024-09-13T18:52:23.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fc18ed9d700 1 -- 172.21.15.7:0/4080973988 reap_dead start 2024-09-13T18:52:23.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fc18f59e700 1 -- 172.21.15.7:0/4080973988 shutdown_connections 2024-09-13T18:52:23.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fc18f59e700 1 -- 172.21.15.7:0/4080973988 >> 172.21.15.7:0/4080973988 conn(0x7fc1900b0e10 msgr2=0x7fc1900af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fc18f59e700 1 -- 172.21.15.7:0/4080973988 shutdown_connections 2024-09-13T18:52:23.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.602+0000 7fc18f59e700 1 -- 172.21.15.7:0/4080973988 wait complete. 2024-09-13T18:52:23.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.606+0000 7f5d4ffff700 1 -- 172.21.15.7:0/1322986134 --> [v2:172.21.15.7:6824/303159811,v1:172.21.15.7:6825/303159811] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5d48002d90 con 0x7f5d5006eb30 2024-09-13T18:52:23.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.606+0000 7f5d5a7fc700 1 -- 172.21.15.7:0/1322986134 <== osd.2 v2:172.21.15.7:6824/303159811 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5d48002d90 con 0x7f5d5006eb30 2024-09-13T18:52:23.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f5d4ffff700 1 -- 172.21.15.7:0/1322986134 >> [v2:172.21.15.7:6824/303159811,v1:172.21.15.7:6825/303159811] conn(0x7f5d5006eb30 msgr2=0x7f5d50070f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f5d4ffff700 1 --2- 172.21.15.7:0/1322986134 >> [v2:172.21.15.7:6824/303159811,v1:172.21.15.7:6825/303159811] conn(0x7f5d5006eb30 0x7f5d50070f50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f5d4ffff700 1 -- 172.21.15.7:0/1322986134 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5d50066910 msgr2=0x7f5d50068dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f5d4ffff700 1 --2- 172.21.15.7:0/1322986134 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5d50066910 0x7f5d50068dd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5d64071500 tx=0x7f5d54008040 comp rx=0 tx=0).stop 2024-09-13T18:52:23.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f5d4ffff700 1 -- 172.21.15.7:0/1322986134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6406d3f0 msgr2=0x7f5d6415f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f5d4ffff700 1 --2- 172.21.15.7:0/1322986134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6406d3f0 0x7f5d6415f1b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5d64157850 tx=0x7f5d60004770 comp rx=0 tx=0).stop 2024-09-13T18:52:23.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f5d6a264700 1 -- 172.21.15.7:0/1322986134 reap_dead start 2024-09-13T18:52:23.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f5d4ffff700 1 -- 172.21.15.7:0/1322986134 shutdown_connections 2024-09-13T18:52:23.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f5d4ffff700 1 -- 172.21.15.7:0/1322986134 >> 172.21.15.7:0/1322986134 conn(0x7f5d640b0e10 msgr2=0x7f5d6406a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f5d4ffff700 1 -- 172.21.15.7:0/1322986134 shutdown_connections 2024-09-13T18:52:23.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f5d4ffff700 1 -- 172.21.15.7:0/1322986134 wait complete. 2024-09-13T18:52:23.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f2db2e6d700 1 -- 172.21.15.7:0/2048557056 --> [v2:172.21.15.171:6808/2277626206,v1:172.21.15.171:6809/2277626206] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2d94005cd0 con 0x7f2d940015e0 2024-09-13T18:52:23.645 INFO:teuthology.orchestra.run.smithi007.stdout:60129542150 2024-09-13T18:52:23.645 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T18:52:23.646 INFO:teuthology.orchestra.run.smithi007.stdout:60129542150 2024-09-13T18:52:23.646 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T18:52:23.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f2da27fc700 1 -- 172.21.15.7:0/2048557056 <== osd.6 v2:172.21.15.171:6808/2277626206 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2d94005cd0 con 0x7f2d940015e0 2024-09-13T18:52:23.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f2db2e6d700 1 -- 172.21.15.7:0/2048557056 >> [v2:172.21.15.171:6808/2277626206,v1:172.21.15.171:6809/2277626206] conn(0x7f2d940015e0 msgr2=0x7f2d94003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f2db2e6d700 1 --2- 172.21.15.7:0/2048557056 >> [v2:172.21.15.171:6808/2277626206,v1:172.21.15.171:6809/2277626206] conn(0x7f2d940015e0 0x7f2d94003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f2db2e6d700 1 -- 172.21.15.7:0/2048557056 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2d98066be0 msgr2=0x7f2d980690a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f2db2e6d700 1 --2- 172.21.15.7:0/2048557056 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2d98066be0 0x7f2d980690a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2dac0aed30 tx=0x7f2da8009040 comp rx=0 tx=0).stop 2024-09-13T18:52:23.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f2db2e6d700 1 -- 172.21.15.7:0/2048557056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dac15dd90 msgr2=0x7f2dac1399b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f2db2e6d700 1 --2- 172.21.15.7:0/2048557056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2dac15dd90 0x7f2dac1399b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c015000 tx=0x7f2d9c004770 comp rx=0 tx=0).stop 2024-09-13T18:52:23.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.610+0000 7f2db266c700 1 -- 172.21.15.7:0/2048557056 reap_dead start 2024-09-13T18:52:23.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.614+0000 7f2db2e6d700 1 -- 172.21.15.7:0/2048557056 shutdown_connections 2024-09-13T18:52:23.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.614+0000 7f2db2e6d700 1 -- 172.21.15.7:0/2048557056 >> 172.21.15.7:0/2048557056 conn(0x7f2dac0ae590 msgr2=0x7f2dac0af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.614+0000 7f2db2e6d700 1 -- 172.21.15.7:0/2048557056 shutdown_connections 2024-09-13T18:52:23.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.614+0000 7f2db2e6d700 1 -- 172.21.15.7:0/2048557056 wait complete. 2024-09-13T18:52:23.649 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-13T18:52:23.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T18:52:23.650 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-13T18:52:23.650 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T18:52:23.651 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-13T18:52:23.651 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T18:52:23.652 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-13T18:52:23.652 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T18:52:23.653 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-13T18:52:23.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T18:52:23.655 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-13T18:52:23.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T18:52:23.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.790+0000 7f24aabdb700 1 Processor -- start 2024-09-13T18:52:23.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.790+0000 7f24aabdb700 1 -- start start 2024-09-13T18:52:23.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.790+0000 7f24aabdb700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f24a4070e80 0x7f24a40ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.790+0000 7f24aabdb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f24a40cd450 0x7f24a4162660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.790+0000 7f24aabdb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a406a5d0 con 0x7f24a406fa90 2024-09-13T18:52:23.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.790+0000 7f24aabdb700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a406bd20 con 0x7f24a40cd450 2024-09-13T18:52:23.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.790+0000 7f24aabdb700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a40cb760 con 0x7f24a4070e80 2024-09-13T18:52:23.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.790+0000 7f24aa3da700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f24a40cd450 0x7f24a4162660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.790+0000 7f24aa3da700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f24a40cd450 0x7f24a4162660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49384/0 (socket says 172.21.15.7:49384) 2024-09-13T18:52:23.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.790+0000 7f24aa3da700 1 -- 172.21.15.7:0/413818928 learned_addr learned my addr 172.21.15.7:0/413818928 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.790+0000 7f24aa3da700 1 -- 172.21.15.7:0/413818928 >> v2:172.21.15.171:3300/0 conn(0x7f24a4070e80 msgr2=0x7f24a40ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:23.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.790+0000 7f24aa3da700 1 --2- 172.21.15.7:0/413818928 >> v2:172.21.15.171:3300/0 conn(0x7f24a4070e80 0x7f24a40ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.790+0000 7f24aa3da700 1 -- 172.21.15.7:0/413818928 >> v1:172.21.15.7:6789/0 conn(0x7f24a406fa90 legacy=0x7f24a40cc800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:52:23.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.790+0000 7f24aa3da700 1 -- 172.21.15.7:0/413818928 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a40cba40 con 0x7f24a40cd450 2024-09-13T18:52:23.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.790+0000 7f24aa3da700 1 --2- 172.21.15.7:0/413818928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a40cd450 0x7f24a4162660 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f24a4071db0 tx=0x7f249c00d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577d7d5d816e75de server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.790+0000 7f24a8bd7700 1 -- 172.21.15.7:0/413818928 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c010070 con 0x7f24a40cd450 2024-09-13T18:52:23.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.790+0000 7f24a8bd7700 1 -- 172.21.15.7:0/413818928 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249c00dc70 con 0x7f24a40cd450 2024-09-13T18:52:23.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.790+0000 7f24a8bd7700 1 -- 172.21.15.7:0/413818928 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c007b30 con 0x7f24a40cd450 2024-09-13T18:52:23.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.794+0000 7f24aabdb700 1 -- 172.21.15.7:0/413818928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a40cd450 msgr2=0x7f24a4162660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.794+0000 7f24aabdb700 1 --2- 172.21.15.7:0/413818928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a40cd450 0x7f24a4162660 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f24a4071db0 tx=0x7f249c00d650 comp rx=0 tx=0).stop 2024-09-13T18:52:23.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.794+0000 7f24aabdb700 1 -- 172.21.15.7:0/413818928 shutdown_connections 2024-09-13T18:52:23.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.794+0000 7f24aabdb700 1 --2- 172.21.15.7:0/413818928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a40cd450 0x7f24a4162660 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.794+0000 7f24aabdb700 1 --2- 172.21.15.7:0/413818928 >> v2:172.21.15.171:3300/0 conn(0x7f24a4070e80 0x7f24a40ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.794+0000 7f24aabdb700 1 -- 172.21.15.7:0/413818928 >> 172.21.15.7:0/413818928 conn(0x7f24a40ae590 msgr2=0x7f24a40ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.794+0000 7f24aabdb700 1 -- 172.21.15.7:0/413818928 shutdown_connections 2024-09-13T18:52:23.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.794+0000 7f24aabdb700 1 -- 172.21.15.7:0/413818928 wait complete. 2024-09-13T18:52:23.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24aabdb700 1 Processor -- start 2024-09-13T18:52:23.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24aabdb700 1 -- start start 2024-09-13T18:52:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24aabdb700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a406fa90 0x7f24a4157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24aabdb700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24a4070e80 0x7f24a413dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24aabdb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a40cd450 0x7f24a406cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24aa3da700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a40cd450 0x7f24a406cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24aa3da700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a40cd450 0x7f24a406cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49388/0 (socket says 172.21.15.7:49388) 2024-09-13T18:52:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24a9bd9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a406fa90 0x7f24a4157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24a93d8700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24a4070e80 0x7f24a413dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24a9bd9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a406fa90 0x7f24a4157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:40810/0 (socket says 172.21.15.7:40810) 2024-09-13T18:52:23.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24a9bd9700 1 -- 172.21.15.7:0/1811120596 learned_addr learned my addr 172.21.15.7:0/1811120596 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24aabdb700 1 -- 172.21.15.7:0/1811120596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a413f1d0 con 0x7f24a40cd450 2024-09-13T18:52:23.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24aabdb700 1 -- 172.21.15.7:0/1811120596 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a4158ad0 con 0x7f24a406fa90 2024-09-13T18:52:23.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24aabdb700 1 -- 172.21.15.7:0/1811120596 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a413f420 con 0x7f24a4070e80 2024-09-13T18:52:23.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24aa3da700 1 -- 172.21.15.7:0/1811120596 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24a4070e80 msgr2=0x7f24a413dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24aa3da700 1 --2- 172.21.15.7:0/1811120596 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24a4070e80 0x7f24a413dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24aa3da700 1 -- 172.21.15.7:0/1811120596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a406fa90 msgr2=0x7f24a4157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24aa3da700 1 --2- 172.21.15.7:0/1811120596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a406fa90 0x7f24a4157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24aa3da700 1 -- 172.21.15.7:0/1811120596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249c00d050 con 0x7f24a40cd450 2024-09-13T18:52:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24a9bd9700 1 --2- 172.21.15.7:0/1811120596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a406fa90 0x7f24a4157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24aa3da700 1 --2- 172.21.15.7:0/1811120596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a40cd450 0x7f24a406cb80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f249c018000 tx=0x7f249c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f249a7fc700 1 -- 172.21.15.7:0/1811120596 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c004510 con 0x7f24a40cd450 2024-09-13T18:52:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24aabdb700 1 -- 172.21.15.7:0/1811120596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a413f700 con 0x7f24a40cd450 2024-09-13T18:52:23.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.798+0000 7f24aabdb700 1 -- 172.21.15.7:0/1811120596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a413dab0 con 0x7f24a40cd450 2024-09-13T18:52:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.802+0000 7f249a7fc700 1 -- 172.21.15.7:0/1811120596 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249c004670 con 0x7f24a40cd450 2024-09-13T18:52:23.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.802+0000 7f249a7fc700 1 -- 172.21.15.7:0/1811120596 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c008030 con 0x7f24a40cd450 2024-09-13T18:52:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.802+0000 7f249a7fc700 1 -- 172.21.15.7:0/1811120596 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f249c0081d0 con 0x7f24a40cd450 2024-09-13T18:52:23.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.802+0000 7f248ffff700 1 -- 172.21.15.7:0/1811120596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2484005310 con 0x7f24a40cd450 2024-09-13T18:52:23.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.802+0000 7f249a7fc700 1 --2- 172.21.15.7:0/1811120596 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2488066780 0x7f2488068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.802+0000 7f249a7fc700 1 -- 172.21.15.7:0/1811120596 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f249c085130 con 0x7f24a40cd450 2024-09-13T18:52:23.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.802+0000 7ff1b7fff700 1 Processor -- start 2024-09-13T18:52:23.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.802+0000 7f24a9bd9700 1 --2- 172.21.15.7:0/1811120596 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2488066780 0x7f2488068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.802+0000 7ff1b7fff700 1 -- start start 2024-09-13T18:52:23.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.802+0000 7f24a9bd9700 1 --2- 172.21.15.7:0/1811120596 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2488066780 0x7f2488068c40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2494001e00 tx=0x7f24940018c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7ff1b7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff1b815dd90 0x7ff1b80cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7ff1b7fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b8070e80 con 0x7ff1b8146480 2024-09-13T18:52:23.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7ff1b7fff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b8071160 con 0x7ff1b80cc640 2024-09-13T18:52:23.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7ff1b7fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b80ccaf0 con 0x7ff1b815dd90 2024-09-13T18:52:23.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f6521145700 1 Processor -- start 2024-09-13T18:52:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7ff1b67fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff1b815dd90 0x7ff1b80cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7ff1b6ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff1b8146480 0x7ff1b80704f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35778/0 (socket says 172.21.15.7:35778) 2024-09-13T18:52:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7ff1b6ffd700 1 -- 172.21.15.7:0/384229970 learned_addr learned my addr 172.21.15.7:0/384229970 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f6521145700 1 -- start start 2024-09-13T18:52:23.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7ff1b67fc700 1 -- 172.21.15.7:0/384229970 >> v1:172.21.15.171:6789/0 conn(0x7ff1b80cc640 legacy=0x7ff1b80ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7ff1b67fc700 1 -- 172.21.15.7:0/384229970 >> v1:172.21.15.7:6789/0 conn(0x7ff1b8146480 legacy=0x7ff1b80704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7ff1b67fc700 1 -- 172.21.15.7:0/384229970 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b8071440 con 0x7ff1b815dd90 2024-09-13T18:52:23.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7ff1b67fc700 1 --2- 172.21.15.7:0/384229970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1b815dd90 0x7ff1b80cc100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009850 tx=0x7ff1a8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109ac2d17e6dd7be server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f48fa624700 1 Processor -- start 2024-09-13T18:52:23.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f48fa624700 1 -- start start 2024-09-13T18:52:23.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f6521145700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f651c164290 0x7f651c070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f6521145700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651c162570 con 0x7f651c06d8c0 2024-09-13T18:52:23.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f6521145700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651c071240 con 0x7f651c165160 2024-09-13T18:52:23.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f6521145700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f651c162e60 con 0x7f651c164290 2024-09-13T18:52:23.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f651bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f651c06d8c0 0x7f651c164a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35794/0 (socket says 172.21.15.7:35794) 2024-09-13T18:52:23.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f651bfff700 1 -- 172.21.15.7:0/658727706 learned_addr learned my addr 172.21.15.7:0/658727706 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:23.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f6520944700 1 --2- 172.21.15.7:0/658727706 >> v2:172.21.15.171:3300/0 conn(0x7f651c164290 0x7f651c070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f7319034700 1 Processor -- start 2024-09-13T18:52:23.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f7319034700 1 -- start start 2024-09-13T18:52:23.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f651affd700 1 -- 172.21.15.7:0/658727706 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155622555 0 0) 0x7f651c071240 con 0x7f651c165160 2024-09-13T18:52:23.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f651affd700 1 -- 172.21.15.7:0/658727706 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65040035d0 con 0x7f651c165160 2024-09-13T18:52:23.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f48fa624700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f48f415dd90 0x7f48f40cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f48fa624700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f48f40cc640 0x7f48f40ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f48fa624700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f4070e80 con 0x7f48f4146480 2024-09-13T18:52:23.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f48fa624700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f40ccaf0 con 0x7f48f415dd90 2024-09-13T18:52:23.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f48fa624700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f4071160 con 0x7f48f40cc640 2024-09-13T18:52:23.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f48f8e21700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f48f415dd90 0x7f48f40cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f48f8e21700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f48f415dd90 0x7f48f40cc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49404/0 (socket says 172.21.15.7:49404) 2024-09-13T18:52:23.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f48f8e21700 1 -- 172.21.15.7:0/4241871654 learned_addr learned my addr 172.21.15.7:0/4241871654 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f48f9622700 1 --1- 172.21.15.7:0/4241871654 >> v1:172.21.15.7:6789/0 conn(0x7f48f4146480 0x7f48f40704f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35808/0 (socket says 172.21.15.7:35808) 2024-09-13T18:52:23.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f48f9e23700 1 --2- 172.21.15.7:0/4241871654 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f48f40cc640 0x7f48f40ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f651affd700 1 -- 172.21.15.7:0/658727706 <== mon.1 v1:172.21.15.171:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 926565586 0 0) 0x7f65040035d0 con 0x7f651c165160 2024-09-13T18:52:23.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f651affd700 1 -- 172.21.15.7:0/658727706 >> v2:172.21.15.171:3300/0 conn(0x7f651c164290 msgr2=0x7f651c070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:23.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.806+0000 7f651affd700 1 --2- 172.21.15.7:0/658727706 >> v2:172.21.15.171:3300/0 conn(0x7f651c164290 0x7f651c070bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f651affd700 1 -- 172.21.15.7:0/658727706 >> v1:172.21.15.7:6789/0 conn(0x7f651c06d8c0 legacy=0x7f651c164a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f48f8e21700 1 -- 172.21.15.7:0/4241871654 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f48f40cc640 msgr2=0x7f48f40ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:23.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f48f8e21700 1 --2- 172.21.15.7:0/4241871654 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f48f40cc640 0x7f48f40ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f48f8e21700 1 -- 172.21.15.7:0/4241871654 >> v1:172.21.15.7:6789/0 conn(0x7f48f4146480 legacy=0x7f48f40704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f48f8e21700 1 -- 172.21.15.7:0/4241871654 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f4071440 con 0x7f48f415dd90 2024-09-13T18:52:23.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f651affd700 1 -- 172.21.15.7:0/658727706 --> v1:172.21.15.171:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c1627a0 con 0x7f651c165160 2024-09-13T18:52:23.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f651affd700 1 -- 172.21.15.7:0/658727706 <== mon.1 v1:172.21.15.171:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7f6510001670 con 0x7f651c165160 2024-09-13T18:52:23.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f7319034700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f731415dd90 0x7f73140cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f7319034700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7314070e80 con 0x7f7314146480 2024-09-13T18:52:23.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f7319034700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7314071160 con 0x7f73140cc640 2024-09-13T18:52:23.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f7319034700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73140ccaf0 con 0x7f731415dd90 2024-09-13T18:52:23.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f48f8e21700 1 --2- 172.21.15.7:0/4241871654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f415dd90 0x7f48f40cc100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009910 tx=0x7f48e4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4cdaf6184480f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f651affd700 1 -- 172.21.15.7:0/658727706 >> v1:172.21.15.171:6789/0 conn(0x7f651c165160 legacy=0x7f651c0704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f7313fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7314146480 0x7f73140704f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35816/0 (socket says 172.21.15.7:35816) 2024-09-13T18:52:23.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f7313fff700 1 -- 172.21.15.7:0/1092989028 learned_addr learned my addr 172.21.15.7:0/1092989028 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:23.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f651affd700 1 --2- 172.21.15.7:0/658727706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6504003a70 0x7f6504003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f651affd700 1 --2- 172.21.15.7:0/658727706 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6504004870 0x7f6504004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f651affd700 1 --2- 172.21.15.7:0/658727706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6504005350 0x7f650400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f651affd700 1 -- 172.21.15.7:0/658727706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c162e60 con 0x7f6504005350 2024-09-13T18:52:23.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f651affd700 1 -- 172.21.15.7:0/658727706 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650400c250 con 0x7f6504003a70 2024-09-13T18:52:23.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f651affd700 1 -- 172.21.15.7:0/658727706 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650400c410 con 0x7f6504004870 2024-09-13T18:52:23.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f73137fe700 1 --2- 172.21.15.7:0/1092989028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f731415dd90 0x7f73140cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f6520944700 1 --2- 172.21.15.7:0/658727706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6504005350 0x7f650400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f651bfff700 1 --2- 172.21.15.7:0/658727706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6504003a70 0x7f6504003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f651b7fe700 1 --2- 172.21.15.7:0/658727706 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6504004870 0x7f6504004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f6520944700 1 -- 172.21.15.7:0/658727706 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6504004870 msgr2=0x7f6504004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:23.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f6520944700 1 --2- 172.21.15.7:0/658727706 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6504004870 0x7f6504004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f6520944700 1 -- 172.21.15.7:0/658727706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6504003a70 msgr2=0x7f6504003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:23.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f6520944700 1 --2- 172.21.15.7:0/658727706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6504003a70 0x7f6504003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f7312ffd700 1 -- 172.21.15.7:0/1092989028 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3295589685 0 0) 0x7f7314071160 con 0x7f73140cc640 2024-09-13T18:52:23.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f6520944700 1 -- 172.21.15.7:0/658727706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65040035d0 con 0x7f6504005350 2024-09-13T18:52:23.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f651b7fe700 1 --2- 172.21.15.7:0/658727706 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6504004870 0x7f6504004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:23.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f7312ffd700 1 -- 172.21.15.7:0/1092989028 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72fc0035d0 con 0x7f73140cc640 2024-09-13T18:52:23.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f6520944700 1 -- 172.21.15.7:0/658727706 reap_dead start 2024-09-13T18:52:23.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f73137fe700 1 -- 172.21.15.7:0/1092989028 >> v1:172.21.15.171:6789/0 conn(0x7f73140cc640 legacy=0x7f73140ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f73137fe700 1 -- 172.21.15.7:0/1092989028 >> v1:172.21.15.7:6789/0 conn(0x7f7314146480 legacy=0x7f73140704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f6520944700 1 --2- 172.21.15.7:0/658727706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6504005350 0x7f650400ba30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6514009f90 tx=0x7f65040058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fa253a475c088e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f73137fe700 1 -- 172.21.15.7:0/1092989028 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7314071440 con 0x7f731415dd90 2024-09-13T18:52:23.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f249a7fc700 1 -- 172.21.15.7:0/1811120596 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f249c0525e0 con 0x7f24a40cd450 2024-09-13T18:52:23.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f48ebfff700 1 -- 172.21.15.7:0/4241871654 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4011830 con 0x7f48f415dd90 2024-09-13T18:52:23.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f48ebfff700 1 -- 172.21.15.7:0/4241871654 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e4011990 con 0x7f48f415dd90 2024-09-13T18:52:23.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f48ebfff700 1 -- 172.21.15.7:0/4241871654 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e4011d40 con 0x7f48f415dd90 2024-09-13T18:52:23.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7ff1b5ffb700 1 -- 172.21.15.7:0/384229970 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8011830 con 0x7ff1b815dd90 2024-09-13T18:52:23.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7ff1b5ffb700 1 -- 172.21.15.7:0/384229970 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a8011990 con 0x7ff1b815dd90 2024-09-13T18:52:23.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7ff1b5ffb700 1 -- 172.21.15.7:0/384229970 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8011d20 con 0x7ff1b815dd90 2024-09-13T18:52:23.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f651affd700 1 -- 172.21.15.7:0/658727706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400ec30 con 0x7f6504005350 2024-09-13T18:52:23.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f651affd700 1 -- 172.21.15.7:0/658727706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651400f040 con 0x7f6504005350 2024-09-13T18:52:23.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f651affd700 1 -- 172.21.15.7:0/658727706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514014480 con 0x7f6504005350 2024-09-13T18:52:23.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7ff1b7fff700 1 -- 172.21.15.7:0/384229970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1b815dd90 msgr2=0x7ff1b80cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7ff1b7fff700 1 --2- 172.21.15.7:0/384229970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1b815dd90 0x7ff1b80cc100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009850 tx=0x7ff1a8009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:23.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7ff1b7fff700 1 -- 172.21.15.7:0/384229970 shutdown_connections 2024-09-13T18:52:23.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7ff1b7fff700 1 --2- 172.21.15.7:0/384229970 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1b815dd90 0x7ff1b80cc100 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f48fa624700 1 -- 172.21.15.7:0/4241871654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f415dd90 msgr2=0x7f48f40cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7ff1b7fff700 1 -- 172.21.15.7:0/384229970 >> 172.21.15.7:0/384229970 conn(0x7ff1b80ae590 msgr2=0x7ff1b80ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f73137fe700 1 --2- 172.21.15.7:0/1092989028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f731415dd90 0x7f73140cc100 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7308009880 tx=0x7f7308009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5addfc34dc3b6a20 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f6521145700 1 -- 172.21.15.7:0/658727706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6504005350 msgr2=0x7f650400ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f48fa624700 1 --2- 172.21.15.7:0/4241871654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f415dd90 0x7f48f40cc100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009910 tx=0x7f48e4009c20 comp rx=0 tx=0).stop 2024-09-13T18:52:23.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f6521145700 1 --2- 172.21.15.7:0/658727706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6504005350 0x7f650400ba30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6514009f90 tx=0x7f65040058a0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7ff1b7fff700 1 -- 172.21.15.7:0/384229970 shutdown_connections 2024-09-13T18:52:23.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f48fa624700 1 -- 172.21.15.7:0/4241871654 shutdown_connections 2024-09-13T18:52:23.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f48fa624700 1 --2- 172.21.15.7:0/4241871654 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f48f40cc640 0x7f48f40ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f48fa624700 1 --2- 172.21.15.7:0/4241871654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f415dd90 0x7f48f40cc100 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7f48fa624700 1 -- 172.21.15.7:0/4241871654 >> 172.21.15.7:0/4241871654 conn(0x7f48f40ae590 msgr2=0x7f48f40ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.810+0000 7ff1b7fff700 1 -- 172.21.15.7:0/384229970 wait complete. 2024-09-13T18:52:23.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f7312ffd700 1 -- 172.21.15.7:0/1092989028 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7308011830 con 0x7f731415dd90 2024-09-13T18:52:23.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f7312ffd700 1 -- 172.21.15.7:0/1092989028 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7308011990 con 0x7f731415dd90 2024-09-13T18:52:23.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f7312ffd700 1 -- 172.21.15.7:0/1092989028 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7308011d20 con 0x7f731415dd90 2024-09-13T18:52:23.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f7319034700 1 -- 172.21.15.7:0/1092989028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f731415dd90 msgr2=0x7f73140cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f7319034700 1 --2- 172.21.15.7:0/1092989028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f731415dd90 0x7f73140cc100 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7308009880 tx=0x7f7308009b90 comp rx=0 tx=0).stop 2024-09-13T18:52:23.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f7319034700 1 -- 172.21.15.7:0/1092989028 shutdown_connections 2024-09-13T18:52:23.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f7319034700 1 --2- 172.21.15.7:0/1092989028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f731415dd90 0x7f73140cc100 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f7319034700 1 -- 172.21.15.7:0/1092989028 >> 172.21.15.7:0/1092989028 conn(0x7f73140ae590 msgr2=0x7f73140ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7ff1b7fff700 1 Processor -- start 2024-09-13T18:52:23.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7ff1b7fff700 1 -- start start 2024-09-13T18:52:23.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f48fa624700 1 -- 172.21.15.7:0/4241871654 shutdown_connections 2024-09-13T18:52:23.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f02bf59e700 1 Processor -- start 2024-09-13T18:52:23.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f7319034700 1 -- 172.21.15.7:0/1092989028 shutdown_connections 2024-09-13T18:52:23.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f48fa624700 1 -- 172.21.15.7:0/4241871654 wait complete. 2024-09-13T18:52:23.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f02bf59e700 1 -- start start 2024-09-13T18:52:23.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f02bf59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02c015dd90 0x7f02c00cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f02bf59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f02c00cc640 0x7f02c00ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f02bf59e700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c0070e80 con 0x7f02c0146480 2024-09-13T18:52:23.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f02bf59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02c00ccaf0 con 0x7f02c015dd90 2024-09-13T18:52:23.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f02bf59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c0071160 con 0x7f02c00cc640 2024-09-13T18:52:23.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7ff1b7fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1b80cc640 0x7ff1b81f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7ff1b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1b8146480 0x7ff1b815c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7ff1b7fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff1b815dd90 0x7ff1b815f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7ff1b7fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b806a540 con 0x7ff1b8146480 2024-09-13T18:52:23.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7ff1b7fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b81f6d70 con 0x7ff1b80cc640 2024-09-13T18:52:23.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7ff1b7fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b81f64d0 con 0x7ff1b815dd90 2024-09-13T18:52:23.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f7319034700 1 -- 172.21.15.7:0/1092989028 wait complete. 2024-09-13T18:52:23.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f6521145700 1 -- 172.21.15.7:0/658727706 shutdown_connections 2024-09-13T18:52:23.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f6521145700 1 --2- 172.21.15.7:0/658727706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6504005350 0x7f650400ba30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f6521145700 1 -- 172.21.15.7:0/658727706 >> 172.21.15.7:0/658727706 conn(0x7f651c0b0e10 msgr2=0x7f651c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f6521145700 1 -- 172.21.15.7:0/658727706 shutdown_connections 2024-09-13T18:52:23.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f02be59c700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f02c0146480 0x7f02c00704f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:51618/0 (socket says 172.21.15.7:51618) 2024-09-13T18:52:23.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f02be59c700 1 -- 172.21.15.7:0/3895094399 learned_addr learned my addr 172.21.15.7:0/3895094399 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:23.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f02bed9d700 1 --2- 172.21.15.7:0/3895094399 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f02c00cc640 0x7f02c00ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f6521145700 1 -- 172.21.15.7:0/658727706 wait complete. 2024-09-13T18:52:23.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f02bdd9b700 1 --2- 172.21.15.7:0/3895094399 >> v2:172.21.15.7:3300/0 conn(0x7f02c015dd90 0x7f02c00cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f02bd59a700 1 -- 172.21.15.7:0/3895094399 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164313872 0 0) 0x7f02c0070e80 con 0x7f02c0146480 2024-09-13T18:52:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f02bd59a700 1 -- 172.21.15.7:0/3895094399 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02a80035d0 con 0x7f02c0146480 2024-09-13T18:52:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f02bed9d700 1 -- 172.21.15.7:0/3895094399 >> v2:172.21.15.7:3300/0 conn(0x7f02c015dd90 msgr2=0x7f02c00cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f02bed9d700 1 --2- 172.21.15.7:0/3895094399 >> v2:172.21.15.7:3300/0 conn(0x7f02c015dd90 0x7f02c00cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f02bed9d700 1 -- 172.21.15.7:0/3895094399 >> v1:172.21.15.171:6789/0 conn(0x7f02c0146480 legacy=0x7f02c00704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7f02bed9d700 1 -- 172.21.15.7:0/3895094399 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c0071440 con 0x7f02c00cc640 2024-09-13T18:52:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f02bed9d700 1 --2- 172.21.15.7:0/3895094399 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f02c00cc640 0x7f02c00ac8c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009850 tx=0x7f02b0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcc2e72c0ca5c9a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f02bd59a700 1 -- 172.21.15.7:0/3895094399 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0011830 con 0x7f02c00cc640 2024-09-13T18:52:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f02bd59a700 1 -- 172.21.15.7:0/3895094399 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b0011990 con 0x7f02c00cc640 2024-09-13T18:52:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f02bd59a700 1 -- 172.21.15.7:0/3895094399 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0011d00 con 0x7f02c00cc640 2024-09-13T18:52:23.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f7319034700 1 Processor -- start 2024-09-13T18:52:23.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f48fa624700 1 Processor -- start 2024-09-13T18:52:23.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f48fa624700 1 -- start start 2024-09-13T18:52:23.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7ff1b67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1b8146480 0x7ff1b815c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7ff1b67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1b8146480 0x7ff1b815c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49412/0 (socket says 172.21.15.7:49412) 2024-09-13T18:52:23.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7ff1b67fc700 1 -- 172.21.15.7:0/2553614325 learned_addr learned my addr 172.21.15.7:0/2553614325 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7ff1b6ffd700 1 --2- 172.21.15.7:0/2553614325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1b80cc640 0x7ff1b81f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7ff1b77fe700 1 --2- 172.21.15.7:0/2553614325 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff1b815dd90 0x7ff1b815f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7ff1b6ffd700 1 -- 172.21.15.7:0/2553614325 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff1b815dd90 msgr2=0x7ff1b815f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7ff1b6ffd700 1 --2- 172.21.15.7:0/2553614325 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff1b815dd90 0x7ff1b815f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7ff1b6ffd700 1 -- 172.21.15.7:0/2553614325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1b8146480 msgr2=0x7ff1b815c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.814+0000 7ff1b6ffd700 1 --2- 172.21.15.7:0/2553614325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1b8146480 0x7ff1b815c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7ff1b6ffd700 1 -- 172.21.15.7:0/2553614325 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a80094b0 con 0x7ff1b80cc640 2024-09-13T18:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7ff1b6ffd700 1 --2- 172.21.15.7:0/2553614325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1b80cc640 0x7ff1b81f5e40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000a380 tx=0x7ff1b000a3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f7319034700 1 -- start start 2024-09-13T18:52:23.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f6521145700 1 Processor -- start 2024-09-13T18:52:23.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f6521145700 1 -- start start 2024-09-13T18:52:23.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f02bf59e700 1 -- 172.21.15.7:0/3895094399 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f02c00cc640 msgr2=0x7f02c00ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f02bf59e700 1 --2- 172.21.15.7:0/3895094399 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f02c00cc640 0x7f02c00ac8c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f02b0009850 tx=0x7f02b0009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:23.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7ff19f7fe700 1 -- 172.21.15.7:0/2553614325 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b000ac70 con 0x7ff1b80cc640 2024-09-13T18:52:23.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7ff19f7fe700 1 -- 172.21.15.7:0/2553614325 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b0029070 con 0x7ff1b80cc640 2024-09-13T18:52:23.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7ff1b7fff700 1 -- 172.21.15.7:0/2553614325 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b81f67b0 con 0x7ff1b80cc640 2024-09-13T18:52:23.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7ff1b7fff700 1 -- 172.21.15.7:0/2553614325 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b81456f0 con 0x7ff1b80cc640 2024-09-13T18:52:23.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7ff19f7fe700 1 -- 172.21.15.7:0/2553614325 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0014080 con 0x7ff1b80cc640 2024-09-13T18:52:23.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f7319034700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73140cc640 0x7f7314144d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f7319034700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7314146480 0x7f7314151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f7319034700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731415dd90 0x7f7314152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f7319034700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7314138e70 con 0x7f731415dd90 2024-09-13T18:52:23.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f7319034700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7314139110 con 0x7f73140cc640 2024-09-13T18:52:23.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f7319034700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73141393f0 con 0x7f7314146480 2024-09-13T18:52:23.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f73137fe700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7314146480 0x7f7314151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f6521145700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6504005350 0x7f651c1509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f6521145700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f651c15c530 0x7f651c1ebae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f6521145700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f651c06b710 0x7f651c06bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f02bf59e700 1 -- 172.21.15.7:0/3895094399 shutdown_connections 2024-09-13T18:52:23.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f02bf59e700 1 --2- 172.21.15.7:0/3895094399 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f02c00cc640 0x7f02c00ac8c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f02bf59e700 1 --2- 172.21.15.7:0/3895094399 >> v2:172.21.15.7:3300/0 conn(0x7f02c015dd90 0x7f02c00cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f02bf59e700 1 -- 172.21.15.7:0/3895094399 >> 172.21.15.7:0/3895094399 conn(0x7f02c00ae590 msgr2=0x7f02c00ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f73137fe700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7314146480 0x7f7314151c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:53088/0 (socket says 172.21.15.7:53088) 2024-09-13T18:52:23.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f73137fe700 1 -- 172.21.15.7:0/1659508043 learned_addr learned my addr 172.21.15.7:0/1659508043 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f6521145700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c1588d0 con 0x7f651c15c530 2024-09-13T18:52:23.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f6521145700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c06c100 con 0x7f651c06b710 2024-09-13T18:52:23.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f6521145700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651c158580 con 0x7f6504005350 2024-09-13T18:52:23.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f651bfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6504005350 0x7f651c1509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f651bfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6504005350 0x7f651c1509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:53098/0 (socket says 172.21.15.7:53098) 2024-09-13T18:52:23.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f651bfff700 1 -- 172.21.15.7:0/470871405 learned_addr learned my addr 172.21.15.7:0/470871405 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f02bf59e700 1 -- 172.21.15.7:0/3895094399 shutdown_connections 2024-09-13T18:52:23.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f02bf59e700 1 -- 172.21.15.7:0/3895094399 wait complete. 2024-09-13T18:52:23.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f651b7fe700 1 --2- 172.21.15.7:0/470871405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f651c15c530 0x7f651c1ebae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f48fa624700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f40cc640 0x7f48f413fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f48fa624700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48f4146480 0x7f48f40b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f48fa624700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f48f415dd90 0x7f48f40b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f48fa624700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f406b320 con 0x7f48f40cc640 2024-09-13T18:52:23.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f48fa624700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f406a7a0 con 0x7f48f4146480 2024-09-13T18:52:23.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f48fa624700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f406aa80 con 0x7f48f415dd90 2024-09-13T18:52:23.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f65011b1700 1 Processor -- start 2024-09-13T18:52:23.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f73137fe700 1 -- 172.21.15.7:0/1659508043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73140cc640 msgr2=0x7f7314144d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f73137fe700 1 --2- 172.21.15.7:0/1659508043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73140cc640 0x7f7314144d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f73137fe700 1 -- 172.21.15.7:0/1659508043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731415dd90 msgr2=0x7f7314152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f73137fe700 1 --2- 172.21.15.7:0/1659508043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731415dd90 0x7f7314152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f73137fe700 1 -- 172.21.15.7:0/1659508043 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73080094b0 con 0x7f7314146480 2024-09-13T18:52:23.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f65011b1700 1 -- start start 2024-09-13T18:52:23.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7ff1b7fff700 1 -- 172.21.15.7:0/2553614325 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b806a680 con 0x7ff1b80cc640 2024-09-13T18:52:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f73137fe700 1 --2- 172.21.15.7:0/1659508043 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7314146480 0x7f7314151c40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7308013040 tx=0x7f7308004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f7310ff9700 1 -- 172.21.15.7:0/1659508043 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73080038c0 con 0x7f7314146480 2024-09-13T18:52:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f7310ff9700 1 -- 172.21.15.7:0/1659508043 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7308003a40 con 0x7f7314146480 2024-09-13T18:52:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f7310ff9700 1 -- 172.21.15.7:0/1659508043 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7308003df0 con 0x7f7314146480 2024-09-13T18:52:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f7319034700 1 -- 172.21.15.7:0/1659508043 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73141396d0 con 0x7f7314146480 2024-09-13T18:52:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f7319034700 1 -- 172.21.15.7:0/1659508043 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7314139ed0 con 0x7f7314146480 2024-09-13T18:52:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f48f9e23700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f48f415dd90 0x7f48f40b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f48f9e23700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f48f415dd90 0x7f48f40b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:53114/0 (socket says 172.21.15.7:53114) 2024-09-13T18:52:23.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f48f9e23700 1 -- 172.21.15.7:0/3786839683 learned_addr learned my addr 172.21.15.7:0/3786839683 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f65011b1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f64fc146480 0x7f64fc0704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f65011b1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64fc070e80 con 0x7f64fc0cc810 2024-09-13T18:52:23.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f65011b1700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64fc071160 con 0x7f64fc15dd90 2024-09-13T18:52:23.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f65011b1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64fc071440 con 0x7f64fc146480 2024-09-13T18:52:23.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f48f9e23700 1 -- 172.21.15.7:0/3786839683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48f4146480 msgr2=0x7f48f40b2e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:52:23.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f48f9e23700 1 --2- 172.21.15.7:0/3786839683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48f4146480 0x7f48f40b2e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f48f9e23700 1 -- 172.21.15.7:0/3786839683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f40cc640 msgr2=0x7f48f413fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f48f9e23700 1 --2- 172.21.15.7:0/3786839683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f40cc640 0x7f48f413fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f48f9e23700 1 -- 172.21.15.7:0/3786839683 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e4009570 con 0x7f48f415dd90 2024-09-13T18:52:23.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f02bf59e700 1 Processor -- start 2024-09-13T18:52:23.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7ff19f7fe700 1 -- 172.21.15.7:0/2553614325 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff1b000f070 con 0x7ff1b80cc640 2024-09-13T18:52:23.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7ff19f7fe700 1 --2- 172.21.15.7:0/2553614325 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff1a4066730 0x7ff1a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7ff19f7fe700 1 -- 172.21.15.7:0/2553614325 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1b00943d0 con 0x7ff1b80cc640 2024-09-13T18:52:23.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7ff1b67fc700 1 --2- 172.21.15.7:0/2553614325 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff1a4066730 0x7ff1a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f48f9e23700 1 --2- 172.21.15.7:0/3786839683 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f48f415dd90 0x7f48f40b3350 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f48f000eb60 tx=0x7f48f000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f6520944700 1 --2- 172.21.15.7:0/470871405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f651c06b710 0x7f651c06bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f651b7fe700 1 -- 172.21.15.7:0/470871405 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6504005350 msgr2=0x7f651c1509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f651b7fe700 1 --2- 172.21.15.7:0/470871405 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6504005350 0x7f651c1509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f651b7fe700 1 -- 172.21.15.7:0/470871405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f651c06b710 msgr2=0x7f651c06bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f651b7fe700 1 --2- 172.21.15.7:0/470871405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f651c06b710 0x7f651c06bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f651b7fe700 1 -- 172.21.15.7:0/470871405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c15ca30 con 0x7f651c15c530 2024-09-13T18:52:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.818+0000 7f651b7fe700 1 --2- 172.21.15.7:0/470871405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f651c15c530 0x7f651c1ebae0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f651000e7b0 tx=0x7f651000eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f6518ff9700 1 -- 172.21.15.7:0/470871405 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65100052e0 con 0x7f651c15c530 2024-09-13T18:52:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f6521145700 1 -- 172.21.15.7:0/470871405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651c15cd10 con 0x7f651c15c530 2024-09-13T18:52:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f6521145700 1 -- 172.21.15.7:0/470871405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c150f20 con 0x7f651c15c530 2024-09-13T18:52:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f6521145700 1 -- 172.21.15.7:0/470871405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f651c158a10 con 0x7f651c15c530 2024-09-13T18:52:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f6518ff9700 1 -- 172.21.15.7:0/470871405 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6510005440 con 0x7f651c15c530 2024-09-13T18:52:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f6518ff9700 1 -- 172.21.15.7:0/470871405 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65100038c0 con 0x7f651c15c530 2024-09-13T18:52:23.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f64fb7fe700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f64fc15dd90 0x7f64fc0cc100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:51634/0 (socket says 172.21.15.7:51634) 2024-09-13T18:52:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f64fb7fe700 1 -- 172.21.15.7:0/2995406502 learned_addr learned my addr 172.21.15.7:0/2995406502 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f02bf59e700 1 -- start start 2024-09-13T18:52:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7ff1b67fc700 1 --2- 172.21.15.7:0/2553614325 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff1a4066730 0x7ff1a4068bf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8000c00 tx=0x7ff1a801c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f48e9ffb700 1 -- 172.21.15.7:0/3786839683 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f000c850 con 0x7f48f415dd90 2024-09-13T18:52:23.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f48e9ffb700 1 -- 172.21.15.7:0/3786839683 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f000c9b0 con 0x7f48f415dd90 2024-09-13T18:52:23.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f48e9ffb700 1 -- 172.21.15.7:0/3786839683 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f000cd20 con 0x7f48f415dd90 2024-09-13T18:52:23.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f02bf59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f02c00cc640 0x7f02c006c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f02bf59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02c0146480 0x7f02c015f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f7310ff9700 1 -- 172.21.15.7:0/1659508043 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f730801b400 con 0x7f7314146480 2024-09-13T18:52:23.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f48fa624700 1 -- 172.21.15.7:0/3786839683 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f406ad60 con 0x7f48f415dd90 2024-09-13T18:52:23.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f48fa624700 1 -- 172.21.15.7:0/3786839683 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f41401e0 con 0x7f48f415dd90 2024-09-13T18:52:23.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f02bdd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02c0146480 0x7f02c015f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f02bdd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02c0146480 0x7f02c015f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49474/0 (socket says 172.21.15.7:49474) 2024-09-13T18:52:23.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f02bdd9b700 1 -- 172.21.15.7:0/2677437851 learned_addr learned my addr 172.21.15.7:0/2677437851 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f02be59c700 1 --2- 172.21.15.7:0/2677437851 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f02c00cc640 0x7f02c006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f02bf59e700 1 --2- 172.21.15.7:0/2677437851 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02c015dd90 0x7f02c015fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f02bf59e700 1 -- 172.21.15.7:0/2677437851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c01437c0 con 0x7f02c0146480 2024-09-13T18:52:23.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f02bf59e700 1 -- 172.21.15.7:0/2677437851 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c0142bb0 con 0x7f02c015dd90 2024-09-13T18:52:23.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f02bf59e700 1 -- 172.21.15.7:0/2677437851 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c0142e90 con 0x7f02c00cc640 2024-09-13T18:52:23.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.822+0000 7f7310ff9700 1 --2- 172.21.15.7:0/1659508043 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7300066730 0x7f7300068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f02bed9d700 1 --2- 172.21.15.7:0/2677437851 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02c015dd90 0x7f02c015fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f02bdd9b700 1 -- 172.21.15.7:0/2677437851 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f02c00cc640 msgr2=0x7f02c006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f02bdd9b700 1 --2- 172.21.15.7:0/2677437851 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f02c00cc640 0x7f02c006c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f02bdd9b700 1 -- 172.21.15.7:0/2677437851 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02c015dd90 msgr2=0x7f02c015fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f02bdd9b700 1 --2- 172.21.15.7:0/2677437851 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02c015dd90 0x7f02c015fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f02bdd9b700 1 -- 172.21.15.7:0/2677437851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b00094b0 con 0x7f02c0146480 2024-09-13T18:52:23.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f7310ff9700 1 -- 172.21.15.7:0/1659508043 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7308085830 con 0x7f7314146480 2024-09-13T18:52:23.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f02bdd9b700 1 --2- 172.21.15.7:0/2677437851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02c0146480 0x7f02c015f770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f02b800efc0 tx=0x7f02b800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f7313fff700 1 --2- 172.21.15.7:0/1659508043 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7300066730 0x7f7300068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f64fbfff700 1 --2- 172.21.15.7:0/2995406502 >> v2:172.21.15.7:3300/0 conn(0x7f64fc146480 0x7f64fc0704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f64faffd700 1 -- 172.21.15.7:0/2995406502 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118653654 0 0) 0x7f64fc070e80 con 0x7f64fc0cc810 2024-09-13T18:52:23.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f64faffd700 1 -- 172.21.15.7:0/2995406502 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64e40035d0 con 0x7f64fc0cc810 2024-09-13T18:52:23.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f6518ff9700 1 -- 172.21.15.7:0/470871405 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6510003aa0 con 0x7f651c15c530 2024-09-13T18:52:23.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f64fbfff700 1 -- 172.21.15.7:0/2995406502 >> v1:172.21.15.171:6789/0 conn(0x7f64fc15dd90 legacy=0x7f64fc0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f64fbfff700 1 -- 172.21.15.7:0/2995406502 >> v1:172.21.15.7:6789/0 conn(0x7f64fc0cc810 legacy=0x7f64fc0ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f64fbfff700 1 -- 172.21.15.7:0/2995406502 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc071720 con 0x7f64fc146480 2024-09-13T18:52:23.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f64fbfff700 1 --2- 172.21.15.7:0/2995406502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64fc146480 0x7f64fc0704f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009850 tx=0x7f64ec009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f95e2f21476761f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f7313fff700 1 --2- 172.21.15.7:0/1659508043 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7300066730 0x7f7300068bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7304001690 tx=0x7f7304007af0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f6518ff9700 1 --2- 172.21.15.7:0/470871405 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6508066830 0x7f6508068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f6518ff9700 1 -- 172.21.15.7:0/470871405 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65100838b0 con 0x7f651c15c530 2024-09-13T18:52:23.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f651bfff700 1 --2- 172.21.15.7:0/470871405 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6508066830 0x7f6508068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f48e9ffb700 1 -- 172.21.15.7:0/3786839683 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f48f0013470 con 0x7f48f415dd90 2024-09-13T18:52:23.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f48e9ffb700 1 --2- 172.21.15.7:0/3786839683 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f48d8066780 0x7f48d8068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f48e9ffb700 1 -- 172.21.15.7:0/3786839683 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48f00840d0 con 0x7f48f415dd90 2024-09-13T18:52:23.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f48f9622700 1 --2- 172.21.15.7:0/3786839683 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f48d8066780 0x7f48d8068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f651bfff700 1 --2- 172.21.15.7:0/470871405 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6508066830 0x7f6508068cf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f650c004eb0 tx=0x7f650c007e20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.826+0000 7f48f9622700 1 --2- 172.21.15.7:0/3786839683 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f48d8066780 0x7f48d8068c40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0012e0 tx=0x7f48ec0016e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.830+0000 7f7319034700 1 -- 172.21.15.7:0/1659508043 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f4005310 con 0x7f7314146480 2024-09-13T18:52:23.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.830+0000 7f02aeffd700 1 -- 172.21.15.7:0/2677437851 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b800cc00 con 0x7f02c0146480 2024-09-13T18:52:23.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.830+0000 7f6518ff9700 1 -- 172.21.15.7:0/470871405 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f651c158a10 con 0x7f651c15c530 2024-09-13T18:52:23.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.830+0000 7f02aeffd700 1 -- 172.21.15.7:0/2677437851 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b800f040 con 0x7f02c0146480 2024-09-13T18:52:23.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.830+0000 7f02bf59e700 1 -- 172.21.15.7:0/2677437851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c0143170 con 0x7f02c0146480 2024-09-13T18:52:23.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.830+0000 7f02aeffd700 1 -- 172.21.15.7:0/2677437851 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b8014480 con 0x7f02c0146480 2024-09-13T18:52:23.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.830+0000 7f02bf59e700 1 -- 172.21.15.7:0/2677437851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c006c960 con 0x7f02c0146480 2024-09-13T18:52:23.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.834+0000 7f64faffd700 1 -- 172.21.15.7:0/2995406502 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec004230 con 0x7f64fc146480 2024-09-13T18:52:23.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.834+0000 7f64faffd700 1 -- 172.21.15.7:0/2995406502 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ec004390 con 0x7f64fc146480 2024-09-13T18:52:23.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.834+0000 7f64faffd700 1 -- 172.21.15.7:0/2995406502 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0036c0 con 0x7f64fc146480 2024-09-13T18:52:23.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.834+0000 7ff19f7fe700 1 -- 172.21.15.7:0/2553614325 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff1b00617b0 con 0x7ff1b80cc640 2024-09-13T18:52:23.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.834+0000 7f02bf59e700 1 -- 172.21.15.7:0/2677437851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c0143900 con 0x7f02c0146480 2024-09-13T18:52:23.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.834+0000 7f02aeffd700 1 -- 172.21.15.7:0/2677437851 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f02b80145e0 con 0x7f02c0146480 2024-09-13T18:52:23.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.834+0000 7f48fa624700 1 -- 172.21.15.7:0/3786839683 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d4005310 con 0x7f48f415dd90 2024-09-13T18:52:23.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.834+0000 7f65011b1700 1 -- 172.21.15.7:0/2995406502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64fc146480 msgr2=0x7f64fc0704f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.834+0000 7f65011b1700 1 --2- 172.21.15.7:0/2995406502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64fc146480 0x7f64fc0704f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009850 tx=0x7f64ec009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:23.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.834+0000 7f65011b1700 1 -- 172.21.15.7:0/2995406502 shutdown_connections 2024-09-13T18:52:23.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.834+0000 7f65011b1700 1 --2- 172.21.15.7:0/2995406502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64fc146480 0x7f64fc0704f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.834+0000 7f65011b1700 1 -- 172.21.15.7:0/2995406502 >> 172.21.15.7:0/2995406502 conn(0x7f64fc0ae590 msgr2=0x7f64fc0ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.834+0000 7f02aeffd700 1 --2- 172.21.15.7:0/2677437851 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f02a8066be0 0x7f02a80690a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.834+0000 7f02aeffd700 1 -- 172.21.15.7:0/2677437851 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02b8084860 con 0x7f02c0146480 2024-09-13T18:52:23.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.834+0000 7f02be59c700 1 --2- 172.21.15.7:0/2677437851 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f02a8066be0 0x7f02a80690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.834+0000 7f02be59c700 1 --2- 172.21.15.7:0/2677437851 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f02a8066be0 0x7f02a80690a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f02b4007fd0 tx=0x7f02b4007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.838+0000 7f7310ff9700 1 -- 172.21.15.7:0/1659508043 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7308052d50 con 0x7f7314146480 2024-09-13T18:52:23.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.838+0000 7f65011b1700 1 -- 172.21.15.7:0/2995406502 shutdown_connections 2024-09-13T18:52:23.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.838+0000 7f65011b1700 1 -- 172.21.15.7:0/2995406502 wait complete. 2024-09-13T18:52:23.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.842+0000 7f65011b1700 1 Processor -- start 2024-09-13T18:52:23.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.842+0000 7f48e9ffb700 1 -- 172.21.15.7:0/3786839683 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f48f0051520 con 0x7f48f415dd90 2024-09-13T18:52:23.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.842+0000 7f65011b1700 1 -- start start 2024-09-13T18:52:23.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.842+0000 7f02aeffd700 1 -- 172.21.15.7:0/2677437851 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f02b8051d40 con 0x7f02c0146480 2024-09-13T18:52:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.842+0000 7f65011b1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64fc0cc810 0x7f64fc139700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.842+0000 7f64fbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64fc0cc810 0x7f64fc139700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.842+0000 7f64fbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64fc0cc810 0x7f64fc139700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49488/0 (socket says 172.21.15.7:49488) 2024-09-13T18:52:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.842+0000 7f64fbfff700 1 -- 172.21.15.7:0/574706735 learned_addr learned my addr 172.21.15.7:0/574706735 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.842+0000 7f65011b1700 1 --2- 172.21.15.7:0/574706735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64fc146480 0x7f64fc13ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.842+0000 7f65011b1700 1 --2- 172.21.15.7:0/574706735 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f64fc15dd90 0x7f64fc13bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.842+0000 7f65011b1700 1 -- 172.21.15.7:0/574706735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc1392e0 con 0x7f64fc0cc810 2024-09-13T18:52:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.842+0000 7f65011b1700 1 -- 172.21.15.7:0/574706735 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc13c4a0 con 0x7f64fc146480 2024-09-13T18:52:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.842+0000 7f65011b1700 1 -- 172.21.15.7:0/574706735 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc138c70 con 0x7f64fc15dd90 2024-09-13T18:52:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.842+0000 7f64fb7fe700 1 --2- 172.21.15.7:0/574706735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64fc146480 0x7f64fc13ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.842+0000 7f65009b0700 1 --2- 172.21.15.7:0/574706735 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f64fc15dd90 0x7f64fc13bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.846+0000 7f64fbfff700 1 -- 172.21.15.7:0/574706735 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f64fc15dd90 msgr2=0x7f64fc13bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.846+0000 7f64fbfff700 1 --2- 172.21.15.7:0/574706735 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f64fc15dd90 0x7f64fc13bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.846+0000 7f64fbfff700 1 -- 172.21.15.7:0/574706735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64fc146480 msgr2=0x7f64fc13ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.846+0000 7f64fbfff700 1 --2- 172.21.15.7:0/574706735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64fc146480 0x7f64fc13ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.846+0000 7f64fbfff700 1 -- 172.21.15.7:0/574706735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec0094b0 con 0x7f64fc0cc810 2024-09-13T18:52:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.846+0000 7f64fbfff700 1 --2- 172.21.15.7:0/574706735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64fc0cc810 0x7f64fc139700 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f64ec004000 tx=0x7f64ec004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.846+0000 7f64f8ff9700 1 -- 172.21.15.7:0/574706735 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec003e80 con 0x7f64fc0cc810 2024-09-13T18:52:23.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.846+0000 7f64f8ff9700 1 -- 172.21.15.7:0/574706735 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ec01e070 con 0x7f64fc0cc810 2024-09-13T18:52:23.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.846+0000 7f64f8ff9700 1 -- 172.21.15.7:0/574706735 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec011850 con 0x7f64fc0cc810 2024-09-13T18:52:23.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.846+0000 7f65011b1700 1 -- 172.21.15.7:0/574706735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64fc13b290 con 0x7f64fc0cc810 2024-09-13T18:52:23.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.846+0000 7f65011b1700 1 -- 172.21.15.7:0/574706735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc13c960 con 0x7f64fc0cc810 2024-09-13T18:52:23.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.850+0000 7f64f8ff9700 1 -- 172.21.15.7:0/574706735 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f64ec0119b0 con 0x7f64fc0cc810 2024-09-13T18:52:23.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.850+0000 7f64f8ff9700 1 --2- 172.21.15.7:0/574706735 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f64e4088c40 0x7f64e408b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.850+0000 7f64f8ff9700 1 -- 172.21.15.7:0/574706735 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64ec0850e0 con 0x7f64fc0cc810 2024-09-13T18:52:23.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.850+0000 7f64fb7fe700 1 --2- 172.21.15.7:0/574706735 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f64e4088c40 0x7f64e408b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.850+0000 7f64fb7fe700 1 --2- 172.21.15.7:0/574706735 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f64e4088c40 0x7f64e408b100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f64fc139fc0 tx=0x7f64f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.854+0000 7f65011b1700 1 -- 172.21.15.7:0/574706735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64dc005310 con 0x7f64fc0cc810 2024-09-13T18:52:23.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.862+0000 7f8e0951d700 1 Processor -- start 2024-09-13T18:52:23.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.862+0000 7f8e0951d700 1 -- start start 2024-09-13T18:52:23.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.862+0000 7f64f8ff9700 1 -- 172.21.15.7:0/574706735 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f64ec0525c0 con 0x7f64fc0cc810 2024-09-13T18:52:23.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.862+0000 7f8e0951d700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f8e040ce080 0x7f8e041446e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.862+0000 7f8e0951d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e04161fb0 0x7f8e0415b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.862+0000 7f8e0951d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e040b3550 con 0x7f8e04146770 2024-09-13T18:52:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.862+0000 7f8e0951d700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e040ce510 con 0x7f8e040ce080 2024-09-13T18:52:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.862+0000 7f8e0951d700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e040ce770 con 0x7f8e04161fb0 2024-09-13T18:52:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.862+0000 7f8e02ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8e04146770 0x7f8e04146b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35836/0 (socket says 172.21.15.7:35836) 2024-09-13T18:52:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.862+0000 7f8e02ffd700 1 -- 172.21.15.7:0/3927615762 learned_addr learned my addr 172.21.15.7:0/3927615762 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.862+0000 7f8e027fc700 1 --2- 172.21.15.7:0/3927615762 >> v2:172.21.15.171:3300/0 conn(0x7f8e040ce080 0x7f8e041446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.862+0000 7f8e037fe700 1 --2- 172.21.15.7:0/3927615762 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e04161fb0 0x7f8e0415b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.866+0000 7f8e01ffb700 1 -- 172.21.15.7:0/3927615762 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2739199940 0 0) 0x7f8e040b3550 con 0x7f8e04146770 2024-09-13T18:52:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.866+0000 7f8e01ffb700 1 -- 172.21.15.7:0/3927615762 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dec0035d0 con 0x7f8e04146770 2024-09-13T18:52:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.866+0000 7f8e037fe700 1 -- 172.21.15.7:0/3927615762 >> v2:172.21.15.171:3300/0 conn(0x7f8e040ce080 msgr2=0x7f8e041446e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.866+0000 7f8e037fe700 1 --2- 172.21.15.7:0/3927615762 >> v2:172.21.15.171:3300/0 conn(0x7f8e040ce080 0x7f8e041446e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.866+0000 7f8e037fe700 1 -- 172.21.15.7:0/3927615762 >> v1:172.21.15.7:6789/0 conn(0x7f8e04146770 legacy=0x7f8e04146b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.866+0000 7f8e037fe700 1 -- 172.21.15.7:0/3927615762 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e040b2c20 con 0x7f8e04161fb0 2024-09-13T18:52:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.866+0000 7f8e037fe700 1 --2- 172.21.15.7:0/3927615762 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e04161fb0 0x7f8e0415b070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009850 tx=0x7f8df8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=388da761d59a45a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.866+0000 7f8e01ffb700 1 -- 172.21.15.7:0/3927615762 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8011830 con 0x7f8e04161fb0 2024-09-13T18:52:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.866+0000 7f8e01ffb700 1 -- 172.21.15.7:0/3927615762 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df8011990 con 0x7f8e04161fb0 2024-09-13T18:52:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.866+0000 7f8e0951d700 1 -- 172.21.15.7:0/3927615762 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e04161fb0 msgr2=0x7f8e0415b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.866+0000 7f8e0951d700 1 --2- 172.21.15.7:0/3927615762 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e04161fb0 0x7f8e0415b070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009850 tx=0x7f8df8009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.870+0000 7f8e0951d700 1 -- 172.21.15.7:0/3927615762 shutdown_connections 2024-09-13T18:52:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.870+0000 7f8e0951d700 1 --2- 172.21.15.7:0/3927615762 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e04161fb0 0x7f8e0415b070 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.870+0000 7f8e0951d700 1 --2- 172.21.15.7:0/3927615762 >> v2:172.21.15.171:3300/0 conn(0x7f8e040ce080 0x7f8e041446e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.870+0000 7f8e0951d700 1 -- 172.21.15.7:0/3927615762 >> 172.21.15.7:0/3927615762 conn(0x7f8e040af5d0 msgr2=0x7f8e040af9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:23.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.870+0000 7f8e0951d700 1 -- 172.21.15.7:0/3927615762 shutdown_connections 2024-09-13T18:52:23.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.870+0000 7f8e0951d700 1 -- 172.21.15.7:0/3927615762 wait complete. 2024-09-13T18:52:23.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.874+0000 7f8e0951d700 1 Processor -- start 2024-09-13T18:52:23.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.874+0000 7f8e0951d700 1 -- start start 2024-09-13T18:52:23.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.874+0000 7f8e0951d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e040ce080 0x7f8e041feb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.874+0000 7f8e0951d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e04146770 0x7f8e041389d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.874+0000 7f8e0951d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e04161fb0 0x7f8e04202f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.874+0000 7f8e0951d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e041ff0a0 con 0x7f8e040ce080 2024-09-13T18:52:23.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.874+0000 7f8e0951d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e041616a0 con 0x7f8e04161fb0 2024-09-13T18:52:23.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.874+0000 7f8e0951d700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e04202760 con 0x7f8e04146770 2024-09-13T18:52:23.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.874+0000 7f8e02ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e040ce080 0x7f8e041feb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.874+0000 7f8e02ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e040ce080 0x7f8e041feb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49500/0 (socket says 172.21.15.7:49500) 2024-09-13T18:52:23.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.874+0000 7f8e02ffd700 1 -- 172.21.15.7:0/1606849755 learned_addr learned my addr 172.21.15.7:0/1606849755 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:23.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.874+0000 7f8e037fe700 1 --2- 172.21.15.7:0/1606849755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e04161fb0 0x7f8e04202f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.874+0000 7f8e02ffd700 1 -- 172.21.15.7:0/1606849755 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e04146770 msgr2=0x7f8e041389d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-13T18:52:23.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.874+0000 7f8e02ffd700 1 --2- 172.21.15.7:0/1606849755 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e04146770 0x7f8e041389d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.874+0000 7f8e02ffd700 1 -- 172.21.15.7:0/1606849755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e04161fb0 msgr2=0x7f8e04202f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:23.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.874+0000 7f8e02ffd700 1 --2- 172.21.15.7:0/1606849755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e04161fb0 0x7f8e04202f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:23.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.874+0000 7f8e02ffd700 1 -- 172.21.15.7:0/1606849755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df80094b0 con 0x7f8e040ce080 2024-09-13T18:52:23.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.874+0000 7f8e02ffd700 1 --2- 172.21.15.7:0/1606849755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e040ce080 0x7f8e041feb60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00cbc0 tx=0x7f8dfc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.878+0000 7f8df37fe700 1 -- 172.21.15.7:0/1606849755 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc00a820 con 0x7f8e040ce080 2024-09-13T18:52:23.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.878+0000 7f8e0951d700 1 -- 172.21.15.7:0/1606849755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e04139220 con 0x7f8e040ce080 2024-09-13T18:52:23.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.878+0000 7f8e0951d700 1 -- 172.21.15.7:0/1606849755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e041397e0 con 0x7f8e040ce080 2024-09-13T18:52:23.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.878+0000 7f8df37fe700 1 -- 172.21.15.7:0/1606849755 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dfc00a980 con 0x7f8e040ce080 2024-09-13T18:52:23.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.882+0000 7f8df37fe700 1 -- 172.21.15.7:0/1606849755 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc00acf0 con 0x7f8e040ce080 2024-09-13T18:52:23.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.882+0000 7f8df37fe700 1 -- 172.21.15.7:0/1606849755 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8dfc012400 con 0x7f8e040ce080 2024-09-13T18:52:23.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.882+0000 7f8df37fe700 1 --2- 172.21.15.7:0/1606849755 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8de8066780 0x7f8de8068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:23.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.882+0000 7f8df37fe700 1 -- 172.21.15.7:0/1606849755 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8dfc083960 con 0x7f8e040ce080 2024-09-13T18:52:23.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.882+0000 7f8e027fc700 1 --2- 172.21.15.7:0/1606849755 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8de8066780 0x7f8de8068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:23.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.882+0000 7f8e027fc700 1 --2- 172.21.15.7:0/1606849755 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8de8066780 0x7f8de8068c40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e04141900 tx=0x7f8df4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:23.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.890+0000 7f8e0951d700 1 -- 172.21.15.7:0/1606849755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de0005310 con 0x7f8e040ce080 2024-09-13T18:52:23.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:23.898+0000 7f8df37fe700 1 -- 172.21.15.7:0/1606849755 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8dfc050cc0 con 0x7f8e040ce080 2024-09-13T18:52:24.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.150+0000 7f248ffff700 1 -- 172.21.15.7:0/1811120596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2484005890 con 0x7f24a40cd450 2024-09-13T18:52:24.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.150+0000 7f249a7fc700 1 -- 172.21.15.7:0/1811120596 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f249c055c00 con 0x7f24a40cd450 2024-09-13T18:52:24.157 INFO:teuthology.orchestra.run.smithi007.stdout:60129542148 2024-09-13T18:52:24.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.158+0000 7f24aabdb700 1 -- 172.21.15.7:0/1811120596 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2488066780 msgr2=0x7f2488068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:24.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.158+0000 7f24aabdb700 1 --2- 172.21.15.7:0/1811120596 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2488066780 0x7f2488068c40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2494001e00 tx=0x7f24940018c0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.158+0000 7f24aabdb700 1 -- 172.21.15.7:0/1811120596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a40cd450 msgr2=0x7f24a406cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:24.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.158+0000 7f24aabdb700 1 --2- 172.21.15.7:0/1811120596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a40cd450 0x7f24a406cb80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f249c018000 tx=0x7f249c004af0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.158+0000 7f24aabdb700 1 -- 172.21.15.7:0/1811120596 shutdown_connections 2024-09-13T18:52:24.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.158+0000 7f24aabdb700 1 --2- 172.21.15.7:0/1811120596 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2488066780 0x7f2488068c40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.158+0000 7f24aabdb700 1 --2- 172.21.15.7:0/1811120596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24a40cd450 0x7f24a406cb80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.158+0000 7f24aabdb700 1 --2- 172.21.15.7:0/1811120596 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24a4070e80 0x7f24a413dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.158+0000 7f24aabdb700 1 --2- 172.21.15.7:0/1811120596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24a406fa90 0x7f24a4157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.158+0000 7f24aabdb700 1 -- 172.21.15.7:0/1811120596 >> 172.21.15.7:0/1811120596 conn(0x7f24a40ae590 msgr2=0x7f24a413bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.158+0000 7f24aabdb700 1 -- 172.21.15.7:0/1811120596 shutdown_connections 2024-09-13T18:52:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.158+0000 7f24aabdb700 1 -- 172.21.15.7:0/1811120596 wait complete. 2024-09-13T18:52:24.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.158+0000 7ff1b7fff700 1 -- 172.21.15.7:0/2553614325 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff1b80cc100 con 0x7ff1b80cc640 2024-09-13T18:52:24.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.158+0000 7ff19f7fe700 1 -- 172.21.15.7:0/2553614325 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff1b0064dd0 con 0x7ff1b80cc640 2024-09-13T18:52:24.169 INFO:teuthology.orchestra.run.smithi007.stdout:60129542148 2024-09-13T18:52:24.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.166+0000 7f7319034700 1 -- 172.21.15.7:0/1659508043 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f72f4005890 con 0x7f7314146480 2024-09-13T18:52:24.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.166+0000 7f7310ff9700 1 -- 172.21.15.7:0/1659508043 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f730802a030 con 0x7f7314146480 2024-09-13T18:52:24.172 INFO:teuthology.orchestra.run.smithi007.stdout:60129542148 2024-09-13T18:52:24.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.170+0000 7f6521145700 1 -- 172.21.15.7:0/470871405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f651c158a10 con 0x7f651c15c530 2024-09-13T18:52:24.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.170+0000 7f6518ff9700 1 -- 172.21.15.7:0/470871405 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f651c158a10 con 0x7f651c15c530 2024-09-13T18:52:24.174 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-13T18:52:24.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f7319034700 1 -- 172.21.15.7:0/1659508043 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7300066730 msgr2=0x7f7300068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:24.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f7319034700 1 --2- 172.21.15.7:0/1659508043 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7300066730 0x7f7300068bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7304001690 tx=0x7f7304007af0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f7319034700 1 -- 172.21.15.7:0/1659508043 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7314146480 msgr2=0x7f7314151c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:24.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f7319034700 1 --2- 172.21.15.7:0/1659508043 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7314146480 0x7f7314151c40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7308013040 tx=0x7f7308004770 comp rx=0 tx=0).stop 2024-09-13T18:52:24.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f6521145700 1 -- 172.21.15.7:0/470871405 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6508066830 msgr2=0x7f6508068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:24.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f6521145700 1 --2- 172.21.15.7:0/470871405 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6508066830 0x7f6508068cf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f650c004eb0 tx=0x7f650c007e20 comp rx=0 tx=0).stop 2024-09-13T18:52:24.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f6521145700 1 -- 172.21.15.7:0/470871405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f651c15c530 msgr2=0x7f651c1ebae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:24.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f6521145700 1 --2- 172.21.15.7:0/470871405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f651c15c530 0x7f651c1ebae0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f651000e7b0 tx=0x7f651000eb70 comp rx=0 tx=0).stop 2024-09-13T18:52:24.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f7319034700 1 -- 172.21.15.7:0/1659508043 shutdown_connections 2024-09-13T18:52:24.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f6521145700 1 -- 172.21.15.7:0/470871405 shutdown_connections 2024-09-13T18:52:24.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f6521145700 1 --2- 172.21.15.7:0/470871405 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6508066830 0x7f6508068cf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f6521145700 1 --2- 172.21.15.7:0/470871405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f651c06b710 0x7f651c06bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f6521145700 1 --2- 172.21.15.7:0/470871405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f651c15c530 0x7f651c1ebae0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f6521145700 1 --2- 172.21.15.7:0/470871405 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6504005350 0x7f651c1509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f6521145700 1 -- 172.21.15.7:0/470871405 >> 172.21.15.7:0/470871405 conn(0x7f651c0b0e10 msgr2=0x7f651c156b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:24.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f7319034700 1 --2- 172.21.15.7:0/1659508043 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7300066730 0x7f7300068bf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f7319034700 1 --2- 172.21.15.7:0/1659508043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731415dd90 0x7f7314152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f7319034700 1 --2- 172.21.15.7:0/1659508043 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7314146480 0x7f7314151c40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f7319034700 1 --2- 172.21.15.7:0/1659508043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73140cc640 0x7f7314144d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f7319034700 1 -- 172.21.15.7:0/1659508043 >> 172.21.15.7:0/1659508043 conn(0x7f73140ae590 msgr2=0x7f731415cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:24.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f6521145700 1 -- 172.21.15.7:0/470871405 shutdown_connections 2024-09-13T18:52:24.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f6521145700 1 -- 172.21.15.7:0/470871405 wait complete. 2024-09-13T18:52:24.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f7319034700 1 -- 172.21.15.7:0/1659508043 shutdown_connections 2024-09-13T18:52:24.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.174+0000 7f7319034700 1 -- 172.21.15.7:0/1659508043 wait complete. 2024-09-13T18:52:24.181 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.3 2024-09-13T18:52:24.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.178+0000 7ff1b7fff700 1 -- 172.21.15.7:0/2553614325 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff1a4066730 msgr2=0x7ff1a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:24.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.178+0000 7ff1b7fff700 1 --2- 172.21.15.7:0/2553614325 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff1a4066730 0x7ff1a4068bf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8000c00 tx=0x7ff1a801c040 comp rx=0 tx=0).stop 2024-09-13T18:52:24.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.178+0000 7ff1b7fff700 1 -- 172.21.15.7:0/2553614325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1b80cc640 msgr2=0x7ff1b81f5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:24.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.178+0000 7ff1b7fff700 1 --2- 172.21.15.7:0/2553614325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1b80cc640 0x7ff1b81f5e40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000a380 tx=0x7ff1b000a3b0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.178+0000 7ff1b7fff700 1 -- 172.21.15.7:0/2553614325 shutdown_connections 2024-09-13T18:52:24.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.178+0000 7ff1b7fff700 1 --2- 172.21.15.7:0/2553614325 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff1a4066730 0x7ff1a4068bf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.178+0000 7ff1b7fff700 1 --2- 172.21.15.7:0/2553614325 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff1b815dd90 0x7ff1b815f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.178+0000 7ff1b7fff700 1 --2- 172.21.15.7:0/2553614325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1b8146480 0x7ff1b815c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.178+0000 7ff1b7fff700 1 --2- 172.21.15.7:0/2553614325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1b80cc640 0x7ff1b81f5e40 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000a380 tx=0x7ff1b000a3b0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.178+0000 7ff1b7fff700 1 -- 172.21.15.7:0/2553614325 >> 172.21.15.7:0/2553614325 conn(0x7ff1b80ae590 msgr2=0x7ff1b80b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:24.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.178+0000 7ff1b7fff700 1 -- 172.21.15.7:0/2553614325 shutdown_connections 2024-09-13T18:52:24.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.178+0000 7ff1b7fff700 1 -- 172.21.15.7:0/2553614325 wait complete. 2024-09-13T18:52:24.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.182+0000 7f48fa624700 1 -- 172.21.15.7:0/3786839683 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f48d4005890 con 0x7f48f415dd90 2024-09-13T18:52:24.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.182+0000 7f02bf59e700 1 -- 172.21.15.7:0/2677437851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f02c00af2b0 con 0x7f02c0146480 2024-09-13T18:52:24.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.182+0000 7f48e9ffb700 1 -- 172.21.15.7:0/3786839683 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f48f0021070 con 0x7f48f415dd90 2024-09-13T18:52:24.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.182+0000 7f02aeffd700 1 -- 172.21.15.7:0/2677437851 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f02b8055360 con 0x7f02c0146480 2024-09-13T18:52:24.187 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-13T18:52:24.188 INFO:teuthology.orchestra.run.smithi007.stdout:60129542148 2024-09-13T18:52:24.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.4 2024-09-13T18:52:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f02bf59e700 1 -- 172.21.15.7:0/2677437851 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f02a8066be0 msgr2=0x7f02a80690a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f02bf59e700 1 --2- 172.21.15.7:0/2677437851 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f02a8066be0 0x7f02a80690a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f02b4007fd0 tx=0x7f02b4007de0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f02bf59e700 1 -- 172.21.15.7:0/2677437851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02c0146480 msgr2=0x7f02c015f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f02bf59e700 1 --2- 172.21.15.7:0/2677437851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02c0146480 0x7f02c015f770 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f02b800efc0 tx=0x7f02b800c0c0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f02bf59e700 1 -- 172.21.15.7:0/2677437851 shutdown_connections 2024-09-13T18:52:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f02bf59e700 1 --2- 172.21.15.7:0/2677437851 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f02a8066be0 0x7f02a80690a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f02bf59e700 1 --2- 172.21.15.7:0/2677437851 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02c015dd90 0x7f02c015fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f02bf59e700 1 --2- 172.21.15.7:0/2677437851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02c0146480 0x7f02c015f770 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f02bf59e700 1 --2- 172.21.15.7:0/2677437851 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f02c00cc640 0x7f02c006c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f02bf59e700 1 -- 172.21.15.7:0/2677437851 >> 172.21.15.7:0/2677437851 conn(0x7f02c00ae590 msgr2=0x7f02c00b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f48fa624700 1 -- 172.21.15.7:0/3786839683 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f48d8066780 msgr2=0x7f48d8068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f48fa624700 1 --2- 172.21.15.7:0/3786839683 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f48d8066780 0x7f48d8068c40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0012e0 tx=0x7f48ec0016e0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f48fa624700 1 -- 172.21.15.7:0/3786839683 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f48f415dd90 msgr2=0x7f48f40b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f48fa624700 1 --2- 172.21.15.7:0/3786839683 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f48f415dd90 0x7f48f40b3350 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f48f000eb60 tx=0x7f48f000ef20 comp rx=0 tx=0).stop 2024-09-13T18:52:24.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f02bf59e700 1 -- 172.21.15.7:0/2677437851 shutdown_connections 2024-09-13T18:52:24.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f48fa624700 1 -- 172.21.15.7:0/3786839683 shutdown_connections 2024-09-13T18:52:24.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f48fa624700 1 --2- 172.21.15.7:0/3786839683 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f48d8066780 0x7f48d8068c40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f48fa624700 1 --2- 172.21.15.7:0/3786839683 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f48f415dd90 0x7f48f40b3350 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f48fa624700 1 --2- 172.21.15.7:0/3786839683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48f4146480 0x7f48f40b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f48fa624700 1 --2- 172.21.15.7:0/3786839683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48f40cc640 0x7f48f413fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f48fa624700 1 -- 172.21.15.7:0/3786839683 >> 172.21.15.7:0/3786839683 conn(0x7f48f40ae590 msgr2=0x7f48f40af630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:24.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f02bf59e700 1 -- 172.21.15.7:0/2677437851 wait complete. 2024-09-13T18:52:24.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f48fa624700 1 -- 172.21.15.7:0/3786839683 shutdown_connections 2024-09-13T18:52:24.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.186+0000 7f48fa624700 1 -- 172.21.15.7:0/3786839683 wait complete. 2024-09-13T18:52:24.194 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.0 2024-09-13T18:52:24.199 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.1 2024-09-13T18:52:24.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.198+0000 7f65011b1700 1 -- 172.21.15.7:0/574706735 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f64dc005890 con 0x7f64fc0cc810 2024-09-13T18:52:24.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.198+0000 7f64f8ff9700 1 -- 172.21.15.7:0/574706735 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f64ec020070 con 0x7f64fc0cc810 2024-09-13T18:52:24.202 INFO:teuthology.orchestra.run.smithi007.stdout:60129542148 2024-09-13T18:52:24.203 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.2 2024-09-13T18:52:24.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.198+0000 7f65011b1700 1 -- 172.21.15.7:0/574706735 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f64e4088c40 msgr2=0x7f64e408b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:24.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.198+0000 7f65011b1700 1 --2- 172.21.15.7:0/574706735 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f64e4088c40 0x7f64e408b100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f64fc139fc0 tx=0x7f64f0009040 comp rx=0 tx=0).stop 2024-09-13T18:52:24.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.198+0000 7f65011b1700 1 -- 172.21.15.7:0/574706735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64fc0cc810 msgr2=0x7f64fc139700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:24.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.198+0000 7f65011b1700 1 --2- 172.21.15.7:0/574706735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64fc0cc810 0x7f64fc139700 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f64ec004000 tx=0x7f64ec004770 comp rx=0 tx=0).stop 2024-09-13T18:52:24.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.198+0000 7f65011b1700 1 -- 172.21.15.7:0/574706735 shutdown_connections 2024-09-13T18:52:24.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.198+0000 7f65011b1700 1 --2- 172.21.15.7:0/574706735 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f64e4088c40 0x7f64e408b100 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.198+0000 7f65011b1700 1 --2- 172.21.15.7:0/574706735 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f64fc15dd90 0x7f64fc13bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.198+0000 7f65011b1700 1 --2- 172.21.15.7:0/574706735 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64fc146480 0x7f64fc13ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.198+0000 7f65011b1700 1 --2- 172.21.15.7:0/574706735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64fc0cc810 0x7f64fc139700 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.198+0000 7f65011b1700 1 -- 172.21.15.7:0/574706735 >> 172.21.15.7:0/574706735 conn(0x7f64fc0ae590 msgr2=0x7f64fc0b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:24.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.198+0000 7f65011b1700 1 -- 172.21.15.7:0/574706735 shutdown_connections 2024-09-13T18:52:24.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.198+0000 7f65011b1700 1 -- 172.21.15.7:0/574706735 wait complete. 2024-09-13T18:52:24.207 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.5 2024-09-13T18:52:24.214 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.7 2024-09-13T18:52:24.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.218+0000 7f8e0951d700 1 -- 172.21.15.7:0/1606849755 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f8de0005890 con 0x7f8e040ce080 2024-09-13T18:52:24.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.218+0000 7f8df37fe700 1 -- 172.21.15.7:0/1606849755 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8dfc017090 con 0x7f8e040ce080 2024-09-13T18:52:24.225 INFO:teuthology.orchestra.run.smithi007.stdout:60129542148 2024-09-13T18:52:24.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.222+0000 7f8e0951d700 1 -- 172.21.15.7:0/1606849755 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8de8066780 msgr2=0x7f8de8068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:24.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.222+0000 7f8e0951d700 1 --2- 172.21.15.7:0/1606849755 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8de8066780 0x7f8de8068c40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e04141900 tx=0x7f8df4008040 comp rx=0 tx=0).stop 2024-09-13T18:52:24.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.222+0000 7f8e0951d700 1 -- 172.21.15.7:0/1606849755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e040ce080 msgr2=0x7f8e041feb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:24.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.222+0000 7f8e0951d700 1 --2- 172.21.15.7:0/1606849755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e040ce080 0x7f8e041feb60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00cbc0 tx=0x7f8dfc00cbf0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.222+0000 7f8e0951d700 1 -- 172.21.15.7:0/1606849755 shutdown_connections 2024-09-13T18:52:24.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.222+0000 7f8e0951d700 1 --2- 172.21.15.7:0/1606849755 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8de8066780 0x7f8de8068c40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.222+0000 7f8e0951d700 1 --2- 172.21.15.7:0/1606849755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8e04161fb0 0x7f8e04202f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.222+0000 7f8e0951d700 1 --2- 172.21.15.7:0/1606849755 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8e04146770 0x7f8e041389d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.222+0000 7f8e0951d700 1 --2- 172.21.15.7:0/1606849755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e040ce080 0x7f8e041feb60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:24.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.222+0000 7f8e0951d700 1 -- 172.21.15.7:0/1606849755 >> 172.21.15.7:0/1606849755 conn(0x7f8e040af5d0 msgr2=0x7f8e040cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:24.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.222+0000 7f8e0951d700 1 -- 172.21.15.7:0/1606849755 shutdown_connections 2024-09-13T18:52:24.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:24.222+0000 7f8e0951d700 1 -- 172.21.15.7:0/1606849755 wait complete. 2024-09-13T18:52:24.237 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.6 2024-09-13T18:52:25.182 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T18:52:25.191 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T18:52:25.195 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T18:52:25.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T18:52:25.203 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T18:52:25.208 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T18:52:25.214 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T18:52:25.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T18:52:25.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.326+0000 7f5c8cb9e700 1 Processor -- start 2024-09-13T18:52:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.326+0000 7f5c8cb9e700 1 -- start start 2024-09-13T18:52:25.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.326+0000 7f5c8cb9e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5c88070e80 0x7f5c880ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.326+0000 7f5c8cb9e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f5c880cd450 0x7f5c88162660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.326+0000 7f5c8cb9e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8806a5d0 con 0x7f5c8806fa90 2024-09-13T18:52:25.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.326+0000 7f5c8cb9e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c8806bd20 con 0x7f5c88070e80 2024-09-13T18:52:25.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.326+0000 7f5c8cb9e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c880cb760 con 0x7f5c880cd450 2024-09-13T18:52:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.326+0000 7f5c8659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5c8806fa90 0x7f5c880cc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35852/0 (socket says 172.21.15.7:35852) 2024-09-13T18:52:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.326+0000 7f5c8659c700 1 -- 172.21.15.7:0/3445531039 learned_addr learned my addr 172.21.15.7:0/3445531039 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.326+0000 7f5c86d9d700 1 --2- 172.21.15.7:0/3445531039 >> v2:172.21.15.171:3300/0 conn(0x7f5c880cd450 0x7f5c88162660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.326+0000 7f5c86d9d700 1 -- 172.21.15.7:0/3445531039 >> v2:172.21.15.7:3300/0 conn(0x7f5c88070e80 msgr2=0x7f5c880ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:25.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.326+0000 7f5c85d9b700 1 --2- 172.21.15.7:0/3445531039 >> v2:172.21.15.7:3300/0 conn(0x7f5c88070e80 0x7f5c880ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.326+0000 7f5c86d9d700 1 --2- 172.21.15.7:0/3445531039 >> v2:172.21.15.7:3300/0 conn(0x7f5c88070e80 0x7f5c880ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.326+0000 7f5c86d9d700 1 -- 172.21.15.7:0/3445531039 >> v1:172.21.15.7:6789/0 conn(0x7f5c8806fa90 legacy=0x7f5c880cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.326+0000 7f5c86d9d700 1 -- 172.21.15.7:0/3445531039 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c880cba40 con 0x7f5c880cd450 2024-09-13T18:52:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.330+0000 7f5c85d9b700 1 --2- 172.21.15.7:0/3445531039 >> v2:172.21.15.7:3300/0 conn(0x7f5c88070e80 0x7f5c880ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.330+0000 7f5c86d9d700 1 --2- 172.21.15.7:0/3445531039 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c880cd450 0x7f5c88162660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5c88071db0 tx=0x7f5c80009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab48eb0bd35dd25b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.330+0000 7f5c8559a700 1 -- 172.21.15.7:0/3445531039 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c80003a40 con 0x7f5c880cd450 2024-09-13T18:52:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.330+0000 7f5c8559a700 1 -- 172.21.15.7:0/3445531039 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c80003ba0 con 0x7f5c880cd450 2024-09-13T18:52:25.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.330+0000 7f5c8cb9e700 1 -- 172.21.15.7:0/3445531039 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c880cd450 msgr2=0x7f5c88162660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.330+0000 7f5c8cb9e700 1 --2- 172.21.15.7:0/3445531039 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c880cd450 0x7f5c88162660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5c88071db0 tx=0x7f5c80009f60 comp rx=0 tx=0).stop 2024-09-13T18:52:25.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.330+0000 7f5c8cb9e700 1 -- 172.21.15.7:0/3445531039 shutdown_connections 2024-09-13T18:52:25.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.330+0000 7f5c8cb9e700 1 --2- 172.21.15.7:0/3445531039 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c880cd450 0x7f5c88162660 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.330+0000 7f5c8cb9e700 1 --2- 172.21.15.7:0/3445531039 >> v2:172.21.15.7:3300/0 conn(0x7f5c88070e80 0x7f5c880ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.330+0000 7f5c8cb9e700 1 -- 172.21.15.7:0/3445531039 >> 172.21.15.7:0/3445531039 conn(0x7f5c880ae590 msgr2=0x7f5c880ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:25.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.330+0000 7f5c8cb9e700 1 -- 172.21.15.7:0/3445531039 shutdown_connections 2024-09-13T18:52:25.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.330+0000 7f5c8cb9e700 1 -- 172.21.15.7:0/3445531039 wait complete. 2024-09-13T18:52:25.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c8cb9e700 1 Processor -- start 2024-09-13T18:52:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c8cb9e700 1 -- start start 2024-09-13T18:52:25.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c8cb9e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c8806fa90 0x7f5c881401c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c8cb9e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c88070e80 0x7f5c88140700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c8cb9e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5c880cd450 0x7f5c881539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c8cb9e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c881571a0 con 0x7f5c88070e80 2024-09-13T18:52:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c8cb9e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c88156310 con 0x7f5c8806fa90 2024-09-13T18:52:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c8cb9e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c88156590 con 0x7f5c880cd450 2024-09-13T18:52:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c85d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c88070e80 0x7f5c88140700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c8659c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c8806fa90 0x7f5c881401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c86d9d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5c880cd450 0x7f5c881539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c8659c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c8806fa90 0x7f5c881401c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:40882/0 (socket says 172.21.15.7:40882) 2024-09-13T18:52:25.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c8659c700 1 -- 172.21.15.7:0/645381926 learned_addr learned my addr 172.21.15.7:0/645381926 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:25.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c8659c700 1 -- 172.21.15.7:0/645381926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5c880cd450 msgr2=0x7f5c881539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c8659c700 1 --2- 172.21.15.7:0/645381926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5c880cd450 0x7f5c881539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c8659c700 1 -- 172.21.15.7:0/645381926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c88070e80 msgr2=0x7f5c88140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c8659c700 1 --2- 172.21.15.7:0/645381926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c88070e80 0x7f5c88140700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c8659c700 1 -- 172.21.15.7:0/645381926 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8000d040 con 0x7f5c8806fa90 2024-09-13T18:52:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c85d9b700 1 --2- 172.21.15.7:0/645381926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c88070e80 0x7f5c88140700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.334+0000 7f5c8659c700 1 --2- 172.21.15.7:0/645381926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c8806fa90 0x7f5c881401c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800bfd0 tx=0x7f5c7800da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.338+0000 7f5c76ffd700 1 -- 172.21.15.7:0/645381926 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7800f8f0 con 0x7f5c8806fa90 2024-09-13T18:52:25.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.338+0000 7f5c76ffd700 1 -- 172.21.15.7:0/645381926 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7800fa50 con 0x7f5c8806fa90 2024-09-13T18:52:25.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.338+0000 7f5c76ffd700 1 -- 172.21.15.7:0/645381926 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7800fe00 con 0x7f5c8806fa90 2024-09-13T18:52:25.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.338+0000 7f5c8cb9e700 1 -- 172.21.15.7:0/645381926 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c88156870 con 0x7f5c8806fa90 2024-09-13T18:52:25.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.338+0000 7f5c8cb9e700 1 -- 172.21.15.7:0/645381926 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c88156e30 con 0x7f5c8806fa90 2024-09-13T18:52:25.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.338+0000 7f5c76ffd700 1 -- 172.21.15.7:0/645381926 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5c7800e410 con 0x7f5c8806fa90 2024-09-13T18:52:25.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.342+0000 7f5c76ffd700 1 --2- 172.21.15.7:0/645381926 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5c70066950 0x7f5c70068e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.342+0000 7f5c76ffd700 1 -- 172.21.15.7:0/645381926 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c78084cc0 con 0x7f5c8806fa90 2024-09-13T18:52:25.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.342+0000 7f5c85d9b700 1 --2- 172.21.15.7:0/645381926 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5c70066950 0x7f5c70068e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.342+0000 7f5c85d9b700 1 --2- 172.21.15.7:0/645381926 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5c70066950 0x7f5c70068e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5c88202930 tx=0x7f5c7c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.342+0000 7f5c8cb9e700 1 -- 172.21.15.7:0/645381926 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6c005310 con 0x7f5c8806fa90 2024-09-13T18:52:25.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.342+0000 7f69bd8b3700 1 Processor -- start 2024-09-13T18:52:25.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.342+0000 7f69bd8b3700 1 -- start start 2024-09-13T18:52:25.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.342+0000 7f69bd8b3700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f69b815dd90 0x7f69b80cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.342+0000 7f69bd8b3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b8070e80 con 0x7f69b8146480 2024-09-13T18:52:25.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.342+0000 7f69bd8b3700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b8071160 con 0x7f69b80cc640 2024-09-13T18:52:25.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.342+0000 7f69bd8b3700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b80ccaf0 con 0x7f69b815dd90 2024-09-13T18:52:25.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.346+0000 7f69bc8b1700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f69b8146480 0x7f69b80704f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35858/0 (socket says 172.21.15.7:35858) 2024-09-13T18:52:25.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.346+0000 7f69bc8b1700 1 -- 172.21.15.7:0/4091985522 learned_addr learned my addr 172.21.15.7:0/4091985522 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:25.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.346+0000 7f69b7fff700 1 --2- 172.21.15.7:0/4091985522 >> v2:172.21.15.171:3300/0 conn(0x7f69b815dd90 0x7f69b80cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.346+0000 7f69b7fff700 1 -- 172.21.15.7:0/4091985522 >> v1:172.21.15.171:6789/0 conn(0x7f69b80cc640 legacy=0x7f69b80ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.346+0000 7f69b7fff700 1 -- 172.21.15.7:0/4091985522 >> v1:172.21.15.7:6789/0 conn(0x7f69b8146480 legacy=0x7f69b80704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.346+0000 7f69b7fff700 1 -- 172.21.15.7:0/4091985522 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b8071440 con 0x7f69b815dd90 2024-09-13T18:52:25.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.346+0000 7f69b7fff700 1 --2- 172.21.15.7:0/4091985522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69b815dd90 0x7f69b80cc100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009850 tx=0x7f69ac009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45be4c410a089bed server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.354+0000 7f69b77fe700 1 -- 172.21.15.7:0/4091985522 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac011830 con 0x7f69b815dd90 2024-09-13T18:52:25.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.354+0000 7f69b77fe700 1 -- 172.21.15.7:0/4091985522 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ac011990 con 0x7f69b815dd90 2024-09-13T18:52:25.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.354+0000 7f69b77fe700 1 -- 172.21.15.7:0/4091985522 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac011d20 con 0x7f69b815dd90 2024-09-13T18:52:25.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.354+0000 7f69bd8b3700 1 -- 172.21.15.7:0/4091985522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69b815dd90 msgr2=0x7f69b80cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.354+0000 7f69bd8b3700 1 --2- 172.21.15.7:0/4091985522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69b815dd90 0x7f69b80cc100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009850 tx=0x7f69ac009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:25.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.354+0000 7f5c76ffd700 1 -- 172.21.15.7:0/645381926 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5c78052010 con 0x7f5c8806fa90 2024-09-13T18:52:25.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.354+0000 7f69bd8b3700 1 -- 172.21.15.7:0/4091985522 shutdown_connections 2024-09-13T18:52:25.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.354+0000 7f69bd8b3700 1 --2- 172.21.15.7:0/4091985522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69b815dd90 0x7f69b80cc100 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.354+0000 7f69bd8b3700 1 -- 172.21.15.7:0/4091985522 >> 172.21.15.7:0/4091985522 conn(0x7f69b80ae590 msgr2=0x7f69b80ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:25.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69bd8b3700 1 -- 172.21.15.7:0/4091985522 shutdown_connections 2024-09-13T18:52:25.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69bd8b3700 1 -- 172.21.15.7:0/4091985522 wait complete. 2024-09-13T18:52:25.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69bd8b3700 1 Processor -- start 2024-09-13T18:52:25.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69bd8b3700 1 -- start start 2024-09-13T18:52:25.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69bd8b3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69b80cc640 0x7f69b8140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69bd8b3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b8146480 0x7f69b806cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69bd8b3700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f69b815dd90 0x7f69b81603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69bd8b3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b80cb200 con 0x7f69b8146480 2024-09-13T18:52:25.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69bd8b3700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b815e6e0 con 0x7f69b80cc640 2024-09-13T18:52:25.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69bd8b3700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b815e390 con 0x7f69b815dd90 2024-09-13T18:52:25.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b8146480 0x7f69b806cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b8146480 0x7f69b806cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49530/0 (socket says 172.21.15.7:49530) 2024-09-13T18:52:25.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69b7fff700 1 -- 172.21.15.7:0/3652778121 learned_addr learned my addr 172.21.15.7:0/3652778121 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:25.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69bc8b1700 1 --2- 172.21.15.7:0/3652778121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69b80cc640 0x7f69b8140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69bd0b2700 1 --2- 172.21.15.7:0/3652778121 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f69b815dd90 0x7f69b81603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69b7fff700 1 -- 172.21.15.7:0/3652778121 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f69b815dd90 msgr2=0x7f69b81603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69b7fff700 1 --2- 172.21.15.7:0/3652778121 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f69b815dd90 0x7f69b81603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69b7fff700 1 -- 172.21.15.7:0/3652778121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69b80cc640 msgr2=0x7f69b8140fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69b7fff700 1 --2- 172.21.15.7:0/3652778121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69b80cc640 0x7f69b8140fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69b7fff700 1 -- 172.21.15.7:0/3652778121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ac0094b0 con 0x7f69b8146480 2024-09-13T18:52:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.358+0000 7f69b7fff700 1 --2- 172.21.15.7:0/3652778121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b8146480 0x7f69b806cc10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f69ac014040 tx=0x7f69ac004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.362+0000 7f69b57fa700 1 -- 172.21.15.7:0/3652778121 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac004610 con 0x7f69b8146480 2024-09-13T18:52:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.362+0000 7f69b57fa700 1 -- 172.21.15.7:0/3652778121 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ac01e070 con 0x7f69b8146480 2024-09-13T18:52:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.362+0000 7f69bd8b3700 1 -- 172.21.15.7:0/3652778121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b80cb450 con 0x7f69b8146480 2024-09-13T18:52:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.362+0000 7f69bd8b3700 1 -- 172.21.15.7:0/3652778121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b81414e0 con 0x7f69b8146480 2024-09-13T18:52:25.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.362+0000 7f69b57fa700 1 -- 172.21.15.7:0/3652778121 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac0038e0 con 0x7f69b8146480 2024-09-13T18:52:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.362+0000 7f69b57fa700 1 -- 172.21.15.7:0/3652778121 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f69ac003a40 con 0x7f69b8146480 2024-09-13T18:52:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.362+0000 7f69b57fa700 1 --2- 172.21.15.7:0/3652778121 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f69a4066830 0x7f69a4068cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.362+0000 7f69b57fa700 1 -- 172.21.15.7:0/3652778121 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69ac020070 con 0x7f69b8146480 2024-09-13T18:52:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.362+0000 7f69bc8b1700 1 --2- 172.21.15.7:0/3652778121 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f69a4066830 0x7f69a4068cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.366+0000 7f69bc8b1700 1 --2- 172.21.15.7:0/3652778121 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f69a4066830 0x7f69a4068cf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f69a8005930 tx=0x7f69a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.370+0000 7f69bd8b3700 1 -- 172.21.15.7:0/3652778121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6998005310 con 0x7f69b8146480 2024-09-13T18:52:25.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.378+0000 7f69b57fa700 1 -- 172.21.15.7:0/3652778121 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f69ac052b30 con 0x7f69b8146480 2024-09-13T18:52:25.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.378+0000 7f2f66a34700 1 Processor -- start 2024-09-13T18:52:25.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.378+0000 7f2f66a34700 1 -- start start 2024-09-13T18:52:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.378+0000 7f2f66a34700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2f600ce270 0x7f2f6015caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.378+0000 7f2f66a34700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6015d480 con 0x7f2f6015d030 2024-09-13T18:52:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.378+0000 7f2f66a34700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6006dad0 con 0x7f2f6006d6f0 2024-09-13T18:52:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.378+0000 7f2f66a34700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f600ae3c0 con 0x7f2f600ce270 2024-09-13T18:52:25.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.378+0000 7f71fccd3700 1 Processor -- start 2024-09-13T18:52:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.378+0000 7f2f66233700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2f6015d030 0x7f2f6006baf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35860/0 (socket says 172.21.15.7:35860) 2024-09-13T18:52:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.378+0000 7f2f66233700 1 -- 172.21.15.7:0/3401435247 learned_addr learned my addr 172.21.15.7:0/3401435247 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:25.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.378+0000 7f2f65231700 1 --2- 172.21.15.7:0/3401435247 >> v2:172.21.15.7:3300/0 conn(0x7f2f600ce270 0x7f2f6015caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.378+0000 7f2f64a30700 1 -- 172.21.15.7:0/3401435247 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781160218 0 0) 0x7f2f58001490 con 0x7f2f6015d030 2024-09-13T18:52:25.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.378+0000 7f2f64a30700 1 -- 172.21.15.7:0/3401435247 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f4c0035d0 con 0x7f2f6015d030 2024-09-13T18:52:25.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.378+0000 7f2f65231700 1 -- 172.21.15.7:0/3401435247 >> v1:172.21.15.171:6789/0 conn(0x7f2f6006d6f0 legacy=0x7f2f600adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.378+0000 7f2f64a30700 1 -- 172.21.15.7:0/3401435247 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960088881 0 0) 0x7f2f6006dad0 con 0x7f2f6006d6f0 2024-09-13T18:52:25.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.378+0000 7f2f65231700 1 -- 172.21.15.7:0/3401435247 >> v1:172.21.15.7:6789/0 conn(0x7f2f6015d030 legacy=0x7f2f6006baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.378+0000 7f2f65231700 1 -- 172.21.15.7:0/3401435247 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f600705b0 con 0x7f2f600ce270 2024-09-13T18:52:25.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7f2f65231700 1 --2- 172.21.15.7:0/3401435247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f600ce270 0x7f2f6015caf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000aa20 tx=0x7f2f5000ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d603f6a9e6ff06 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7f2f64a30700 1 -- 172.21.15.7:0/3401435247 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5000cb30 con 0x7f2f600ce270 2024-09-13T18:52:25.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7f2f64a30700 1 -- 172.21.15.7:0/3401435247 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5000cc90 con 0x7f2f600ce270 2024-09-13T18:52:25.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7f2f64a30700 1 -- 172.21.15.7:0/3401435247 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f50018400 con 0x7f2f600ce270 2024-09-13T18:52:25.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7f2f66a34700 1 -- 172.21.15.7:0/3401435247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f600ce270 msgr2=0x7f2f6015caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7f2f66a34700 1 --2- 172.21.15.7:0/3401435247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f600ce270 0x7f2f6015caf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000aa20 tx=0x7f2f5000ad30 comp rx=0 tx=0).stop 2024-09-13T18:52:25.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7fe6e0a0b700 1 Processor -- start 2024-09-13T18:52:25.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7f2f66a34700 1 -- 172.21.15.7:0/3401435247 shutdown_connections 2024-09-13T18:52:25.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7f71fccd3700 1 -- start start 2024-09-13T18:52:25.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7fe6e0a0b700 1 -- start start 2024-09-13T18:52:25.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7f2f66a34700 1 --2- 172.21.15.7:0/3401435247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f600ce270 0x7f2f6015caf0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7f2f66a34700 1 -- 172.21.15.7:0/3401435247 >> 172.21.15.7:0/3401435247 conn(0x7f2f600b0e10 msgr2=0x7f2f600b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7fe6e0a0b700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fe6dc0712b0 0x7fe6dc159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7fe6e0a0b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6dc15ac50 con 0x7fe6dc15a460 2024-09-13T18:52:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7fe6e0a0b700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6dc0cb3f0 con 0x7fe6dc1573e0 2024-09-13T18:52:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7fe6e0a0b700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6dc0cc1b0 con 0x7fe6dc0712b0 2024-09-13T18:52:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7fe6dbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe6dc15a460 0x7fe6dc06c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35868/0 (socket says 172.21.15.7:35868) 2024-09-13T18:52:25.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7fe6dbfff700 1 -- 172.21.15.7:0/1627989654 learned_addr learned my addr 172.21.15.7:0/1627989654 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:25.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7f2f66a34700 1 -- 172.21.15.7:0/3401435247 shutdown_connections 2024-09-13T18:52:25.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7f2f66a34700 1 -- 172.21.15.7:0/3401435247 wait complete. 2024-09-13T18:52:25.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7fe6daffd700 1 --2- 172.21.15.7:0/1627989654 >> v2:172.21.15.171:3300/0 conn(0x7fe6dc0712b0 0x7fe6dc159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7f71fccd3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71f8144340 0x7f71f806d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7f71fccd3700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f71f806bdb0 0x7f71f80cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7fe6da7fc700 1 -- 172.21.15.7:0/1627989654 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345080207 0 0) 0x7fe6dc15ac50 con 0x7fe6dc15a460 2024-09-13T18:52:25.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7fe6da7fc700 1 -- 172.21.15.7:0/1627989654 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6c40035d0 con 0x7fe6dc15a460 2024-09-13T18:52:25.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7fe6daffd700 1 -- 172.21.15.7:0/1627989654 >> v1:172.21.15.171:6789/0 conn(0x7fe6dc1573e0 legacy=0x7fe6dc070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7f71fccd3700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71f815f140 0x7f71f80cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7fe6daffd700 1 -- 172.21.15.7:0/1627989654 >> v1:172.21.15.7:6789/0 conn(0x7fe6dc15a460 legacy=0x7fe6dc06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7fe6daffd700 1 -- 172.21.15.7:0/1627989654 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6dc0cb6d0 con 0x7fe6dc0712b0 2024-09-13T18:52:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7f71fccd3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f8144720 con 0x7f71f8144340 2024-09-13T18:52:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.382+0000 7f71f659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71f8144340 0x7f71f806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71f659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f71f8144340 0x7f71f806d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49550/0 (socket says 172.21.15.7:49550) 2024-09-13T18:52:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71f659c700 1 -- 172.21.15.7:0/2471268895 learned_addr learned my addr 172.21.15.7:0/2471268895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71fccd3700 1 -- 172.21.15.7:0/2471268895 --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f806dd40 con 0x7f71f806bdb0 2024-09-13T18:52:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71fccd3700 1 -- 172.21.15.7:0/2471268895 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f806da60 con 0x7f71f815f140 2024-09-13T18:52:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71f6d9d700 1 --2- 172.21.15.7:0/2471268895 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71f815f140 0x7f71f80cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71f5d9b700 1 --2- 172.21.15.7:0/2471268895 >> v2:172.21.15.171:3300/0 conn(0x7f71f806bdb0 0x7f71f80cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71f659c700 1 -- 172.21.15.7:0/2471268895 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71f815f140 msgr2=0x7f71f80cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71f659c700 1 --2- 172.21.15.7:0/2471268895 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71f815f140 0x7f71f80cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71f659c700 1 -- 172.21.15.7:0/2471268895 >> v2:172.21.15.171:3300/0 conn(0x7f71f806bdb0 msgr2=0x7f71f80cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2778ce5700 1 Processor -- start 2024-09-13T18:52:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71f659c700 1 --2- 172.21.15.7:0/2471268895 >> v2:172.21.15.171:3300/0 conn(0x7f71f806bdb0 0x7f71f80cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71f659c700 1 -- 172.21.15.7:0/2471268895 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f815e660 con 0x7f71f8144340 2024-09-13T18:52:25.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7fe6daffd700 1 --2- 172.21.15.7:0/1627989654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6dc0712b0 0x7fe6dc159f20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4009d90 tx=0x7fe6d400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1eb58be3c749acb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2778ce5700 1 -- start start 2024-09-13T18:52:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71f659c700 1 --2- 172.21.15.7:0/2471268895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f8144340 0x7f71f806d480 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f71f80ce500 tx=0x7f71f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756ad4c51bc410a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7fe6da7fc700 1 -- 172.21.15.7:0/1627989654 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d400e070 con 0x7fe6dc0712b0 2024-09-13T18:52:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7fe6da7fc700 1 -- 172.21.15.7:0/1627989654 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d400bb20 con 0x7fe6dc0712b0 2024-09-13T18:52:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7fe6da7fc700 1 -- 172.21.15.7:0/1627989654 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4012430 con 0x7fe6dc0712b0 2024-09-13T18:52:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71f559a700 1 -- 172.21.15.7:0/2471268895 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f000e070 con 0x7f71f8144340 2024-09-13T18:52:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71f559a700 1 -- 172.21.15.7:0/2471268895 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f000b9c0 con 0x7f71f8144340 2024-09-13T18:52:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71f559a700 1 -- 172.21.15.7:0/2471268895 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f000bd70 con 0x7f71f8144340 2024-09-13T18:52:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2778ce5700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f27740cd620 0x7f2774163670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2778ce5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f277406a5d0 con 0x7f277406fa90 2024-09-13T18:52:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2778ce5700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f277406bd20 con 0x7f2774070e80 2024-09-13T18:52:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2778ce5700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27740cdab0 con 0x7f27740cd620 2024-09-13T18:52:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f27737fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f277406fa90 0x7f27740cc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35870/0 (socket says 172.21.15.7:35870) 2024-09-13T18:52:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f27737fe700 1 -- 172.21.15.7:0/2326584067 learned_addr learned my addr 172.21.15.7:0/2326584067 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2773fff700 1 --2- 172.21.15.7:0/2326584067 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f27740cd620 0x7f2774163670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2f66a34700 1 Processor -- start 2024-09-13T18:52:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7fe6e0a0b700 1 -- 172.21.15.7:0/1627989654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6dc0712b0 msgr2=0x7fe6dc159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7fe6e0a0b700 1 --2- 172.21.15.7:0/1627989654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6dc0712b0 0x7fe6dc159f20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4009d90 tx=0x7fe6d400b330 comp rx=0 tx=0).stop 2024-09-13T18:52:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2f66a34700 1 -- start start 2024-09-13T18:52:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7fe6e0a0b700 1 -- 172.21.15.7:0/1627989654 shutdown_connections 2024-09-13T18:52:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7fe6e0a0b700 1 --2- 172.21.15.7:0/1627989654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6dc0712b0 0x7fe6dc159f20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7fe6e0a0b700 1 -- 172.21.15.7:0/1627989654 >> 172.21.15.7:0/1627989654 conn(0x7fe6dc0b0e10 msgr2=0x7fe6dc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71fccd3700 1 -- 172.21.15.7:0/2471268895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f8144340 msgr2=0x7f71f806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2773fff700 1 -- 172.21.15.7:0/2326584067 >> v1:172.21.15.171:6789/0 conn(0x7f2774070e80 legacy=0x7f27740ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2773fff700 1 -- 172.21.15.7:0/2326584067 >> v1:172.21.15.7:6789/0 conn(0x7f277406fa90 legacy=0x7f27740cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2773fff700 1 -- 172.21.15.7:0/2326584067 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27740cb760 con 0x7f27740cd620 2024-09-13T18:52:25.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71fccd3700 1 --2- 172.21.15.7:0/2471268895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f8144340 0x7f71f806d480 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f71f80ce500 tx=0x7f71f000b330 comp rx=0 tx=0).stop 2024-09-13T18:52:25.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71fccd3700 1 -- 172.21.15.7:0/2471268895 shutdown_connections 2024-09-13T18:52:25.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7fe6e0a0b700 1 -- 172.21.15.7:0/1627989654 shutdown_connections 2024-09-13T18:52:25.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71fccd3700 1 --2- 172.21.15.7:0/2471268895 >> v2:172.21.15.171:3300/0 conn(0x7f71f806bdb0 0x7f71f80cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71fccd3700 1 --2- 172.21.15.7:0/2471268895 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71f815f140 0x7f71f80cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71fccd3700 1 --2- 172.21.15.7:0/2471268895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f8144340 0x7f71f806d480 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71fccd3700 1 -- 172.21.15.7:0/2471268895 >> 172.21.15.7:0/2471268895 conn(0x7f71f80b0e10 msgr2=0x7f71f80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:25.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71fccd3700 1 -- 172.21.15.7:0/2471268895 shutdown_connections 2024-09-13T18:52:25.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f71fccd3700 1 -- 172.21.15.7:0/2471268895 wait complete. 2024-09-13T18:52:25.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7fe6e0a0b700 1 -- 172.21.15.7:0/1627989654 wait complete. 2024-09-13T18:52:25.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2773fff700 1 --2- 172.21.15.7:0/2326584067 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f27740cd620 0x7f2774163670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f276c00aed0 tx=0x7f276c00c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7c7295f09c162c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2f66a34700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f6006d6f0 0x7f2f600abd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2f66a34700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2f600ce270 0x7f2f60153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2f66a34700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f6015d030 0x7f2f60154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2f66a34700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f601fa970 con 0x7f2f6015d030 2024-09-13T18:52:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2f66233700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f6015d030 0x7f2f60154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2f66233700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f6015d030 0x7f2f60154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49564/0 (socket says 172.21.15.7:49564) 2024-09-13T18:52:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2f66233700 1 -- 172.21.15.7:0/860921042 learned_addr learned my addr 172.21.15.7:0/860921042 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2f66a34700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f60153110 con 0x7f2f6006d6f0 2024-09-13T18:52:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2f66a34700 1 -- 172.21.15.7:0/860921042 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f601fa010 con 0x7f2f600ce270 2024-09-13T18:52:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2f65231700 1 --2- 172.21.15.7:0/860921042 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2f600ce270 0x7f2f60153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.386+0000 7f2f65a32700 1 --2- 172.21.15.7:0/860921042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f6006d6f0 0x7f2f600abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f2f66233700 1 -- 172.21.15.7:0/860921042 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2f600ce270 msgr2=0x7f2f60153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f2f66233700 1 --2- 172.21.15.7:0/860921042 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2f600ce270 0x7f2f60153b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f2f66233700 1 -- 172.21.15.7:0/860921042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f6006d6f0 msgr2=0x7f2f600abd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f2f66233700 1 --2- 172.21.15.7:0/860921042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f6006d6f0 0x7f2f600abd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f2f66233700 1 -- 172.21.15.7:0/860921042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5000a650 con 0x7f2f6015d030 2024-09-13T18:52:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f2f66233700 1 --2- 172.21.15.7:0/860921042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f6015d030 0x7f2f60154070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800cae0 tx=0x7f2f5800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6e0a0b700 1 Processor -- start 2024-09-13T18:52:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71fccd3700 1 Processor -- start 2024-09-13T18:52:25.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f27727fc700 1 -- 172.21.15.7:0/2326584067 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c003c40 con 0x7f27740cd620 2024-09-13T18:52:25.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f27727fc700 1 -- 172.21.15.7:0/2326584067 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c003da0 con 0x7f27740cd620 2024-09-13T18:52:25.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f27727fc700 1 -- 172.21.15.7:0/2326584067 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c00cae0 con 0x7f27740cd620 2024-09-13T18:52:25.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6e0a0b700 1 -- start start 2024-09-13T18:52:25.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f2778ce5700 1 -- 172.21.15.7:0/2326584067 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f27740cd620 msgr2=0x7f2774163670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f2778ce5700 1 --2- 172.21.15.7:0/2326584067 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f27740cd620 0x7f2774163670 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f276c00aed0 tx=0x7f276c00c330 comp rx=0 tx=0).stop 2024-09-13T18:52:25.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71fccd3700 1 -- start start 2024-09-13T18:52:25.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f2778ce5700 1 -- 172.21.15.7:0/2326584067 shutdown_connections 2024-09-13T18:52:25.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f2778ce5700 1 --2- 172.21.15.7:0/2326584067 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f27740cd620 0x7f2774163670 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f2778ce5700 1 -- 172.21.15.7:0/2326584067 >> 172.21.15.7:0/2326584067 conn(0x7f27740ae590 msgr2=0x7f27740ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:25.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71fccd3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71f806bdb0 0x7f71f815a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71fccd3700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71f8144340 0x7f71f815d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71fccd3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f815f140 0x7f71f815b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71fccd3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f813f1d0 con 0x7f71f815f140 2024-09-13T18:52:25.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71fccd3700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f815de90 con 0x7f71f806bdb0 2024-09-13T18:52:25.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6e0a0b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe6dc0712b0 0x7fe6dc15bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71fccd3700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f815e170 con 0x7f71f8144340 2024-09-13T18:52:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6e0a0b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6dc1573e0 0x7fe6dc158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6e0a0b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6dc15a460 0x7fe6dc1442c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6e0a0b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc148330 con 0x7fe6dc15a460 2024-09-13T18:52:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6e0a0b700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc15cda0 con 0x7fe6dc1573e0 2024-09-13T18:52:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6e0a0b700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc159210 con 0x7fe6dc0712b0 2024-09-13T18:52:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6dc15a460 0x7fe6dc1442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6dc15a460 0x7fe6dc1442c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49572/0 (socket says 172.21.15.7:49572) 2024-09-13T18:52:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6dbfff700 1 -- 172.21.15.7:0/3998623249 learned_addr learned my addr 172.21.15.7:0/3998623249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6db7fe700 1 --2- 172.21.15.7:0/3998623249 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe6dc0712b0 0x7fe6dc15bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6daffd700 1 --2- 172.21.15.7:0/3998623249 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6dc1573e0 0x7fe6dc158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6dbfff700 1 -- 172.21.15.7:0/3998623249 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe6dc0712b0 msgr2=0x7fe6dc15bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6dbfff700 1 --2- 172.21.15.7:0/3998623249 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe6dc0712b0 0x7fe6dc15bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6dbfff700 1 -- 172.21.15.7:0/3998623249 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6dc1573e0 msgr2=0x7fe6dc158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6dbfff700 1 --2- 172.21.15.7:0/3998623249 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6dc1573e0 0x7fe6dc158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6dbfff700 1 -- 172.21.15.7:0/3998623249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d4009a70 con 0x7fe6dc15a460 2024-09-13T18:52:25.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71f6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f815f140 0x7f71f815b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71f6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f815f140 0x7f71f815b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49570/0 (socket says 172.21.15.7:49570) 2024-09-13T18:52:25.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71f5d9b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71f8144340 0x7f71f815d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71f6d9d700 1 -- 172.21.15.7:0/2369548083 learned_addr learned my addr 172.21.15.7:0/2369548083 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:25.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71f659c700 1 --2- 172.21.15.7:0/2369548083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71f806bdb0 0x7f71f815a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71f6d9d700 1 -- 172.21.15.7:0/2369548083 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71f8144340 msgr2=0x7f71f815d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71f6d9d700 1 --2- 172.21.15.7:0/2369548083 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71f8144340 0x7f71f815d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71f6d9d700 1 -- 172.21.15.7:0/2369548083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71f806bdb0 msgr2=0x7f71f815a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71f6d9d700 1 --2- 172.21.15.7:0/2369548083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71f806bdb0 0x7f71f815a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71f6d9d700 1 -- 172.21.15.7:0/2369548083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f0009b30 con 0x7f71f815f140 2024-09-13T18:52:25.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6dbfff700 1 --2- 172.21.15.7:0/3998623249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6dc15a460 0x7fe6dc1442c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000c4d0 tx=0x7fe6d000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71f6d9d700 1 --2- 172.21.15.7:0/2369548083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f815f140 0x7f71f815b740 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f71ec012680 tx=0x7f71ec012990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6c3fff700 1 -- 172.21.15.7:0/3998623249 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0005d90 con 0x7fe6dc15a460 2024-09-13T18:52:25.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71e6ffd700 1 -- 172.21.15.7:0/2369548083 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec019070 con 0x7f71f815f140 2024-09-13T18:52:25.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6c3fff700 1 -- 172.21.15.7:0/3998623249 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d00070d0 con 0x7fe6dc15a460 2024-09-13T18:52:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71e6ffd700 1 -- 172.21.15.7:0/2369548083 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ec01f420 con 0x7f71f815f140 2024-09-13T18:52:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6c3fff700 1 -- 172.21.15.7:0/3998623249 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0007440 con 0x7fe6dc15a460 2024-09-13T18:52:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71e6ffd700 1 -- 172.21.15.7:0/2369548083 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec01f7b0 con 0x7f71f815f140 2024-09-13T18:52:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6e0a0b700 1 -- 172.21.15.7:0/3998623249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6dc15c500 con 0x7fe6dc15a460 2024-09-13T18:52:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7fe6e0a0b700 1 -- 172.21.15.7:0/3998623249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6dc15cac0 con 0x7fe6dc15a460 2024-09-13T18:52:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71fccd3700 1 -- 172.21.15.7:0/2369548083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f813f420 con 0x7f71f815f140 2024-09-13T18:52:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f71fccd3700 1 -- 172.21.15.7:0/2369548083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f815a770 con 0x7f71f815f140 2024-09-13T18:52:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f2f567fc700 1 -- 172.21.15.7:0/860921042 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f58007480 con 0x7f2f6015d030 2024-09-13T18:52:25.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f2778ce5700 1 -- 172.21.15.7:0/2326584067 shutdown_connections 2024-09-13T18:52:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f2f66a34700 1 -- 172.21.15.7:0/860921042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f601fa2b0 con 0x7f2f6015d030 2024-09-13T18:52:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f2f66a34700 1 -- 172.21.15.7:0/860921042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f600ac280 con 0x7f2f6015d030 2024-09-13T18:52:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f2f567fc700 1 -- 172.21.15.7:0/860921042 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f580075e0 con 0x7f2f6015d030 2024-09-13T18:52:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.389+0000 7f2f567fc700 1 -- 172.21.15.7:0/860921042 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f58007970 con 0x7f2f6015d030 2024-09-13T18:52:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f2778ce5700 1 -- 172.21.15.7:0/2326584067 wait complete. 2024-09-13T18:52:25.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7fe6c3fff700 1 -- 172.21.15.7:0/3998623249 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe6d00075a0 con 0x7fe6dc15a460 2024-09-13T18:52:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f43f52b2700 1 Processor -- start 2024-09-13T18:52:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f43f52b2700 1 -- start start 2024-09-13T18:52:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f2778ce5700 1 Processor -- start 2024-09-13T18:52:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f2778ce5700 1 -- start start 2024-09-13T18:52:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f2778ce5700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f277406fa90 0x7f277415be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f2778ce5700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2774070e80 0x7f277413d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f2778ce5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f277415a6e0 0x7f27741ee740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f2778ce5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277415ba40 con 0x7f277415a6e0 2024-09-13T18:52:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f2778ce5700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27741eec80 con 0x7f277406fa90 2024-09-13T18:52:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f2778ce5700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277415ab70 con 0x7f2774070e80 2024-09-13T18:52:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f2773fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f277415a6e0 0x7f27741ee740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f2773fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f277415a6e0 0x7f27741ee740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49584/0 (socket says 172.21.15.7:49584) 2024-09-13T18:52:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f27737fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f277406fa90 0x7f277415be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f2773fff700 1 -- 172.21.15.7:0/2727580228 learned_addr learned my addr 172.21.15.7:0/2727580228 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7fe6c3fff700 1 --2- 172.21.15.7:0/3998623249 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe6c4066b40 0x7fe6c4069000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7fe6e0a0b700 1 -- 172.21.15.7:0/3998623249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6bc005310 con 0x7fe6dc15a460 2024-09-13T18:52:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7fe6c3fff700 1 -- 172.21.15.7:0/3998623249 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6d00834d0 con 0x7fe6dc15a460 2024-09-13T18:52:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f71e6ffd700 1 -- 172.21.15.7:0/2369548083 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f71ec01d070 con 0x7f71f815f140 2024-09-13T18:52:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f71e6ffd700 1 --2- 172.21.15.7:0/2369548083 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f71d8066730 0x7f71d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f71e6ffd700 1 -- 172.21.15.7:0/2369548083 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71ec08c350 con 0x7f71f815f140 2024-09-13T18:52:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f71f659c700 1 --2- 172.21.15.7:0/2369548083 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f71d8066730 0x7f71d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f43f52b2700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f43f015bb70 0x7f43f015e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f43f52b2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f43f006add0 0x7f43f006b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f43f52b2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f0155580 con 0x7f43f015b790 2024-09-13T18:52:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f43f52b2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f0071430 con 0x7f43f006add0 2024-09-13T18:52:25.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f43f52b2700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f015eb80 con 0x7f43f015bb70 2024-09-13T18:52:25.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.393+0000 7f2f567fc700 1 -- 172.21.15.7:0/860921042 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2f5801b3b0 con 0x7f2f6015d030 2024-09-13T18:52:25.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7fe6db7fe700 1 --2- 172.21.15.7:0/3998623249 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe6c4066b40 0x7fe6c4069000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f43ee7fc700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f43f015bb70 0x7f43f015e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f2773fff700 1 -- 172.21.15.7:0/2727580228 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2774070e80 msgr2=0x7f277413d3d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T18:52:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f2773fff700 1 --2- 172.21.15.7:0/2727580228 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2774070e80 0x7f277413d3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f2773fff700 1 -- 172.21.15.7:0/2727580228 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f277406fa90 msgr2=0x7f277415be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f43ee7fc700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f43f015bb70 0x7f43f015e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:40948/0 (socket says 172.21.15.7:40948) 2024-09-13T18:52:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f2773fff700 1 --2- 172.21.15.7:0/2727580228 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f277406fa90 0x7f277415be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f2773fff700 1 -- 172.21.15.7:0/2727580228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276c00ab80 con 0x7f277415a6e0 2024-09-13T18:52:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f94c4ef4700 1 Processor -- start 2024-09-13T18:52:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f2f567fc700 1 --2- 172.21.15.7:0/860921042 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2f44066800 0x7f2f44068cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f43ee7fc700 1 -- 172.21.15.7:0/836356888 learned_addr learned my addr 172.21.15.7:0/836356888 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f2f567fc700 1 -- 172.21.15.7:0/860921042 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f580838e0 con 0x7f2f6015d030 2024-09-13T18:52:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f2773fff700 1 --2- 172.21.15.7:0/2727580228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f277415a6e0 0x7f27741ee740 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2774072180 tx=0x7f276c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f94c4ef4700 1 -- start start 2024-09-13T18:52:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f2f65a32700 1 --2- 172.21.15.7:0/860921042 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2f44066800 0x7f2f44068cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7fe6db7fe700 1 --2- 172.21.15.7:0/3998623249 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe6c4066b40 0x7fe6c4069000 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc000f60 tx=0x7fe6cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f43ee7fc700 1 -- 172.21.15.7:0/836356888 >> v2:172.21.15.7:3300/0 conn(0x7f43f006add0 msgr2=0x7f43f006b260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:52:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f43ee7fc700 1 --2- 172.21.15.7:0/836356888 >> v2:172.21.15.7:3300/0 conn(0x7f43f006add0 0x7f43f006b260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f43ee7fc700 1 -- 172.21.15.7:0/836356888 >> v1:172.21.15.7:6789/0 conn(0x7f43f015b790 legacy=0x7f43f0071b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T18:52:25.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f43ee7fc700 1 -- 172.21.15.7:0/836356888 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f00716a0 con 0x7f43f015bb70 2024-09-13T18:52:25.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f43ee7fc700 1 --2- 172.21.15.7:0/836356888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43f015bb70 0x7f43f015e640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f43f00cf120 tx=0x7f43e4009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f56898a3766b7de server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f2f65a32700 1 --2- 172.21.15.7:0/860921042 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2f44066800 0x7f2f44068cc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c007fd0 tx=0x7f2f5c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f43edffb700 1 -- 172.21.15.7:0/836356888 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e4004230 con 0x7f43f015bb70 2024-09-13T18:52:25.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f43edffb700 1 -- 172.21.15.7:0/836356888 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e4004390 con 0x7f43f015bb70 2024-09-13T18:52:25.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f43edffb700 1 -- 172.21.15.7:0/836356888 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e40036c0 con 0x7f43f015bb70 2024-09-13T18:52:25.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f94c4ef4700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94c006fa90 0x7f94c00cc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f94c4ef4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f94c00ccd40 0x7f94c00cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f94c4ef4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c006a5d0 con 0x7f94c00b0090 2024-09-13T18:52:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f94c4ef4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c006bd20 con 0x7f94c00ccd40 2024-09-13T18:52:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f94c4ef4700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c00cb760 con 0x7f94c006fa90 2024-09-13T18:52:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f94bffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f94c00b0090 0x7f94c006fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35880/0 (socket says 172.21.15.7:35880) 2024-09-13T18:52:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f94bffff700 1 -- 172.21.15.7:0/492203547 learned_addr learned my addr 172.21.15.7:0/492203547 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f71f659c700 1 --2- 172.21.15.7:0/2369548083 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f71d8066730 0x7f71d8068bf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f71f000b370 tx=0x7f71f0009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f94bf7fe700 1 --2- 172.21.15.7:0/492203547 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94c006fa90 0x7f94c00cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f94beffd700 1 --2- 172.21.15.7:0/492203547 >> v2:172.21.15.7:3300/0 conn(0x7f94c00ccd40 0x7f94c00cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f43f52b2700 1 -- 172.21.15.7:0/836356888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43f015bb70 msgr2=0x7f43f015e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f43f52b2700 1 --2- 172.21.15.7:0/836356888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43f015bb70 0x7f43f015e640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f43f00cf120 tx=0x7f43e4009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f94beffd700 1 -- 172.21.15.7:0/492203547 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94c006fa90 msgr2=0x7f94c00cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f94beffd700 1 --2- 172.21.15.7:0/492203547 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94c006fa90 0x7f94c00cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f94beffd700 1 -- 172.21.15.7:0/492203547 >> v1:172.21.15.7:6789/0 conn(0x7f94c00b0090 legacy=0x7f94c006fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f94beffd700 1 -- 172.21.15.7:0/492203547 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c00cba40 con 0x7f94c00ccd40 2024-09-13T18:52:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.397+0000 7f94beffd700 1 --2- 172.21.15.7:0/492203547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c00ccd40 0x7f94c00cd180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009850 tx=0x7f94b4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f8e5016322af8b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.401+0000 7f2f66a34700 1 -- 172.21.15.7:0/860921042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f48005310 con 0x7f2f6015d030 2024-09-13T18:52:25.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.401+0000 7f71fccd3700 1 -- 172.21.15.7:0/2369548083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71dc005310 con 0x7f71f815f140 2024-09-13T18:52:25.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.401+0000 7f275bfff700 1 -- 172.21.15.7:0/2727580228 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c004440 con 0x7f277415a6e0 2024-09-13T18:52:25.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.401+0000 7f2778ce5700 1 -- 172.21.15.7:0/2727580228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277415adc0 con 0x7f277415a6e0 2024-09-13T18:52:25.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.401+0000 7f2778ce5700 1 -- 172.21.15.7:0/2727580228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277415b380 con 0x7f277415a6e0 2024-09-13T18:52:25.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.401+0000 7fe6c3fff700 1 -- 172.21.15.7:0/3998623249 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe6d0017080 con 0x7fe6dc15a460 2024-09-13T18:52:25.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.401+0000 7f94be7fc700 1 -- 172.21.15.7:0/492203547 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4011830 con 0x7f94c00ccd40 2024-09-13T18:52:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43f52b2700 1 -- 172.21.15.7:0/836356888 shutdown_connections 2024-09-13T18:52:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43f52b2700 1 --2- 172.21.15.7:0/836356888 >> v2:172.21.15.7:3300/0 conn(0x7f43f006add0 0x7f43f006b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43f52b2700 1 --2- 172.21.15.7:0/836356888 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43f015bb70 0x7f43f015e640 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43f52b2700 1 -- 172.21.15.7:0/836356888 >> 172.21.15.7:0/836356888 conn(0x7f43f006d900 msgr2=0x7f43f006dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43f52b2700 1 -- 172.21.15.7:0/836356888 shutdown_connections 2024-09-13T18:52:25.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43f52b2700 1 -- 172.21.15.7:0/836356888 wait complete. 2024-09-13T18:52:25.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f2778ce5700 1 -- 172.21.15.7:0/2727580228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277415bb80 con 0x7f277415a6e0 2024-09-13T18:52:25.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f2f567fc700 1 -- 172.21.15.7:0/860921042 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2f580546c0 con 0x7f2f6015d030 2024-09-13T18:52:25.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43f52b2700 1 Processor -- start 2024-09-13T18:52:25.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43f52b2700 1 -- start start 2024-09-13T18:52:25.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43f52b2700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f43f006add0 0x7f43f0155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43f52b2700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43f015b790 0x7f43f015d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43f52b2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f015bb70 0x7f43f015d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43f52b2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f01583a0 con 0x7f43f015bb70 2024-09-13T18:52:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43f52b2700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f0155c50 con 0x7f43f015b790 2024-09-13T18:52:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43f52b2700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f0155ef0 con 0x7f43f006add0 2024-09-13T18:52:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f015bb70 0x7f43f015d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f015bb70 0x7f43f015d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49604/0 (socket says 172.21.15.7:49604) 2024-09-13T18:52:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43ef7fe700 1 -- 172.21.15.7:0/582445024 learned_addr learned my addr 172.21.15.7:0/582445024 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43eeffd700 1 --2- 172.21.15.7:0/582445024 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f43f006add0 0x7f43f0155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.405+0000 7f43ee7fc700 1 --2- 172.21.15.7:0/582445024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43f015b790 0x7f43f015d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f43ef7fe700 1 -- 172.21.15.7:0/582445024 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f43f006add0 msgr2=0x7f43f0155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f43ef7fe700 1 --2- 172.21.15.7:0/582445024 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f43f006add0 0x7f43f0155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f43ef7fe700 1 -- 172.21.15.7:0/582445024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43f015b790 msgr2=0x7f43f015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f43ef7fe700 1 --2- 172.21.15.7:0/582445024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43f015b790 0x7f43f015d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f43ef7fe700 1 -- 172.21.15.7:0/582445024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e4009570 con 0x7f43f015bb70 2024-09-13T18:52:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f43ee7fc700 1 --2- 172.21.15.7:0/582445024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43f015b790 0x7f43f015d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f43eeffd700 1 --2- 172.21.15.7:0/582445024 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f43f006add0 0x7f43f0155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f43ef7fe700 1 --2- 172.21.15.7:0/582445024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f015bb70 0x7f43f015d780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f43e000b4a0 tx=0x7f43e000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f275bfff700 1 -- 172.21.15.7:0/2727580228 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c00a2e0 con 0x7f277415a6e0 2024-09-13T18:52:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f275bfff700 1 -- 172.21.15.7:0/2727580228 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c0047a0 con 0x7f277415a6e0 2024-09-13T18:52:25.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f94be7fc700 1 -- 172.21.15.7:0/492203547 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b4011990 con 0x7f94c00ccd40 2024-09-13T18:52:25.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f94be7fc700 1 -- 172.21.15.7:0/492203547 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4011d40 con 0x7f94c00ccd40 2024-09-13T18:52:25.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f71e6ffd700 1 -- 172.21.15.7:0/2369548083 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f71ec0596b0 con 0x7f71f815f140 2024-09-13T18:52:25.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f94c4ef4700 1 -- 172.21.15.7:0/492203547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c00ccd40 msgr2=0x7f94c00cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f94c4ef4700 1 --2- 172.21.15.7:0/492203547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c00ccd40 0x7f94c00cd180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f94b4009850 tx=0x7f94b4009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:25.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f94c4ef4700 1 -- 172.21.15.7:0/492203547 shutdown_connections 2024-09-13T18:52:25.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f94c4ef4700 1 --2- 172.21.15.7:0/492203547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c00ccd40 0x7f94c00cd180 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f94c4ef4700 1 --2- 172.21.15.7:0/492203547 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94c006fa90 0x7f94c00cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f94c4ef4700 1 -- 172.21.15.7:0/492203547 >> 172.21.15.7:0/492203547 conn(0x7f94c00ae590 msgr2=0x7f94c00ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:25.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.409+0000 7f275bfff700 1 -- 172.21.15.7:0/2727580228 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f276c0224d0 con 0x7f277415a6e0 2024-09-13T18:52:25.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.413+0000 7f275bfff700 1 --2- 172.21.15.7:0/2727580228 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2760066730 0x7f2760068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.413+0000 7f275bfff700 1 -- 172.21.15.7:0/2727580228 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f276c089670 con 0x7f277415a6e0 2024-09-13T18:52:25.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.413+0000 7f27737fe700 1 --2- 172.21.15.7:0/2727580228 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2760066730 0x7f2760068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.413+0000 7f27737fe700 1 --2- 172.21.15.7:0/2727580228 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2760066730 0x7f2760068bf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2768000f60 tx=0x7f2768000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.413+0000 7f275bfff700 1 -- 172.21.15.7:0/2727580228 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f276c056b90 con 0x7f277415a6e0 2024-09-13T18:52:25.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.413+0000 7f43df7fe700 1 -- 172.21.15.7:0/582445024 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0010a30 con 0x7f43f015bb70 2024-09-13T18:52:25.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.413+0000 7f43df7fe700 1 -- 172.21.15.7:0/582445024 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e0010b90 con 0x7f43f015bb70 2024-09-13T18:52:25.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.413+0000 7f43df7fe700 1 -- 172.21.15.7:0/582445024 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0018440 con 0x7f43f015bb70 2024-09-13T18:52:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.413+0000 7f94c4ef4700 1 -- 172.21.15.7:0/492203547 shutdown_connections 2024-09-13T18:52:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.413+0000 7f94c4ef4700 1 -- 172.21.15.7:0/492203547 wait complete. 2024-09-13T18:52:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94c4ef4700 1 Processor -- start 2024-09-13T18:52:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94c4ef4700 1 -- start start 2024-09-13T18:52:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94c4ef4700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94c006fa90 0x7f94c0157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94c4ef4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94c00b0090 0x7f94c0157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94c4ef4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c015d290 0x7f94c015fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94c4ef4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c015f870 con 0x7f94c015d290 2024-09-13T18:52:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94c4ef4700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c0157a90 con 0x7f94c00b0090 2024-09-13T18:52:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94c4ef4700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c015ef40 con 0x7f94c006fa90 2024-09-13T18:52:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94bffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c015d290 0x7f94c015fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94bffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c015d290 0x7f94c015fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49616/0 (socket says 172.21.15.7:49616) 2024-09-13T18:52:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94bffff700 1 -- 172.21.15.7:0/2015298668 learned_addr learned my addr 172.21.15.7:0/2015298668 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94beffd700 1 --2- 172.21.15.7:0/2015298668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94c00b0090 0x7f94c0157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94bf7fe700 1 --2- 172.21.15.7:0/2015298668 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94c006fa90 0x7f94c0157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94bffff700 1 -- 172.21.15.7:0/2015298668 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94c006fa90 msgr2=0x7f94c0157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94bffff700 1 --2- 172.21.15.7:0/2015298668 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94c006fa90 0x7f94c0157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94bffff700 1 -- 172.21.15.7:0/2015298668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94c00b0090 msgr2=0x7f94c0157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94bffff700 1 --2- 172.21.15.7:0/2015298668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94c00b0090 0x7f94c0157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94bf7fe700 1 --2- 172.21.15.7:0/2015298668 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94c006fa90 0x7f94c0157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:25.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.417+0000 7f94bffff700 1 -- 172.21.15.7:0/2015298668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b40094b0 con 0x7f94c015d290 2024-09-13T18:52:25.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.421+0000 7f94bffff700 1 --2- 172.21.15.7:0/2015298668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c015d290 0x7f94c015fd00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f94b800cbc0 tx=0x7f94b800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.425+0000 7f94a7fff700 1 -- 172.21.15.7:0/2015298668 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b800a7b0 con 0x7f94c015d290 2024-09-13T18:52:25.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.425+0000 7f94a7fff700 1 -- 172.21.15.7:0/2015298668 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b800a910 con 0x7f94c015d290 2024-09-13T18:52:25.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.425+0000 7f94c4ef4700 1 -- 172.21.15.7:0/2015298668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c015f220 con 0x7f94c015d290 2024-09-13T18:52:25.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.425+0000 7f94c4ef4700 1 -- 172.21.15.7:0/2015298668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c01583e0 con 0x7f94c015d290 2024-09-13T18:52:25.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.425+0000 7f94a7fff700 1 -- 172.21.15.7:0/2015298668 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b800ac80 con 0x7f94c015d290 2024-09-13T18:52:25.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.425+0000 7f94a7fff700 1 -- 172.21.15.7:0/2015298668 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f94b801b3b0 con 0x7f94c015d290 2024-09-13T18:52:25.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.425+0000 7f94a7fff700 1 --2- 172.21.15.7:0/2015298668 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f94ac0667b0 0x7f94ac068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.425+0000 7f94a7fff700 1 -- 172.21.15.7:0/2015298668 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94b8083950 con 0x7f94c015d290 2024-09-13T18:52:25.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.429+0000 7f43f52b2700 1 -- 172.21.15.7:0/582445024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f01561d0 con 0x7f43f015bb70 2024-09-13T18:52:25.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.429+0000 7f43f52b2700 1 -- 172.21.15.7:0/582445024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f0156790 con 0x7f43f015bb70 2024-09-13T18:52:25.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.429+0000 7f94c4ef4700 1 -- 172.21.15.7:0/2015298668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c015f9e0 con 0x7f94c015d290 2024-09-13T18:52:25.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.429+0000 7f43df7fe700 1 -- 172.21.15.7:0/582445024 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f43e00213b0 con 0x7f43f015bb70 2024-09-13T18:52:25.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.429+0000 7f43df7fe700 1 --2- 172.21.15.7:0/582445024 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f43d4066780 0x7f43d4068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.429+0000 7f43df7fe700 1 -- 172.21.15.7:0/582445024 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43e0087ac0 con 0x7f43f015bb70 2024-09-13T18:52:25.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.429+0000 7f43eeffd700 1 --2- 172.21.15.7:0/582445024 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f43d4066780 0x7f43d4068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.433+0000 7f43eeffd700 1 --2- 172.21.15.7:0/582445024 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f43d4066780 0x7f43d4068c40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009fc0 tx=0x7f43e8009550 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.437+0000 7f43f52b2700 1 -- 172.21.15.7:0/582445024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f015a9a0 con 0x7f43f015bb70 2024-09-13T18:52:25.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.437+0000 7f94bf7fe700 1 --2- 172.21.15.7:0/2015298668 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f94ac0667b0 0x7f94ac068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.437+0000 7f94bf7fe700 1 --2- 172.21.15.7:0/2015298668 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f94ac0667b0 0x7f94ac068c70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f94c015bef0 tx=0x7f94b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.437+0000 7f94a7fff700 1 -- 172.21.15.7:0/2015298668 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f94b8050e50 con 0x7f94c015d290 2024-09-13T18:52:25.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.449+0000 7f43df7fe700 1 -- 172.21.15.7:0/582445024 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f43e0054f10 con 0x7f43f015bb70 2024-09-13T18:52:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.649+0000 7f5c8cb9e700 1 -- 172.21.15.7:0/645381926 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5c6c0055c0 con 0x7f5c8806fa90 2024-09-13T18:52:25.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.649+0000 7f5c76ffd700 1 -- 172.21.15.7:0/645381926 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5c78055630 con 0x7f5c8806fa90 2024-09-13T18:52:25.657 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-13T18:52:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.653+0000 7f5c74ff9700 1 -- 172.21.15.7:0/645381926 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5c70066950 msgr2=0x7f5c70068e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.653+0000 7f5c74ff9700 1 --2- 172.21.15.7:0/645381926 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5c70066950 0x7f5c70068e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5c88202930 tx=0x7f5c7c008040 comp rx=0 tx=0).stop 2024-09-13T18:52:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.653+0000 7f5c74ff9700 1 -- 172.21.15.7:0/645381926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c8806fa90 msgr2=0x7f5c881401c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.653+0000 7f5c74ff9700 1 --2- 172.21.15.7:0/645381926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c8806fa90 0x7f5c881401c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800bfd0 tx=0x7f5c7800da60 comp rx=0 tx=0).stop 2024-09-13T18:52:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.653+0000 7f5c74ff9700 1 -- 172.21.15.7:0/645381926 shutdown_connections 2024-09-13T18:52:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.653+0000 7f5c74ff9700 1 --2- 172.21.15.7:0/645381926 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5c70066950 0x7f5c70068e10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.653+0000 7f5c74ff9700 1 --2- 172.21.15.7:0/645381926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5c880cd450 0x7f5c881539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.653+0000 7f5c74ff9700 1 --2- 172.21.15.7:0/645381926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c88070e80 0x7f5c88140700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.653+0000 7f5c74ff9700 1 --2- 172.21.15.7:0/645381926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5c8806fa90 0x7f5c881401c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.653+0000 7f5c74ff9700 1 -- 172.21.15.7:0/645381926 >> 172.21.15.7:0/645381926 conn(0x7f5c880ae590 msgr2=0x7f5c880ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.653+0000 7f5c74ff9700 1 -- 172.21.15.7:0/645381926 shutdown_connections 2024-09-13T18:52:25.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.653+0000 7f5c74ff9700 1 -- 172.21.15.7:0/645381926 wait complete. 2024-09-13T18:52:25.674 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.3 2024-09-13T18:52:25.674 DEBUG:teuthology.parallel:result is None 2024-09-13T18:52:25.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.681+0000 7f69bd8b3700 1 -- 172.21.15.7:0/3652778121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f69980055c0 con 0x7f69b8146480 2024-09-13T18:52:25.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.681+0000 7f69b57fa700 1 -- 172.21.15.7:0/3652778121 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f69ac015070 con 0x7f69b8146480 2024-09-13T18:52:25.686 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-13T18:52:25.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.681+0000 7f69bd8b3700 1 -- 172.21.15.7:0/3652778121 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f69a4066830 msgr2=0x7f69a4068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.681+0000 7f69bd8b3700 1 --2- 172.21.15.7:0/3652778121 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f69a4066830 0x7f69a4068cf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f69a8005930 tx=0x7f69a8009040 comp rx=0 tx=0).stop 2024-09-13T18:52:25.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.681+0000 7f69bd8b3700 1 -- 172.21.15.7:0/3652778121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b8146480 msgr2=0x7f69b806cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.681+0000 7f69bd8b3700 1 --2- 172.21.15.7:0/3652778121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b8146480 0x7f69b806cc10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f69ac014040 tx=0x7f69ac004770 comp rx=0 tx=0).stop 2024-09-13T18:52:25.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.685+0000 7f69bd8b3700 1 -- 172.21.15.7:0/3652778121 shutdown_connections 2024-09-13T18:52:25.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.685+0000 7f69bd8b3700 1 --2- 172.21.15.7:0/3652778121 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f69a4066830 0x7f69a4068cf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.685+0000 7f69bd8b3700 1 --2- 172.21.15.7:0/3652778121 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f69b815dd90 0x7f69b81603e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.685+0000 7f69bd8b3700 1 --2- 172.21.15.7:0/3652778121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69b8146480 0x7f69b806cc10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.685+0000 7f69bd8b3700 1 --2- 172.21.15.7:0/3652778121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69b80cc640 0x7f69b8140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.685+0000 7f69bd8b3700 1 -- 172.21.15.7:0/3652778121 >> 172.21.15.7:0/3652778121 conn(0x7f69b80ae590 msgr2=0x7f69b80af880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:25.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.685+0000 7f69bd8b3700 1 -- 172.21.15.7:0/3652778121 shutdown_connections 2024-09-13T18:52:25.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.685+0000 7f69bd8b3700 1 -- 172.21.15.7:0/3652778121 wait complete. 2024-09-13T18:52:25.703 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.4 2024-09-13T18:52:25.703 DEBUG:teuthology.parallel:result is None 2024-09-13T18:52:25.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.705+0000 7f71fccd3700 1 -- 172.21.15.7:0/2369548083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f71dc0055c0 con 0x7f71f815f140 2024-09-13T18:52:25.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.709+0000 7f71e6ffd700 1 -- 172.21.15.7:0/2369548083 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f71ec024020 con 0x7f71f815f140 2024-09-13T18:52:25.713 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-13T18:52:25.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.709+0000 7f71fccd3700 1 -- 172.21.15.7:0/2369548083 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f71d8066730 msgr2=0x7f71d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.709+0000 7f71fccd3700 1 --2- 172.21.15.7:0/2369548083 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f71d8066730 0x7f71d8068bf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f71f000b370 tx=0x7f71f0009000 comp rx=0 tx=0).stop 2024-09-13T18:52:25.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.709+0000 7f71fccd3700 1 -- 172.21.15.7:0/2369548083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f815f140 msgr2=0x7f71f815b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.709+0000 7f71fccd3700 1 --2- 172.21.15.7:0/2369548083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f815f140 0x7f71f815b740 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f71ec012680 tx=0x7f71ec012990 comp rx=0 tx=0).stop 2024-09-13T18:52:25.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.709+0000 7f71fccd3700 1 -- 172.21.15.7:0/2369548083 shutdown_connections 2024-09-13T18:52:25.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.709+0000 7f71fccd3700 1 --2- 172.21.15.7:0/2369548083 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f71d8066730 0x7f71d8068bf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.709+0000 7f71fccd3700 1 --2- 172.21.15.7:0/2369548083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71f815f140 0x7f71f815b740 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.709+0000 7f71fccd3700 1 --2- 172.21.15.7:0/2369548083 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f71f8144340 0x7f71f815d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.709+0000 7f71fccd3700 1 --2- 172.21.15.7:0/2369548083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f71f806bdb0 0x7f71f815a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.709+0000 7f71fccd3700 1 -- 172.21.15.7:0/2369548083 >> 172.21.15.7:0/2369548083 conn(0x7f71f80b0e10 msgr2=0x7f71f80ae1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:25.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.709+0000 7f71fccd3700 1 -- 172.21.15.7:0/2369548083 shutdown_connections 2024-09-13T18:52:25.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.709+0000 7f71fccd3700 1 -- 172.21.15.7:0/2369548083 wait complete. 2024-09-13T18:52:25.726 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.7 2024-09-13T18:52:25.726 DEBUG:teuthology.parallel:result is None 2024-09-13T18:52:25.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.733+0000 7f94c4ef4700 1 -- 172.21.15.7:0/2015298668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f94c00cd180 con 0x7f94c015d290 2024-09-13T18:52:25.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.733+0000 7f94a7fff700 1 -- 172.21.15.7:0/2015298668 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f94b8017020 con 0x7f94c015d290 2024-09-13T18:52:25.738 INFO:teuthology.orchestra.run.smithi007.stdout:60129542150 2024-09-13T18:52:25.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.733+0000 7f94c4ef4700 1 -- 172.21.15.7:0/2015298668 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f94ac0667b0 msgr2=0x7f94ac068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.733+0000 7f94c4ef4700 1 --2- 172.21.15.7:0/2015298668 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f94ac0667b0 0x7f94ac068c70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f94c015bef0 tx=0x7f94b0008040 comp rx=0 tx=0).stop 2024-09-13T18:52:25.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.733+0000 7f94c4ef4700 1 -- 172.21.15.7:0/2015298668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c015d290 msgr2=0x7f94c015fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.733+0000 7f94c4ef4700 1 --2- 172.21.15.7:0/2015298668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c015d290 0x7f94c015fd00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f94b800cbc0 tx=0x7f94b800cbf0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.737+0000 7f94c4ef4700 1 -- 172.21.15.7:0/2015298668 shutdown_connections 2024-09-13T18:52:25.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.737+0000 7f94c4ef4700 1 --2- 172.21.15.7:0/2015298668 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f94ac0667b0 0x7f94ac068c70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.737+0000 7f94c4ef4700 1 --2- 172.21.15.7:0/2015298668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94c015d290 0x7f94c015fd00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.737+0000 7f94c4ef4700 1 --2- 172.21.15.7:0/2015298668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94c00b0090 0x7f94c0157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.737+0000 7f94c4ef4700 1 --2- 172.21.15.7:0/2015298668 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94c006fa90 0x7f94c0157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.737+0000 7f94c4ef4700 1 -- 172.21.15.7:0/2015298668 >> 172.21.15.7:0/2015298668 conn(0x7f94c00ae590 msgr2=0x7f94c006deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:25.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.737+0000 7f94c4ef4700 1 -- 172.21.15.7:0/2015298668 shutdown_connections 2024-09-13T18:52:25.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.737+0000 7f94c4ef4700 1 -- 172.21.15.7:0/2015298668 wait complete. 2024-09-13T18:52:25.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.737+0000 7f2f66a34700 1 -- 172.21.15.7:0/860921042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2f480055c0 con 0x7f2f6015d030 2024-09-13T18:52:25.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.737+0000 7f2f567fc700 1 -- 172.21.15.7:0/860921042 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2f58017090 con 0x7f2f6015d030 2024-09-13T18:52:25.744 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-13T18:52:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.741+0000 7f2f66a34700 1 -- 172.21.15.7:0/860921042 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2f44066800 msgr2=0x7f2f44068cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.741+0000 7f2f66a34700 1 --2- 172.21.15.7:0/860921042 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2f44066800 0x7f2f44068cc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c007fd0 tx=0x7f2f5c009040 comp rx=0 tx=0).stop 2024-09-13T18:52:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.741+0000 7f2f66a34700 1 -- 172.21.15.7:0/860921042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f6015d030 msgr2=0x7f2f60154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.741+0000 7f2f66a34700 1 --2- 172.21.15.7:0/860921042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f6015d030 0x7f2f60154070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800cae0 tx=0x7f2f5800cb10 comp rx=0 tx=0).stop 2024-09-13T18:52:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.741+0000 7f2f66a34700 1 -- 172.21.15.7:0/860921042 shutdown_connections 2024-09-13T18:52:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.741+0000 7f2f66a34700 1 --2- 172.21.15.7:0/860921042 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2f44066800 0x7f2f44068cc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.741+0000 7f2f66a34700 1 --2- 172.21.15.7:0/860921042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f6015d030 0x7f2f60154070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.741+0000 7f2f66a34700 1 --2- 172.21.15.7:0/860921042 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2f600ce270 0x7f2f60153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.741+0000 7f2f66a34700 1 --2- 172.21.15.7:0/860921042 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f6006d6f0 0x7f2f600abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.741+0000 7f2f66a34700 1 -- 172.21.15.7:0/860921042 >> 172.21.15.7:0/860921042 conn(0x7f2f600b0e10 msgr2=0x7f2f6006bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.741+0000 7f2f66a34700 1 -- 172.21.15.7:0/860921042 shutdown_connections 2024-09-13T18:52:25.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.741+0000 7f2f66a34700 1 -- 172.21.15.7:0/860921042 wait complete. 2024-09-13T18:52:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.745+0000 7fe6e0a0b700 1 -- 172.21.15.7:0/3998623249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe6bc005890 con 0x7fe6dc15a460 2024-09-13T18:52:25.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.745+0000 7f2778ce5700 1 -- 172.21.15.7:0/2727580228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f277415bb80 con 0x7f277415a6e0 2024-09-13T18:52:25.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.745+0000 7f275bfff700 1 -- 172.21.15.7:0/2727580228 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f277415bb80 con 0x7f277415a6e0 2024-09-13T18:52:25.754 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-13T18:52:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.745+0000 7fe6c3fff700 1 -- 172.21.15.7:0/3998623249 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe6d00508d0 con 0x7fe6dc15a460 2024-09-13T18:52:25.754 INFO:teuthology.orchestra.run.smithi007.stdout:60129542150 2024-09-13T18:52:25.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.0 2024-09-13T18:52:25.755 DEBUG:teuthology.parallel:result is None 2024-09-13T18:52:25.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7f2778ce5700 1 -- 172.21.15.7:0/2727580228 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2760066730 msgr2=0x7f2760068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7f2778ce5700 1 --2- 172.21.15.7:0/2727580228 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2760066730 0x7f2760068bf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2768000f60 tx=0x7f2768000ed0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7f2778ce5700 1 -- 172.21.15.7:0/2727580228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f277415a6e0 msgr2=0x7f27741ee740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7f2778ce5700 1 --2- 172.21.15.7:0/2727580228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f277415a6e0 0x7f27741ee740 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2774072180 tx=0x7f276c004af0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7f2778ce5700 1 -- 172.21.15.7:0/2727580228 shutdown_connections 2024-09-13T18:52:25.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7f2778ce5700 1 --2- 172.21.15.7:0/2727580228 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2760066730 0x7f2760068bf0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7f2778ce5700 1 --2- 172.21.15.7:0/2727580228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f277415a6e0 0x7f27741ee740 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7f2778ce5700 1 --2- 172.21.15.7:0/2727580228 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2774070e80 0x7f277413d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7f2778ce5700 1 --2- 172.21.15.7:0/2727580228 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f277406fa90 0x7f277415be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7f2778ce5700 1 -- 172.21.15.7:0/2727580228 >> 172.21.15.7:0/2727580228 conn(0x7f27740ae590 msgr2=0x7f27740ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:25.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7f2778ce5700 1 -- 172.21.15.7:0/2727580228 shutdown_connections 2024-09-13T18:52:25.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7f2778ce5700 1 -- 172.21.15.7:0/2727580228 wait complete. 2024-09-13T18:52:25.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7fe6e0a0b700 1 -- 172.21.15.7:0/3998623249 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe6c4066b40 msgr2=0x7fe6c4069000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7fe6e0a0b700 1 --2- 172.21.15.7:0/3998623249 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe6c4066b40 0x7fe6c4069000 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc000f60 tx=0x7fe6cc009040 comp rx=0 tx=0).stop 2024-09-13T18:52:25.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7fe6e0a0b700 1 -- 172.21.15.7:0/3998623249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6dc15a460 msgr2=0x7fe6dc1442c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7fe6e0a0b700 1 --2- 172.21.15.7:0/3998623249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6dc15a460 0x7fe6dc1442c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000c4d0 tx=0x7fe6d000c890 comp rx=0 tx=0).stop 2024-09-13T18:52:25.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7fe6e0a0b700 1 -- 172.21.15.7:0/3998623249 shutdown_connections 2024-09-13T18:52:25.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7fe6e0a0b700 1 --2- 172.21.15.7:0/3998623249 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe6c4066b40 0x7fe6c4069000 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7fe6e0a0b700 1 --2- 172.21.15.7:0/3998623249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6dc15a460 0x7fe6dc1442c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7fe6e0a0b700 1 --2- 172.21.15.7:0/3998623249 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe6dc1573e0 0x7fe6dc158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7fe6e0a0b700 1 --2- 172.21.15.7:0/3998623249 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe6dc0712b0 0x7fe6dc15bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7fe6e0a0b700 1 -- 172.21.15.7:0/3998623249 >> 172.21.15.7:0/3998623249 conn(0x7fe6dc0b0e10 msgr2=0x7fe6dc0ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:25.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7fe6e0a0b700 1 -- 172.21.15.7:0/3998623249 shutdown_connections 2024-09-13T18:52:25.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.749+0000 7fe6e0a0b700 1 -- 172.21.15.7:0/3998623249 wait complete. 2024-09-13T18:52:25.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.1 2024-09-13T18:52:25.765 DEBUG:teuthology.parallel:result is None 2024-09-13T18:52:25.766 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.2 2024-09-13T18:52:25.766 DEBUG:teuthology.parallel:result is None 2024-09-13T18:52:25.766 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2024-09-13T18:52:25.767 DEBUG:teuthology.parallel:result is None 2024-09-13T18:52:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.765+0000 7f43f52b2700 1 -- 172.21.15.7:0/582445024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f43f006df30 con 0x7f43f015bb70 2024-09-13T18:52:25.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.765+0000 7f43df7fe700 1 -- 172.21.15.7:0/582445024 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f43e0058530 con 0x7f43f015bb70 2024-09-13T18:52:25.771 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-13T18:52:25.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.769+0000 7f43f52b2700 1 -- 172.21.15.7:0/582445024 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f43d4066780 msgr2=0x7f43d4068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.769+0000 7f43f52b2700 1 --2- 172.21.15.7:0/582445024 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f43d4066780 0x7f43d4068c40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009fc0 tx=0x7f43e8009550 comp rx=0 tx=0).stop 2024-09-13T18:52:25.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.769+0000 7f43f52b2700 1 -- 172.21.15.7:0/582445024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f015bb70 msgr2=0x7f43f015d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.769+0000 7f43f52b2700 1 --2- 172.21.15.7:0/582445024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f015bb70 0x7f43f015d780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f43e000b4a0 tx=0x7f43e000b860 comp rx=0 tx=0).stop 2024-09-13T18:52:25.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.769+0000 7f43f52b2700 1 -- 172.21.15.7:0/582445024 shutdown_connections 2024-09-13T18:52:25.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.769+0000 7f43f52b2700 1 --2- 172.21.15.7:0/582445024 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f43d4066780 0x7f43d4068c40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.769+0000 7f43f52b2700 1 --2- 172.21.15.7:0/582445024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43f015bb70 0x7f43f015d780 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.769+0000 7f43f52b2700 1 --2- 172.21.15.7:0/582445024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f43f015b790 0x7f43f015d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.769+0000 7f43f52b2700 1 --2- 172.21.15.7:0/582445024 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f43f006add0 0x7f43f0155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.769+0000 7f43f52b2700 1 -- 172.21.15.7:0/582445024 >> 172.21.15.7:0/582445024 conn(0x7f43f006d900 msgr2=0x7f43f0154710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:25.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.769+0000 7f43f52b2700 1 -- 172.21.15.7:0/582445024 shutdown_connections 2024-09-13T18:52:25.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.769+0000 7f43f52b2700 1 -- 172.21.15.7:0/582445024 wait complete. 2024-09-13T18:52:25.783 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.6 2024-09-13T18:52:25.783 DEBUG:teuthology.parallel:result is None 2024-09-13T18:52:25.784 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T18:52:25.784 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:52:25.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.881+0000 7f61a0982700 1 Processor -- start 2024-09-13T18:52:25.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.881+0000 7f61a0982700 1 -- start start 2024-09-13T18:52:25.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f61a0982700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f619c153da0 0x7f619c06c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f61a0982700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f619c0b29a0 0x7f619c0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f61a0982700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f619c0b3710 con 0x7f619c071bb0 2024-09-13T18:52:25.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f61a0982700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619c15abb0 con 0x7f619c0b29a0 2024-09-13T18:52:25.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f61a0982700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c06be30 con 0x7f619c153da0 2024-09-13T18:52:25.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f619b59e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f619c0b29a0 0x7f619c0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f619a59c700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f619c071bb0 0x7f619c072010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:51668/0 (socket says 172.21.15.7:51668) 2024-09-13T18:52:25.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f619a59c700 1 -- 172.21.15.7:0/1082674513 learned_addr learned my addr 172.21.15.7:0/1082674513 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:25.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f619b59e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f619c0b29a0 0x7f619c0b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:40956/0 (socket says 172.21.15.7:40956) 2024-09-13T18:52:25.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f619ad9d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f619c153da0 0x7f619c06c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f619b59e700 1 -- 172.21.15.7:0/1082674513 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f619c153da0 msgr2=0x7f619c06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:25.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f619b59e700 1 --2- 172.21.15.7:0/1082674513 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f619c153da0 0x7f619c06c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f619b59e700 1 -- 172.21.15.7:0/1082674513 >> v1:172.21.15.171:6789/0 conn(0x7f619c071bb0 legacy=0x7f619c072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f619b59e700 1 -- 172.21.15.7:0/1082674513 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c06c110 con 0x7f619c0b29a0 2024-09-13T18:52:25.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f619b59e700 1 --2- 172.21.15.7:0/1082674513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f619c0b29a0 0x7f619c0b2e30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6194009d90 tx=0x7f619400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6aa93e476ff65b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f6199d9b700 1 -- 172.21.15.7:0/1082674513 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619400e070 con 0x7f619c0b29a0 2024-09-13T18:52:25.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f6199d9b700 1 -- 172.21.15.7:0/1082674513 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619400bb20 con 0x7f619c0b29a0 2024-09-13T18:52:25.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f6199d9b700 1 -- 172.21.15.7:0/1082674513 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194012430 con 0x7f619c0b29a0 2024-09-13T18:52:25.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f61a0982700 1 -- 172.21.15.7:0/1082674513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f619c0b29a0 msgr2=0x7f619c0b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f61a0982700 1 --2- 172.21.15.7:0/1082674513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f619c0b29a0 0x7f619c0b2e30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6194009d90 tx=0x7f619400b330 comp rx=0 tx=0).stop 2024-09-13T18:52:25.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f61a0982700 1 -- 172.21.15.7:0/1082674513 shutdown_connections 2024-09-13T18:52:25.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f61a0982700 1 --2- 172.21.15.7:0/1082674513 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f619c0b29a0 0x7f619c0b2e30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f61a0982700 1 --2- 172.21.15.7:0/1082674513 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f619c153da0 0x7f619c06c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f61a0982700 1 -- 172.21.15.7:0/1082674513 >> 172.21.15.7:0/1082674513 conn(0x7f619c0cf910 msgr2=0x7f619c0cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:25.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f61a0982700 1 -- 172.21.15.7:0/1082674513 shutdown_connections 2024-09-13T18:52:25.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.885+0000 7f61a0982700 1 -- 172.21.15.7:0/1082674513 wait complete. 2024-09-13T18:52:25.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.889+0000 7f61a0982700 1 Processor -- start 2024-09-13T18:52:25.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.889+0000 7f61a0982700 1 -- start start 2024-09-13T18:52:25.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.889+0000 7f61a0982700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f619c071bb0 0x7f619c14d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.889+0000 7f61a0982700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f619c0b29a0 0x7f619c155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.889+0000 7f61a0982700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f619c153da0 0x7f619c155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.889+0000 7f61a0982700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c154f10 con 0x7f619c153da0 2024-09-13T18:52:25.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.889+0000 7f61a0982700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c14de80 con 0x7f619c071bb0 2024-09-13T18:52:25.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.889+0000 7f61a0982700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c154570 con 0x7f619c0b29a0 2024-09-13T18:52:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.889+0000 7f619b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f619c153da0 0x7f619c155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.889+0000 7f619b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f619c153da0 0x7f619c155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49622/0 (socket says 172.21.15.7:49622) 2024-09-13T18:52:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.889+0000 7f619b59e700 1 -- 172.21.15.7:0/4066552051 learned_addr learned my addr 172.21.15.7:0/4066552051 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.889+0000 7f619ad9d700 1 --2- 172.21.15.7:0/4066552051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f619c071bb0 0x7f619c14d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.889+0000 7f619a59c700 1 --2- 172.21.15.7:0/4066552051 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f619c0b29a0 0x7f619c155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.889+0000 7f619b59e700 1 -- 172.21.15.7:0/4066552051 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f619c0b29a0 msgr2=0x7f619c155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.889+0000 7f619b59e700 1 --2- 172.21.15.7:0/4066552051 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f619c0b29a0 0x7f619c155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.893+0000 7f619b59e700 1 -- 172.21.15.7:0/4066552051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f619c071bb0 msgr2=0x7f619c14d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.893+0000 7f619b59e700 1 --2- 172.21.15.7:0/4066552051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f619c071bb0 0x7f619c14d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.893+0000 7f619b59e700 1 -- 172.21.15.7:0/4066552051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6194009a70 con 0x7f619c153da0 2024-09-13T18:52:25.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.893+0000 7f619b59e700 1 --2- 172.21.15.7:0/4066552051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f619c153da0 0x7f619c155cf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6194014040 tx=0x7f6194004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.893+0000 7f61837fe700 1 -- 172.21.15.7:0/4066552051 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194012430 con 0x7f619c153da0 2024-09-13T18:52:25.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.893+0000 7f61837fe700 1 -- 172.21.15.7:0/4066552051 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6194020070 con 0x7f619c153da0 2024-09-13T18:52:25.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.893+0000 7f61a0982700 1 -- 172.21.15.7:0/4066552051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619c154850 con 0x7f619c153da0 2024-09-13T18:52:25.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.893+0000 7f61837fe700 1 -- 172.21.15.7:0/4066552051 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6194004280 con 0x7f619c153da0 2024-09-13T18:52:25.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.893+0000 7f61a0982700 1 -- 172.21.15.7:0/4066552051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619c1567b0 con 0x7f619c153da0 2024-09-13T18:52:25.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.893+0000 7f61a0982700 1 -- 172.21.15.7:0/4066552051 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619c155050 con 0x7f619c153da0 2024-09-13T18:52:25.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.893+0000 7f61837fe700 1 -- 172.21.15.7:0/4066552051 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f61940043e0 con 0x7f619c153da0 2024-09-13T18:52:25.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.897+0000 7f61837fe700 1 --2- 172.21.15.7:0/4066552051 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6188066730 0x7f6188068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:25.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.897+0000 7f61837fe700 1 -- 172.21.15.7:0/4066552051 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6194022070 con 0x7f619c153da0 2024-09-13T18:52:25.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.897+0000 7f619ad9d700 1 --2- 172.21.15.7:0/4066552051 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6188066730 0x7f6188068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:25.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.897+0000 7f619ad9d700 1 --2- 172.21.15.7:0/4066552051 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6188066730 0x7f6188068bf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f618c009bf0 tx=0x7f618c009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:25.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:25.901+0000 7f61837fe700 1 -- 172.21.15.7:0/4066552051 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f619c155050 con 0x7f619c153da0 2024-09-13T18:52:26.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.069+0000 7f61a0982700 1 -- 172.21.15.7:0/4066552051 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f619c154180 con 0x7f6188066730 2024-09-13T18:52:26.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.085+0000 7f61837fe700 1 -- 172.21.15.7:0/4066552051 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264496 (secure 0 0 0) 0x7f618c0118c0 con 0x7f6188066730 2024-09-13T18:52:26.093 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:26.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.093+0000 7f61a0982700 1 -- 172.21.15.7:0/4066552051 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6188066730 msgr2=0x7f6188068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.093+0000 7f61a0982700 1 --2- 172.21.15.7:0/4066552051 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6188066730 0x7f6188068bf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f618c009bf0 tx=0x7f618c009240 comp rx=0 tx=0).stop 2024-09-13T18:52:26.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.093+0000 7f61a0982700 1 -- 172.21.15.7:0/4066552051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f619c153da0 msgr2=0x7f619c155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.093+0000 7f61a0982700 1 --2- 172.21.15.7:0/4066552051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f619c153da0 0x7f619c155cf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6194014040 tx=0x7f6194004af0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.093+0000 7f61a0982700 1 -- 172.21.15.7:0/4066552051 shutdown_connections 2024-09-13T18:52:26.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.093+0000 7f61a0982700 1 --2- 172.21.15.7:0/4066552051 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6188066730 0x7f6188068bf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.093+0000 7f61a0982700 1 --2- 172.21.15.7:0/4066552051 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f619c153da0 0x7f619c155cf0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.093+0000 7f61a0982700 1 --2- 172.21.15.7:0/4066552051 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f619c0b29a0 0x7f619c155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.093+0000 7f61a0982700 1 --2- 172.21.15.7:0/4066552051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f619c071bb0 0x7f619c14d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.093+0000 7f61a0982700 1 -- 172.21.15.7:0/4066552051 >> 172.21.15.7:0/4066552051 conn(0x7f619c0cf910 msgr2=0x7f619c06dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:26.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.093+0000 7f61a0982700 1 -- 172.21.15.7:0/4066552051 shutdown_connections 2024-09-13T18:52:26.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.093+0000 7f61a0982700 1 -- 172.21.15.7:0/4066552051 wait complete. 2024-09-13T18:52:26.100 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-13T18:52:26.113 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-13T18:52:24.212538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235740,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738980,"statfs":{"total":773094113280,"available":772852715520,"internally_reserved":0,"allocated":3055616,"data_stored":1815780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.692640"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200870+0000","last_change":"2024-09-13T18:52:17.941085+0000","last_active":"2024-09-13T18:52:20.200870+0000","last_peered":"2024-09-13T18:52:20.200870+0000","last_clean":"2024-09-13T18:52:20.200870+0000","last_became_active":"2024-09-13T18:52:17.940741+0000","last_became_peered":"2024-09-13T18:52:17.940741+0000","last_unstale":"2024-09-13T18:52:20.200870+0000","last_undegraded":"2024-09-13T18:52:20.200870+0000","last_fullsized":"2024-09-13T18:52:20.200870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:50.609433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195285+0000","last_change":"2024-09-13T18:52:17.942350+0000","last_active":"2024-09-13T18:52:20.195285+0000","last_peered":"2024-09-13T18:52:20.195285+0000","last_clean":"2024-09-13T18:52:20.195285+0000","last_became_active":"2024-09-13T18:52:17.942062+0000","last_became_peered":"2024-09-13T18:52:17.942062+0000","last_unstale":"2024-09-13T18:52:20.195285+0000","last_undegraded":"2024-09-13T18:52:20.195285+0000","last_fullsized":"2024-09-13T18:52:20.195285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:45.768218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195540+0000","last_change":"2024-09-13T18:52:17.953055+0000","last_active":"2024-09-13T18:52:20.195540+0000","last_peered":"2024-09-13T18:52:20.195540+0000","last_clean":"2024-09-13T18:52:20.195540+0000","last_became_active":"2024-09-13T18:52:17.952806+0000","last_became_peered":"2024-09-13T18:52:17.952806+0000","last_unstale":"2024-09-13T18:52:20.195540+0000","last_undegraded":"2024-09-13T18:52:20.195540+0000","last_fullsized":"2024-09-13T18:52:20.195540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:18.149491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198959+0000","last_change":"2024-09-13T18:52:17.943472+0000","last_active":"2024-09-13T18:52:20.198959+0000","last_peered":"2024-09-13T18:52:20.198959+0000","last_clean":"2024-09-13T18:52:20.198959+0000","last_became_active":"2024-09-13T18:52:17.943070+0000","last_became_peered":"2024-09-13T18:52:17.943070+0000","last_unstale":"2024-09-13T18:52:20.198959+0000","last_undegraded":"2024-09-13T18:52:20.198959+0000","last_fullsized":"2024-09-13T18:52:20.198959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.990929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198942+0000","last_change":"2024-09-13T18:52:17.957081+0000","last_active":"2024-09-13T18:52:20.198942+0000","last_peered":"2024-09-13T18:52:20.198942+0000","last_clean":"2024-09-13T18:52:20.198942+0000","last_became_active":"2024-09-13T18:52:17.956787+0000","last_became_peered":"2024-09-13T18:52:17.956787+0000","last_unstale":"2024-09-13T18:52:20.198942+0000","last_undegraded":"2024-09-13T18:52:20.198942+0000","last_fullsized":"2024-09-13T18:52:20.198942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:31.026983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198843+0000","last_change":"2024-09-13T18:52:17.944754+0000","last_active":"2024-09-13T18:52:20.198843+0000","last_peered":"2024-09-13T18:52:20.198843+0000","last_clean":"2024-09-13T18:52:20.198843+0000","last_became_active":"2024-09-13T18:52:17.944495+0000","last_became_peered":"2024-09-13T18:52:17.944495+0000","last_unstale":"2024-09-13T18:52:20.198843+0000","last_undegraded":"2024-09-13T18:52:20.198843+0000","last_fullsized":"2024-09-13T18:52:20.198843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:40:09.595666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198289+0000","last_change":"2024-09-13T18:52:17.945805+0000","last_active":"2024-09-13T18:52:20.198289+0000","last_peered":"2024-09-13T18:52:20.198289+0000","last_clean":"2024-09-13T18:52:20.198289+0000","last_became_active":"2024-09-13T18:52:17.945364+0000","last_became_peered":"2024-09-13T18:52:17.945364+0000","last_unstale":"2024-09-13T18:52:20.198289+0000","last_undegraded":"2024-09-13T18:52:20.198289+0000","last_fullsized":"2024-09-13T18:52:20.198289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:43.043485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.923839+0000","last_change":"2024-09-13T18:52:17.939184+0000","last_active":"2024-09-13T18:52:18.923839+0000","last_peered":"2024-09-13T18:52:18.923839+0000","last_clean":"2024-09-13T18:52:18.923839+0000","last_became_active":"2024-09-13T18:52:17.938855+0000","last_became_peered":"2024-09-13T18:52:17.938855+0000","last_unstale":"2024-09-13T18:52:18.923839+0000","last_undegraded":"2024-09-13T18:52:18.923839+0000","last_fullsized":"2024-09-13T18:52:18.923839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:45.768218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200942+0000","last_change":"2024-09-13T18:52:17.940244+0000","last_active":"2024-09-13T18:52:20.200942+0000","last_peered":"2024-09-13T18:52:20.200942+0000","last_clean":"2024-09-13T18:52:20.200942+0000","last_became_active":"2024-09-13T18:52:17.939977+0000","last_became_peered":"2024-09-13T18:52:17.939977+0000","last_unstale":"2024-09-13T18:52:20.200942+0000","last_undegraded":"2024-09-13T18:52:20.200942+0000","last_fullsized":"2024-09-13T18:52:20.200942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:21.405599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.928272+0000","last_change":"2024-09-13T18:52:17.940027+0000","last_active":"2024-09-13T18:52:18.928272+0000","last_peered":"2024-09-13T18:52:18.928272+0000","last_clean":"2024-09-13T18:52:18.928272+0000","last_became_active":"2024-09-13T18:52:17.939743+0000","last_became_peered":"2024-09-13T18:52:17.939743+0000","last_unstale":"2024-09-13T18:52:18.928272+0000","last_undegraded":"2024-09-13T18:52:18.928272+0000","last_fullsized":"2024-09-13T18:52:18.928272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:26.228888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927956+0000","last_change":"2024-09-13T18:52:17.940006+0000","last_active":"2024-09-13T18:52:18.927956+0000","last_peered":"2024-09-13T18:52:18.927956+0000","last_clean":"2024-09-13T18:52:18.927956+0000","last_became_active":"2024-09-13T18:52:17.939474+0000","last_became_peered":"2024-09-13T18:52:17.939474+0000","last_unstale":"2024-09-13T18:52:18.927956+0000","last_undegraded":"2024-09-13T18:52:18.927956+0000","last_fullsized":"2024-09-13T18:52:18.927956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:24.025045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198153+0000","last_change":"2024-09-13T18:52:17.942513+0000","last_active":"2024-09-13T18:52:20.198153+0000","last_peered":"2024-09-13T18:52:20.198153+0000","last_clean":"2024-09-13T18:52:20.198153+0000","last_became_active":"2024-09-13T18:52:17.940486+0000","last_became_peered":"2024-09-13T18:52:17.940486+0000","last_unstale":"2024-09-13T18:52:20.198153+0000","last_undegraded":"2024-09-13T18:52:20.198153+0000","last_fullsized":"2024-09-13T18:52:20.198153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:08.839071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924421+0000","last_change":"2024-09-13T18:52:17.935439+0000","last_active":"2024-09-13T18:52:18.924421+0000","last_peered":"2024-09-13T18:52:18.924421+0000","last_clean":"2024-09-13T18:52:18.924421+0000","last_became_active":"2024-09-13T18:52:17.935114+0000","last_became_peered":"2024-09-13T18:52:17.935114+0000","last_unstale":"2024-09-13T18:52:18.924421+0000","last_undegraded":"2024-09-13T18:52:18.924421+0000","last_fullsized":"2024-09-13T18:52:18.924421+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:05:38.704891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195436+0000","last_change":"2024-09-13T18:52:17.942475+0000","last_active":"2024-09-13T18:52:20.195436+0000","last_peered":"2024-09-13T18:52:20.195436+0000","last_clean":"2024-09-13T18:52:20.195436+0000","last_became_active":"2024-09-13T18:52:17.942233+0000","last_became_peered":"2024-09-13T18:52:17.942233+0000","last_unstale":"2024-09-13T18:52:20.195436+0000","last_undegraded":"2024-09-13T18:52:20.195436+0000","last_fullsized":"2024-09-13T18:52:20.195436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:24.025045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.194709+0000","last_change":"2024-09-13T18:52:17.926146+0000","last_active":"2024-09-13T18:52:20.194709+0000","last_peered":"2024-09-13T18:52:20.194709+0000","last_clean":"2024-09-13T18:52:20.194709+0000","last_became_active":"2024-09-13T18:52:17.925837+0000","last_became_peered":"2024-09-13T18:52:17.925837+0000","last_unstale":"2024-09-13T18:52:20.194709+0000","last_undegraded":"2024-09-13T18:52:20.194709+0000","last_fullsized":"2024-09-13T18:52:20.194709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:30.871944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196715+0000","last_change":"2024-09-13T18:52:17.945764+0000","last_active":"2024-09-13T18:52:20.196715+0000","last_peered":"2024-09-13T18:52:20.196715+0000","last_clean":"2024-09-13T18:52:20.196715+0000","last_became_active":"2024-09-13T18:52:17.944484+0000","last_became_peered":"2024-09-13T18:52:17.944484+0000","last_unstale":"2024-09-13T18:52:20.196715+0000","last_undegraded":"2024-09-13T18:52:20.196715+0000","last_fullsized":"2024-09-13T18:52:20.196715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:45.768218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199740+0000","last_change":"2024-09-13T18:52:17.943317+0000","last_active":"2024-09-13T18:52:20.199740+0000","last_peered":"2024-09-13T18:52:20.199740+0000","last_clean":"2024-09-13T18:52:20.199740+0000","last_became_active":"2024-09-13T18:52:17.943015+0000","last_became_peered":"2024-09-13T18:52:17.943015+0000","last_unstale":"2024-09-13T18:52:20.199740+0000","last_undegraded":"2024-09-13T18:52:20.199740+0000","last_fullsized":"2024-09-13T18:52:20.199740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:50.609433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925962+0000","last_change":"2024-09-13T18:52:17.956474+0000","last_active":"2024-09-13T18:52:18.925962+0000","last_peered":"2024-09-13T18:52:18.925962+0000","last_clean":"2024-09-13T18:52:18.925962+0000","last_became_active":"2024-09-13T18:52:17.956200+0000","last_became_peered":"2024-09-13T18:52:17.956200+0000","last_unstale":"2024-09-13T18:52:18.925962+0000","last_undegraded":"2024-09-13T18:52:18.925962+0000","last_fullsized":"2024-09-13T18:52:18.925962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:17:56.476262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200865+0000","last_change":"2024-09-13T18:52:17.955849+0000","last_active":"2024-09-13T18:52:20.200865+0000","last_peered":"2024-09-13T18:52:20.200865+0000","last_clean":"2024-09-13T18:52:20.200865+0000","last_became_active":"2024-09-13T18:52:17.955606+0000","last_became_peered":"2024-09-13T18:52:17.955606+0000","last_unstale":"2024-09-13T18:52:20.200865+0000","last_undegraded":"2024-09-13T18:52:20.200865+0000","last_fullsized":"2024-09-13T18:52:20.200865+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:45.768218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927684+0000","last_change":"2024-09-13T18:52:17.947271+0000","last_active":"2024-09-13T18:52:18.927684+0000","last_peered":"2024-09-13T18:52:18.927684+0000","last_clean":"2024-09-13T18:52:18.927684+0000","last_became_active":"2024-09-13T18:52:17.947018+0000","last_became_peered":"2024-09-13T18:52:17.947018+0000","last_unstale":"2024-09-13T18:52:18.927684+0000","last_undegraded":"2024-09-13T18:52:18.927684+0000","last_fullsized":"2024-09-13T18:52:18.927684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:45.768218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195004+0000","last_change":"2024-09-13T18:52:17.949499+0000","last_active":"2024-09-13T18:52:20.195004+0000","last_peered":"2024-09-13T18:52:20.195004+0000","last_clean":"2024-09-13T18:52:20.195004+0000","last_became_active":"2024-09-13T18:52:17.949233+0000","last_became_peered":"2024-09-13T18:52:17.949233+0000","last_unstale":"2024-09-13T18:52:20.195004+0000","last_undegraded":"2024-09-13T18:52:20.195004+0000","last_fullsized":"2024-09-13T18:52:20.195004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:05:38.704891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197193+0000","last_change":"2024-09-13T18:52:17.941847+0000","last_active":"2024-09-13T18:52:20.197193+0000","last_peered":"2024-09-13T18:52:20.197193+0000","last_clean":"2024-09-13T18:52:20.197193+0000","last_became_active":"2024-09-13T18:52:17.941228+0000","last_became_peered":"2024-09-13T18:52:17.941228+0000","last_unstale":"2024-09-13T18:52:20.197193+0000","last_undegraded":"2024-09-13T18:52:20.197193+0000","last_fullsized":"2024-09-13T18:52:20.197193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:41.161193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924015+0000","last_change":"2024-09-13T18:52:17.924625+0000","last_active":"2024-09-13T18:52:18.924015+0000","last_peered":"2024-09-13T18:52:18.924015+0000","last_clean":"2024-09-13T18:52:18.924015+0000","last_became_active":"2024-09-13T18:52:17.923776+0000","last_became_peered":"2024-09-13T18:52:17.923776+0000","last_unstale":"2024-09-13T18:52:18.924015+0000","last_undegraded":"2024-09-13T18:52:18.924015+0000","last_fullsized":"2024-09-13T18:52:18.924015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:36.215528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196911+0000","last_change":"2024-09-13T18:52:17.938687+0000","last_active":"2024-09-13T18:52:20.196911+0000","last_peered":"2024-09-13T18:52:20.196911+0000","last_clean":"2024-09-13T18:52:20.196911+0000","last_became_active":"2024-09-13T18:52:17.938426+0000","last_became_peered":"2024-09-13T18:52:17.938426+0000","last_unstale":"2024-09-13T18:52:20.196911+0000","last_undegraded":"2024-09-13T18:52:20.196911+0000","last_fullsized":"2024-09-13T18:52:20.196911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:53.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924463+0000","last_change":"2024-09-13T18:52:17.924256+0000","last_active":"2024-09-13T18:52:18.924463+0000","last_peered":"2024-09-13T18:52:18.924463+0000","last_clean":"2024-09-13T18:52:18.924463+0000","last_became_active":"2024-09-13T18:52:17.923756+0000","last_became_peered":"2024-09-13T18:52:17.923756+0000","last_unstale":"2024-09-13T18:52:18.924463+0000","last_undegraded":"2024-09-13T18:52:18.924463+0000","last_fullsized":"2024-09-13T18:52:18.924463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:26.228888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.928375+0000","last_change":"2024-09-13T18:52:17.931734+0000","last_active":"2024-09-13T18:52:18.928375+0000","last_peered":"2024-09-13T18:52:18.928375+0000","last_clean":"2024-09-13T18:52:18.928375+0000","last_became_active":"2024-09-13T18:52:17.931034+0000","last_became_peered":"2024-09-13T18:52:17.931034+0000","last_unstale":"2024-09-13T18:52:18.928375+0000","last_undegraded":"2024-09-13T18:52:18.928375+0000","last_fullsized":"2024-09-13T18:52:18.928375+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:31.026983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.194568+0000","last_change":"2024-09-13T18:52:17.927130+0000","last_active":"2024-09-13T18:52:20.194568+0000","last_peered":"2024-09-13T18:52:20.194568+0000","last_clean":"2024-09-13T18:52:20.194568+0000","last_became_active":"2024-09-13T18:52:17.926099+0000","last_became_peered":"2024-09-13T18:52:17.926099+0000","last_unstale":"2024-09-13T18:52:20.194568+0000","last_undegraded":"2024-09-13T18:52:20.194568+0000","last_fullsized":"2024-09-13T18:52:20.194568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:05:38.704891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924857+0000","last_change":"2024-09-13T18:52:17.924220+0000","last_active":"2024-09-13T18:52:18.924857+0000","last_peered":"2024-09-13T18:52:18.924857+0000","last_clean":"2024-09-13T18:52:18.924857+0000","last_became_active":"2024-09-13T18:52:17.923830+0000","last_became_peered":"2024-09-13T18:52:17.923830+0000","last_unstale":"2024-09-13T18:52:18.924857+0000","last_undegraded":"2024-09-13T18:52:18.924857+0000","last_fullsized":"2024-09-13T18:52:18.924857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:24.025045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195245+0000","last_change":"2024-09-13T18:52:17.942480+0000","last_active":"2024-09-13T18:52:20.195245+0000","last_peered":"2024-09-13T18:52:20.195245+0000","last_clean":"2024-09-13T18:52:20.195245+0000","last_became_active":"2024-09-13T18:52:17.942156+0000","last_became_peered":"2024-09-13T18:52:17.942156+0000","last_unstale":"2024-09-13T18:52:20.195245+0000","last_undegraded":"2024-09-13T18:52:20.195245+0000","last_fullsized":"2024-09-13T18:52:20.195245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:30.871944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197425+0000","last_change":"2024-09-13T18:52:17.943608+0000","last_active":"2024-09-13T18:52:20.197425+0000","last_peered":"2024-09-13T18:52:20.197425+0000","last_clean":"2024-09-13T18:52:20.197425+0000","last_became_active":"2024-09-13T18:52:17.943314+0000","last_became_peered":"2024-09-13T18:52:17.943314+0000","last_unstale":"2024-09-13T18:52:20.197425+0000","last_undegraded":"2024-09-13T18:52:20.197425+0000","last_fullsized":"2024-09-13T18:52:20.197425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:43.043485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925985+0000","last_change":"2024-09-13T18:52:17.956342+0000","last_active":"2024-09-13T18:52:18.925985+0000","last_peered":"2024-09-13T18:52:18.925985+0000","last_clean":"2024-09-13T18:52:18.925985+0000","last_became_active":"2024-09-13T18:52:17.956066+0000","last_became_peered":"2024-09-13T18:52:17.956066+0000","last_unstale":"2024-09-13T18:52:18.925985+0000","last_undegraded":"2024-09-13T18:52:18.925985+0000","last_fullsized":"2024-09-13T18:52:18.925985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:24:03.326121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927056+0000","last_change":"2024-09-13T18:52:17.944931+0000","last_active":"2024-09-13T18:52:18.927056+0000","last_peered":"2024-09-13T18:52:18.927056+0000","last_clean":"2024-09-13T18:52:18.927056+0000","last_became_active":"2024-09-13T18:52:17.944522+0000","last_became_peered":"2024-09-13T18:52:17.944522+0000","last_unstale":"2024-09-13T18:52:18.927056+0000","last_undegraded":"2024-09-13T18:52:18.927056+0000","last_fullsized":"2024-09-13T18:52:18.927056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:36.215528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197266+0000","last_change":"2024-09-13T18:52:17.924921+0000","last_active":"2024-09-13T18:52:20.197266+0000","last_peered":"2024-09-13T18:52:20.197266+0000","last_clean":"2024-09-13T18:52:20.197266+0000","last_became_active":"2024-09-13T18:52:17.924708+0000","last_became_peered":"2024-09-13T18:52:17.924708+0000","last_unstale":"2024-09-13T18:52:20.197266+0000","last_undegraded":"2024-09-13T18:52:20.197266+0000","last_fullsized":"2024-09-13T18:52:20.197266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:25.786229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925450+0000","last_change":"2024-09-13T18:52:17.538391+0000","last_active":"2024-09-13T18:52:18.925450+0000","last_peered":"2024-09-13T18:52:18.925450+0000","last_clean":"2024-09-13T18:52:18.925450+0000","last_became_active":"2024-09-13T18:52:17.538114+0000","last_became_peered":"2024-09-13T18:52:17.538114+0000","last_unstale":"2024-09-13T18:52:18.925450+0000","last_undegraded":"2024-09-13T18:52:18.925450+0000","last_fullsized":"2024-09-13T18:52:18.925450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:04.695688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195477+0000","last_change":"2024-09-13T18:52:17.946924+0000","last_active":"2024-09-13T18:52:20.195477+0000","last_peered":"2024-09-13T18:52:20.195477+0000","last_clean":"2024-09-13T18:52:20.195477+0000","last_became_active":"2024-09-13T18:52:17.946599+0000","last_became_peered":"2024-09-13T18:52:17.946599+0000","last_unstale":"2024-09-13T18:52:20.195477+0000","last_undegraded":"2024-09-13T18:52:20.195477+0000","last_fullsized":"2024-09-13T18:52:20.195477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:26.228888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927670+0000","last_change":"2024-09-13T18:52:17.535479+0000","last_active":"2024-09-13T18:52:18.927670+0000","last_peered":"2024-09-13T18:52:18.927670+0000","last_clean":"2024-09-13T18:52:18.927670+0000","last_became_active":"2024-09-13T18:52:17.535185+0000","last_became_peered":"2024-09-13T18:52:17.535185+0000","last_unstale":"2024-09-13T18:52:18.927670+0000","last_undegraded":"2024-09-13T18:52:18.927670+0000","last_fullsized":"2024-09-13T18:52:18.927670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:47.942952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924218+0000","last_change":"2024-09-13T18:52:17.531093+0000","last_active":"2024-09-13T18:52:18.924218+0000","last_peered":"2024-09-13T18:52:18.924218+0000","last_clean":"2024-09-13T18:52:18.924218+0000","last_became_active":"2024-09-13T18:52:17.530499+0000","last_became_peered":"2024-09-13T18:52:17.530499+0000","last_unstale":"2024-09-13T18:52:18.924218+0000","last_undegraded":"2024-09-13T18:52:18.924218+0000","last_fullsized":"2024-09-13T18:52:18.924218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:33:25.953860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.923892+0000","last_change":"2024-09-13T18:52:17.940517+0000","last_active":"2024-09-13T18:52:18.923892+0000","last_peered":"2024-09-13T18:52:18.923892+0000","last_clean":"2024-09-13T18:52:18.923892+0000","last_became_active":"2024-09-13T18:52:17.940250+0000","last_became_peered":"2024-09-13T18:52:17.940250+0000","last_unstale":"2024-09-13T18:52:18.923892+0000","last_undegraded":"2024-09-13T18:52:18.923892+0000","last_fullsized":"2024-09-13T18:52:18.923892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.990929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196937+0000","last_change":"2024-09-13T18:52:17.539508+0000","last_active":"2024-09-13T18:52:20.196937+0000","last_peered":"2024-09-13T18:52:20.196937+0000","last_clean":"2024-09-13T18:52:20.196937+0000","last_became_active":"2024-09-13T18:52:17.539182+0000","last_became_peered":"2024-09-13T18:52:17.539182+0000","last_unstale":"2024-09-13T18:52:20.196937+0000","last_undegraded":"2024-09-13T18:52:20.196937+0000","last_fullsized":"2024-09-13T18:52:20.196937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:49.151192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.194851+0000","last_change":"2024-09-13T18:52:17.926798+0000","last_active":"2024-09-13T18:52:20.194851+0000","last_peered":"2024-09-13T18:52:20.194851+0000","last_clean":"2024-09-13T18:52:20.194851+0000","last_became_active":"2024-09-13T18:52:17.926182+0000","last_became_peered":"2024-09-13T18:52:17.926182+0000","last_unstale":"2024-09-13T18:52:20.194851+0000","last_undegraded":"2024-09-13T18:52:20.194851+0000","last_fullsized":"2024-09-13T18:52:20.194851+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:36.215528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.928162+0000","last_change":"2024-09-13T18:52:17.539633+0000","last_active":"2024-09-13T18:52:18.928162+0000","last_peered":"2024-09-13T18:52:18.928162+0000","last_clean":"2024-09-13T18:52:18.928162+0000","last_became_active":"2024-09-13T18:52:17.539340+0000","last_became_peered":"2024-09-13T18:52:17.539340+0000","last_unstale":"2024-09-13T18:52:18.928162+0000","last_undegraded":"2024-09-13T18:52:18.928162+0000","last_fullsized":"2024-09-13T18:52:18.928162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:06.694707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199408+0000","last_change":"2024-09-13T18:52:17.941631+0000","last_active":"2024-09-13T18:52:20.199408+0000","last_peered":"2024-09-13T18:52:20.199408+0000","last_clean":"2024-09-13T18:52:20.199408+0000","last_became_active":"2024-09-13T18:52:17.940875+0000","last_became_peered":"2024-09-13T18:52:17.940875+0000","last_unstale":"2024-09-13T18:52:20.199408+0000","last_undegraded":"2024-09-13T18:52:20.199408+0000","last_fullsized":"2024-09-13T18:52:20.199408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:42:22.626590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199507+0000","last_change":"2024-09-13T18:52:17.532300+0000","last_active":"2024-09-13T18:52:20.199507+0000","last_peered":"2024-09-13T18:52:20.199507+0000","last_clean":"2024-09-13T18:52:20.199507+0000","last_became_active":"2024-09-13T18:52:17.532019+0000","last_became_peered":"2024-09-13T18:52:17.532019+0000","last_unstale":"2024-09-13T18:52:20.199507+0000","last_undegraded":"2024-09-13T18:52:20.199507+0000","last_fullsized":"2024-09-13T18:52:20.199507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:05:28.094741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196062+0000","last_change":"2024-09-13T18:52:17.945629+0000","last_active":"2024-09-13T18:52:20.196062+0000","last_peered":"2024-09-13T18:52:20.196062+0000","last_clean":"2024-09-13T18:52:20.196062+0000","last_became_active":"2024-09-13T18:52:17.945372+0000","last_became_peered":"2024-09-13T18:52:17.945372+0000","last_unstale":"2024-09-13T18:52:20.196062+0000","last_undegraded":"2024-09-13T18:52:20.196062+0000","last_fullsized":"2024-09-13T18:52:20.196062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:36.215528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.194942+0000","last_change":"2024-09-13T18:52:17.538613+0000","last_active":"2024-09-13T18:52:20.194942+0000","last_peered":"2024-09-13T18:52:20.194942+0000","last_clean":"2024-09-13T18:52:20.194942+0000","last_became_active":"2024-09-13T18:52:17.538372+0000","last_became_peered":"2024-09-13T18:52:17.538372+0000","last_unstale":"2024-09-13T18:52:20.194942+0000","last_undegraded":"2024-09-13T18:52:20.194942+0000","last_fullsized":"2024-09-13T18:52:20.194942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:05:28.094741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198119+0000","last_change":"2024-09-13T18:52:17.941036+0000","last_active":"2024-09-13T18:52:20.198119+0000","last_peered":"2024-09-13T18:52:20.198119+0000","last_clean":"2024-09-13T18:52:20.198119+0000","last_became_active":"2024-09-13T18:52:17.940741+0000","last_became_peered":"2024-09-13T18:52:17.940741+0000","last_unstale":"2024-09-13T18:52:20.198119+0000","last_undegraded":"2024-09-13T18:52:20.198119+0000","last_fullsized":"2024-09-13T18:52:20.198119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:19:17.191755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927753+0000","last_change":"2024-09-13T18:52:17.536588+0000","last_active":"2024-09-13T18:52:18.927753+0000","last_peered":"2024-09-13T18:52:18.927753+0000","last_clean":"2024-09-13T18:52:18.927753+0000","last_became_active":"2024-09-13T18:52:17.536319+0000","last_became_peered":"2024-09-13T18:52:17.536319+0000","last_unstale":"2024-09-13T18:52:18.927753+0000","last_undegraded":"2024-09-13T18:52:18.927753+0000","last_fullsized":"2024-09-13T18:52:18.927753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:14:15.613948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199367+0000","last_change":"2024-09-13T18:52:17.935349+0000","last_active":"2024-09-13T18:52:20.199367+0000","last_peered":"2024-09-13T18:52:20.199367+0000","last_clean":"2024-09-13T18:52:20.199367+0000","last_became_active":"2024-09-13T18:52:17.935064+0000","last_became_peered":"2024-09-13T18:52:17.935064+0000","last_unstale":"2024-09-13T18:52:20.199367+0000","last_undegraded":"2024-09-13T18:52:20.199367+0000","last_fullsized":"2024-09-13T18:52:20.199367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:18.183823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195705+0000","last_change":"2024-09-13T18:52:17.538911+0000","last_active":"2024-09-13T18:52:20.195705+0000","last_peered":"2024-09-13T18:52:20.195705+0000","last_clean":"2024-09-13T18:52:20.195705+0000","last_became_active":"2024-09-13T18:52:17.538512+0000","last_became_peered":"2024-09-13T18:52:17.538512+0000","last_unstale":"2024-09-13T18:52:20.195705+0000","last_undegraded":"2024-09-13T18:52:20.195705+0000","last_fullsized":"2024-09-13T18:52:20.195705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:03:19.727396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195688+0000","last_change":"2024-09-13T18:52:17.949196+0000","last_active":"2024-09-13T18:52:20.195688+0000","last_peered":"2024-09-13T18:52:20.195688+0000","last_clean":"2024-09-13T18:52:20.195688+0000","last_became_active":"2024-09-13T18:52:17.948936+0000","last_became_peered":"2024-09-13T18:52:17.948936+0000","last_unstale":"2024-09-13T18:52:20.195688+0000","last_undegraded":"2024-09-13T18:52:20.195688+0000","last_fullsized":"2024-09-13T18:52:20.195688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:43:49.752608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197616+0000","last_change":"2024-09-13T18:52:17.534190+0000","last_active":"2024-09-13T18:52:20.197616+0000","last_peered":"2024-09-13T18:52:20.197616+0000","last_clean":"2024-09-13T18:52:20.197616+0000","last_became_active":"2024-09-13T18:52:17.532898+0000","last_became_peered":"2024-09-13T18:52:17.532898+0000","last_unstale":"2024-09-13T18:52:20.197616+0000","last_undegraded":"2024-09-13T18:52:20.197616+0000","last_fullsized":"2024-09-13T18:52:20.197616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:14:15.613948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200129+0000","last_change":"2024-09-13T18:52:17.947584+0000","last_active":"2024-09-13T18:52:20.200129+0000","last_peered":"2024-09-13T18:52:20.200129+0000","last_clean":"2024-09-13T18:52:20.200129+0000","last_became_active":"2024-09-13T18:52:17.947333+0000","last_became_peered":"2024-09-13T18:52:17.947333+0000","last_unstale":"2024-09-13T18:52:20.200129+0000","last_undegraded":"2024-09-13T18:52:20.200129+0000","last_fullsized":"2024-09-13T18:52:20.200129+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:53.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927484+0000","last_change":"2024-09-13T18:52:17.539689+0000","last_active":"2024-09-13T18:52:18.927484+0000","last_peered":"2024-09-13T18:52:18.927484+0000","last_clean":"2024-09-13T18:52:18.927484+0000","last_became_active":"2024-09-13T18:52:17.539435+0000","last_became_peered":"2024-09-13T18:52:17.539435+0000","last_unstale":"2024-09-13T18:52:18.927484+0000","last_undegraded":"2024-09-13T18:52:18.927484+0000","last_fullsized":"2024-09-13T18:52:18.927484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:22:26.864392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927431+0000","last_change":"2024-09-13T18:52:17.944767+0000","last_active":"2024-09-13T18:52:18.927431+0000","last_peered":"2024-09-13T18:52:18.927431+0000","last_clean":"2024-09-13T18:52:18.927431+0000","last_became_active":"2024-09-13T18:52:17.944163+0000","last_became_peered":"2024-09-13T18:52:17.944163+0000","last_unstale":"2024-09-13T18:52:18.927431+0000","last_undegraded":"2024-09-13T18:52:18.927431+0000","last_fullsized":"2024-09-13T18:52:18.927431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:30.871944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925915+0000","last_change":"2024-09-13T18:52:17.535230+0000","last_active":"2024-09-13T18:52:18.925915+0000","last_peered":"2024-09-13T18:52:18.925915+0000","last_clean":"2024-09-13T18:52:18.925915+0000","last_became_active":"2024-09-13T18:52:17.534923+0000","last_became_peered":"2024-09-13T18:52:17.534923+0000","last_unstale":"2024-09-13T18:52:18.925915+0000","last_undegraded":"2024-09-13T18:52:18.925915+0000","last_fullsized":"2024-09-13T18:52:18.925915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:47:07.580322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198532+0000","last_change":"2024-09-13T18:52:17.949849+0000","last_active":"2024-09-13T18:52:20.198532+0000","last_peered":"2024-09-13T18:52:20.198532+0000","last_clean":"2024-09-13T18:52:20.198532+0000","last_became_active":"2024-09-13T18:52:17.949503+0000","last_became_peered":"2024-09-13T18:52:17.949503+0000","last_unstale":"2024-09-13T18:52:20.198532+0000","last_undegraded":"2024-09-13T18:52:20.198532+0000","last_fullsized":"2024-09-13T18:52:20.198532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:30.871944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198430+0000","last_change":"2024-09-13T18:52:17.540059+0000","last_active":"2024-09-13T18:52:20.198430+0000","last_peered":"2024-09-13T18:52:20.198430+0000","last_clean":"2024-09-13T18:52:20.198430+0000","last_became_active":"2024-09-13T18:52:17.539768+0000","last_became_peered":"2024-09-13T18:52:17.539768+0000","last_unstale":"2024-09-13T18:52:20.198430+0000","last_undegraded":"2024-09-13T18:52:20.198430+0000","last_fullsized":"2024-09-13T18:52:20.198430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:33.879872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200908+0000","last_change":"2024-09-13T18:52:17.947339+0000","last_active":"2024-09-13T18:52:20.200908+0000","last_peered":"2024-09-13T18:52:20.200908+0000","last_clean":"2024-09-13T18:52:20.200908+0000","last_became_active":"2024-09-13T18:52:17.947073+0000","last_became_peered":"2024-09-13T18:52:17.947073+0000","last_unstale":"2024-09-13T18:52:20.200908+0000","last_undegraded":"2024-09-13T18:52:20.200908+0000","last_fullsized":"2024-09-13T18:52:20.200908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:36.215528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925944+0000","last_change":"2024-09-13T18:52:17.534989+0000","last_active":"2024-09-13T18:52:18.925944+0000","last_peered":"2024-09-13T18:52:18.925944+0000","last_clean":"2024-09-13T18:52:18.925944+0000","last_became_active":"2024-09-13T18:52:17.532634+0000","last_became_peered":"2024-09-13T18:52:17.532634+0000","last_unstale":"2024-09-13T18:52:18.925944+0000","last_undegraded":"2024-09-13T18:52:18.925944+0000","last_fullsized":"2024-09-13T18:52:18.925944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:10:12.228931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200958+0000","last_change":"2024-09-13T18:52:17.949744+0000","last_active":"2024-09-13T18:52:20.200958+0000","last_peered":"2024-09-13T18:52:20.200958+0000","last_clean":"2024-09-13T18:52:20.200958+0000","last_became_active":"2024-09-13T18:52:17.949469+0000","last_became_peered":"2024-09-13T18:52:17.949469+0000","last_unstale":"2024-09-13T18:52:20.200958+0000","last_undegraded":"2024-09-13T18:52:20.200958+0000","last_fullsized":"2024-09-13T18:52:20.200958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:42:22.626590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195041+0000","last_change":"2024-09-13T18:52:17.537724+0000","last_active":"2024-09-13T18:52:20.195041+0000","last_peered":"2024-09-13T18:52:20.195041+0000","last_clean":"2024-09-13T18:52:20.195041+0000","last_became_active":"2024-09-13T18:52:17.537474+0000","last_became_peered":"2024-09-13T18:52:17.537474+0000","last_unstale":"2024-09-13T18:52:20.195041+0000","last_undegraded":"2024-09-13T18:52:20.195041+0000","last_fullsized":"2024-09-13T18:52:20.195041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:06.694707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199207+0000","last_change":"2024-09-13T18:52:17.939592+0000","last_active":"2024-09-13T18:52:20.199207+0000","last_peered":"2024-09-13T18:52:20.199207+0000","last_clean":"2024-09-13T18:52:20.199207+0000","last_became_active":"2024-09-13T18:52:17.939331+0000","last_became_peered":"2024-09-13T18:52:17.939331+0000","last_unstale":"2024-09-13T18:52:20.199207+0000","last_undegraded":"2024-09-13T18:52:20.199207+0000","last_fullsized":"2024-09-13T18:52:20.199207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:31.026983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.926147+0000","last_change":"2024-09-13T18:52:17.536715+0000","last_active":"2024-09-13T18:52:18.926147+0000","last_peered":"2024-09-13T18:52:18.926147+0000","last_clean":"2024-09-13T18:52:18.926147+0000","last_became_active":"2024-09-13T18:52:17.536409+0000","last_became_peered":"2024-09-13T18:52:17.536409+0000","last_unstale":"2024-09-13T18:52:18.926147+0000","last_undegraded":"2024-09-13T18:52:18.926147+0000","last_fullsized":"2024-09-13T18:52:18.926147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:01.849550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200621+0000","last_change":"2024-09-13T18:52:17.940926+0000","last_active":"2024-09-13T18:52:20.200621+0000","last_peered":"2024-09-13T18:52:20.200621+0000","last_clean":"2024-09-13T18:52:20.200621+0000","last_became_active":"2024-09-13T18:52:17.940572+0000","last_became_peered":"2024-09-13T18:52:17.940572+0000","last_unstale":"2024-09-13T18:52:20.200621+0000","last_undegraded":"2024-09-13T18:52:20.200621+0000","last_fullsized":"2024-09-13T18:52:20.200621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.990929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T18:52:17.928742+0000","last_change":"2024-09-13T18:52:15.080080+0000","last_active":"2024-09-13T18:52:17.928742+0000","last_peered":"2024-09-13T18:52:17.928742+0000","last_clean":"2024-09-13T18:52:17.928742+0000","last_became_active":"2024-09-13T18:52:15.078100+0000","last_became_peered":"2024-09-13T18:52:15.078100+0000","last_unstale":"2024-09-13T18:52:17.928742+0000","last_undegraded":"2024-09-13T18:52:17.928742+0000","last_fullsized":"2024-09-13T18:52:17.928742+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_clean_scrub_stamp":"2024-09-13T18:52:13.196473+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:29:59.452757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197092+0000","last_change":"2024-09-13T18:52:17.531117+0000","last_active":"2024-09-13T18:52:20.197092+0000","last_peered":"2024-09-13T18:52:20.197092+0000","last_clean":"2024-09-13T18:52:20.197092+0000","last_became_active":"2024-09-13T18:52:17.530778+0000","last_became_peered":"2024-09-13T18:52:17.530778+0000","last_unstale":"2024-09-13T18:52:20.197092+0000","last_undegraded":"2024-09-13T18:52:20.197092+0000","last_fullsized":"2024-09-13T18:52:20.197092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:06.694707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924347+0000","last_change":"2024-09-13T18:52:17.943655+0000","last_active":"2024-09-13T18:52:18.924347+0000","last_peered":"2024-09-13T18:52:18.924347+0000","last_clean":"2024-09-13T18:52:18.924347+0000","last_became_active":"2024-09-13T18:52:17.943301+0000","last_became_peered":"2024-09-13T18:52:17.943301+0000","last_unstale":"2024-09-13T18:52:18.924347+0000","last_undegraded":"2024-09-13T18:52:18.924347+0000","last_fullsized":"2024-09-13T18:52:18.924347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:53.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197196+0000","last_change":"2024-09-13T18:52:17.548220+0000","last_active":"2024-09-13T18:52:20.197196+0000","last_peered":"2024-09-13T18:52:20.197196+0000","last_clean":"2024-09-13T18:52:20.197196+0000","last_became_active":"2024-09-13T18:52:17.547957+0000","last_became_peered":"2024-09-13T18:52:17.547957+0000","last_unstale":"2024-09-13T18:52:20.197196+0000","last_undegraded":"2024-09-13T18:52:20.197196+0000","last_fullsized":"2024-09-13T18:52:20.197196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:47.942952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.194587+0000","last_change":"2024-09-13T18:52:17.946587+0000","last_active":"2024-09-13T18:52:20.194587+0000","last_peered":"2024-09-13T18:52:20.194587+0000","last_clean":"2024-09-13T18:52:20.194587+0000","last_became_active":"2024-09-13T18:52:17.946352+0000","last_became_peered":"2024-09-13T18:52:17.946352+0000","last_unstale":"2024-09-13T18:52:20.194587+0000","last_undegraded":"2024-09-13T18:52:20.194587+0000","last_fullsized":"2024-09-13T18:52:20.194587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:37.404109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.926130+0000","last_change":"2024-09-13T18:52:17.532972+0000","last_active":"2024-09-13T18:52:18.926130+0000","last_peered":"2024-09-13T18:52:18.926130+0000","last_clean":"2024-09-13T18:52:18.926130+0000","last_became_active":"2024-09-13T18:52:17.532675+0000","last_became_peered":"2024-09-13T18:52:17.532675+0000","last_unstale":"2024-09-13T18:52:18.926130+0000","last_undegraded":"2024-09-13T18:52:18.926130+0000","last_fullsized":"2024-09-13T18:52:18.926130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:41.232168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.194709+0000","last_change":"2024-09-13T18:52:17.955502+0000","last_active":"2024-09-13T18:52:20.194709+0000","last_peered":"2024-09-13T18:52:20.194709+0000","last_clean":"2024-09-13T18:52:20.194709+0000","last_became_active":"2024-09-13T18:52:17.955061+0000","last_became_peered":"2024-09-13T18:52:17.955061+0000","last_unstale":"2024-09-13T18:52:20.194709+0000","last_undegraded":"2024-09-13T18:52:20.194709+0000","last_fullsized":"2024-09-13T18:52:20.194709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:24.025045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197805+0000","last_change":"2024-09-13T18:52:17.535698+0000","last_active":"2024-09-13T18:52:20.197805+0000","last_peered":"2024-09-13T18:52:20.197805+0000","last_clean":"2024-09-13T18:52:20.197805+0000","last_became_active":"2024-09-13T18:52:17.535439+0000","last_became_peered":"2024-09-13T18:52:17.535439+0000","last_unstale":"2024-09-13T18:52:20.197805+0000","last_undegraded":"2024-09-13T18:52:20.197805+0000","last_fullsized":"2024-09-13T18:52:20.197805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:47.942952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197950+0000","last_change":"2024-09-13T18:52:17.945038+0000","last_active":"2024-09-13T18:52:20.197950+0000","last_peered":"2024-09-13T18:52:20.197950+0000","last_clean":"2024-09-13T18:52:20.197950+0000","last_became_active":"2024-09-13T18:52:17.944770+0000","last_became_peered":"2024-09-13T18:52:17.944770+0000","last_unstale":"2024-09-13T18:52:20.197950+0000","last_undegraded":"2024-09-13T18:52:20.197950+0000","last_fullsized":"2024-09-13T18:52:20.197950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:24:03.326121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196209+0000","last_change":"2024-09-13T18:52:17.539013+0000","last_active":"2024-09-13T18:52:20.196209+0000","last_peered":"2024-09-13T18:52:20.196209+0000","last_clean":"2024-09-13T18:52:20.196209+0000","last_became_active":"2024-09-13T18:52:17.538720+0000","last_became_peered":"2024-09-13T18:52:17.538720+0000","last_unstale":"2024-09-13T18:52:20.196209+0000","last_undegraded":"2024-09-13T18:52:20.196209+0000","last_fullsized":"2024-09-13T18:52:20.196209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:49.151192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199583+0000","last_change":"2024-09-13T18:52:17.943875+0000","last_active":"2024-09-13T18:52:20.199583+0000","last_peered":"2024-09-13T18:52:20.199583+0000","last_clean":"2024-09-13T18:52:20.199583+0000","last_became_active":"2024-09-13T18:52:17.943557+0000","last_became_peered":"2024-09-13T18:52:17.943557+0000","last_unstale":"2024-09-13T18:52:20.199583+0000","last_undegraded":"2024-09-13T18:52:20.199583+0000","last_fullsized":"2024-09-13T18:52:20.199583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:18.183823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200835+0000","last_change":"2024-09-13T18:52:17.548571+0000","last_active":"2024-09-13T18:52:20.200835+0000","last_peered":"2024-09-13T18:52:20.200835+0000","last_clean":"2024-09-13T18:52:20.200835+0000","last_became_active":"2024-09-13T18:52:17.548306+0000","last_became_peered":"2024-09-13T18:52:17.548306+0000","last_unstale":"2024-09-13T18:52:20.200835+0000","last_undegraded":"2024-09-13T18:52:20.200835+0000","last_fullsized":"2024-09-13T18:52:20.200835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:22:26.864392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195209+0000","last_change":"2024-09-13T18:52:17.954517+0000","last_active":"2024-09-13T18:52:20.195209+0000","last_peered":"2024-09-13T18:52:20.195209+0000","last_clean":"2024-09-13T18:52:20.195209+0000","last_became_active":"2024-09-13T18:52:17.954264+0000","last_became_peered":"2024-09-13T18:52:17.954264+0000","last_unstale":"2024-09-13T18:52:20.195209+0000","last_undegraded":"2024-09-13T18:52:20.195209+0000","last_fullsized":"2024-09-13T18:52:20.195209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:57:51.000231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:21.214627+0000","last_change":"2024-09-13T18:52:17.538927+0000","last_active":"2024-09-13T18:52:21.214627+0000","last_peered":"2024-09-13T18:52:21.214627+0000","last_clean":"2024-09-13T18:52:21.214627+0000","last_became_active":"2024-09-13T18:52:17.538334+0000","last_became_peered":"2024-09-13T18:52:17.538334+0000","last_unstale":"2024-09-13T18:52:21.214627+0000","last_undegraded":"2024-09-13T18:52:21.214627+0000","last_fullsized":"2024-09-13T18:52:21.214627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:47.942952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924572+0000","last_change":"2024-09-13T18:52:17.929155+0000","last_active":"2024-09-13T18:52:18.924572+0000","last_peered":"2024-09-13T18:52:18.924572+0000","last_clean":"2024-09-13T18:52:18.924572+0000","last_became_active":"2024-09-13T18:52:17.928876+0000","last_became_peered":"2024-09-13T18:52:17.928876+0000","last_unstale":"2024-09-13T18:52:18.924572+0000","last_undegraded":"2024-09-13T18:52:18.924572+0000","last_fullsized":"2024-09-13T18:52:18.924572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:01:39.422836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.928355+0000","last_change":"2024-09-13T18:52:17.543500+0000","last_active":"2024-09-13T18:52:18.928355+0000","last_peered":"2024-09-13T18:52:18.928355+0000","last_clean":"2024-09-13T18:52:18.928355+0000","last_became_active":"2024-09-13T18:52:17.543184+0000","last_became_peered":"2024-09-13T18:52:17.543184+0000","last_unstale":"2024-09-13T18:52:18.928355+0000","last_undegraded":"2024-09-13T18:52:18.928355+0000","last_fullsized":"2024-09-13T18:52:18.928355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:05:28.094741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195839+0000","last_change":"2024-09-13T18:52:17.942277+0000","last_active":"2024-09-13T18:52:20.195839+0000","last_peered":"2024-09-13T18:52:20.195839+0000","last_clean":"2024-09-13T18:52:20.195839+0000","last_became_active":"2024-09-13T18:52:17.941984+0000","last_became_peered":"2024-09-13T18:52:17.941984+0000","last_unstale":"2024-09-13T18:52:20.195839+0000","last_undegraded":"2024-09-13T18:52:20.195839+0000","last_fullsized":"2024-09-13T18:52:20.195839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:20:42.480429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197001+0000","last_change":"2024-09-13T18:52:17.545742+0000","last_active":"2024-09-13T18:52:20.197001+0000","last_peered":"2024-09-13T18:52:20.197001+0000","last_clean":"2024-09-13T18:52:20.197001+0000","last_became_active":"2024-09-13T18:52:17.545448+0000","last_became_peered":"2024-09-13T18:52:17.545448+0000","last_unstale":"2024-09-13T18:52:20.197001+0000","last_undegraded":"2024-09-13T18:52:20.197001+0000","last_fullsized":"2024-09-13T18:52:20.197001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:49.151192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196848+0000","last_change":"2024-09-13T18:52:17.955942+0000","last_active":"2024-09-13T18:52:20.196848+0000","last_peered":"2024-09-13T18:52:20.196848+0000","last_clean":"2024-09-13T18:52:20.196848+0000","last_became_active":"2024-09-13T18:52:17.955598+0000","last_became_peered":"2024-09-13T18:52:17.955598+0000","last_unstale":"2024-09-13T18:52:20.196848+0000","last_undegraded":"2024-09-13T18:52:20.196848+0000","last_fullsized":"2024-09-13T18:52:20.196848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:42.236037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196212+0000","last_change":"2024-09-13T18:52:17.541045+0000","last_active":"2024-09-13T18:52:20.196212+0000","last_peered":"2024-09-13T18:52:20.196212+0000","last_clean":"2024-09-13T18:52:20.196212+0000","last_became_active":"2024-09-13T18:52:17.540627+0000","last_became_peered":"2024-09-13T18:52:17.540627+0000","last_unstale":"2024-09-13T18:52:20.196212+0000","last_undegraded":"2024-09-13T18:52:20.196212+0000","last_fullsized":"2024-09-13T18:52:20.196212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:14:15.613948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924137+0000","last_change":"2024-09-13T18:52:17.924064+0000","last_active":"2024-09-13T18:52:18.924137+0000","last_peered":"2024-09-13T18:52:18.924137+0000","last_clean":"2024-09-13T18:52:18.924137+0000","last_became_active":"2024-09-13T18:52:17.921690+0000","last_became_peered":"2024-09-13T18:52:17.921690+0000","last_unstale":"2024-09-13T18:52:18.924137+0000","last_undegraded":"2024-09-13T18:52:18.924137+0000","last_fullsized":"2024-09-13T18:52:18.924137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:30.871944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195867+0000","last_change":"2024-09-13T18:52:17.533845+0000","last_active":"2024-09-13T18:52:20.195867+0000","last_peered":"2024-09-13T18:52:20.195867+0000","last_clean":"2024-09-13T18:52:20.195867+0000","last_became_active":"2024-09-13T18:52:17.533579+0000","last_became_peered":"2024-09-13T18:52:17.533579+0000","last_unstale":"2024-09-13T18:52:20.195867+0000","last_undegraded":"2024-09-13T18:52:20.195867+0000","last_fullsized":"2024-09-13T18:52:20.195867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:04.695688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925441+0000","last_change":"2024-09-13T18:52:17.944835+0000","last_active":"2024-09-13T18:52:18.925441+0000","last_peered":"2024-09-13T18:52:18.925441+0000","last_clean":"2024-09-13T18:52:18.925441+0000","last_became_active":"2024-09-13T18:52:17.944469+0000","last_became_peered":"2024-09-13T18:52:17.944469+0000","last_unstale":"2024-09-13T18:52:18.925441+0000","last_undegraded":"2024-09-13T18:52:18.925441+0000","last_fullsized":"2024-09-13T18:52:18.925441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:31.026983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200263+0000","last_change":"2024-09-13T18:52:17.545810+0000","last_active":"2024-09-13T18:52:20.200263+0000","last_peered":"2024-09-13T18:52:20.200263+0000","last_clean":"2024-09-13T18:52:20.200263+0000","last_became_active":"2024-09-13T18:52:17.545523+0000","last_became_peered":"2024-09-13T18:52:17.545523+0000","last_unstale":"2024-09-13T18:52:20.200263+0000","last_undegraded":"2024-09-13T18:52:20.200263+0000","last_fullsized":"2024-09-13T18:52:20.200263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:05:28.094741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927471+0000","last_change":"2024-09-13T18:52:17.940857+0000","last_active":"2024-09-13T18:52:18.927471+0000","last_peered":"2024-09-13T18:52:18.927471+0000","last_clean":"2024-09-13T18:52:18.927471+0000","last_became_active":"2024-09-13T18:52:17.940675+0000","last_became_peered":"2024-09-13T18:52:17.940675+0000","last_unstale":"2024-09-13T18:52:18.927471+0000","last_undegraded":"2024-09-13T18:52:18.927471+0000","last_fullsized":"2024-09-13T18:52:18.927471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:53.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199367+0000","last_change":"2024-09-13T18:52:17.947933+0000","last_active":"2024-09-13T18:52:20.199367+0000","last_peered":"2024-09-13T18:52:20.199367+0000","last_clean":"2024-09-13T18:52:20.199367+0000","last_became_active":"2024-09-13T18:52:17.947665+0000","last_became_peered":"2024-09-13T18:52:17.947665+0000","last_unstale":"2024-09-13T18:52:20.199367+0000","last_undegraded":"2024-09-13T18:52:20.199367+0000","last_fullsized":"2024-09-13T18:52:20.199367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:21.405599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199849+0000","last_change":"2024-09-13T18:52:17.548211+0000","last_active":"2024-09-13T18:52:20.199849+0000","last_peered":"2024-09-13T18:52:20.199849+0000","last_clean":"2024-09-13T18:52:20.199849+0000","last_became_active":"2024-09-13T18:52:17.547997+0000","last_became_peered":"2024-09-13T18:52:17.547997+0000","last_unstale":"2024-09-13T18:52:20.199849+0000","last_undegraded":"2024-09-13T18:52:20.199849+0000","last_fullsized":"2024-09-13T18:52:20.199849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:06.694707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199118+0000","last_change":"2024-09-13T18:52:17.948133+0000","last_active":"2024-09-13T18:52:20.199118+0000","last_peered":"2024-09-13T18:52:20.199118+0000","last_clean":"2024-09-13T18:52:20.199118+0000","last_became_active":"2024-09-13T18:52:17.947793+0000","last_became_peered":"2024-09-13T18:52:17.947793+0000","last_unstale":"2024-09-13T18:52:20.199118+0000","last_undegraded":"2024-09-13T18:52:20.199118+0000","last_fullsized":"2024-09-13T18:52:20.199118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:20:43.355560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200592+0000","last_change":"2024-09-13T18:52:17.548922+0000","last_active":"2024-09-13T18:52:20.200592+0000","last_peered":"2024-09-13T18:52:20.200592+0000","last_clean":"2024-09-13T18:52:20.200592+0000","last_became_active":"2024-09-13T18:52:17.548775+0000","last_became_peered":"2024-09-13T18:52:17.548775+0000","last_unstale":"2024-09-13T18:52:20.200592+0000","last_undegraded":"2024-09-13T18:52:20.200592+0000","last_fullsized":"2024-09-13T18:52:20.200592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:14:15.613948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195999+0000","last_change":"2024-09-13T18:52:17.950416+0000","last_active":"2024-09-13T18:52:20.195999+0000","last_peered":"2024-09-13T18:52:20.195999+0000","last_clean":"2024-09-13T18:52:20.195999+0000","last_became_active":"2024-09-13T18:52:17.949988+0000","last_became_peered":"2024-09-13T18:52:17.949988+0000","last_unstale":"2024-09-13T18:52:20.195999+0000","last_undegraded":"2024-09-13T18:52:20.195999+0000","last_fullsized":"2024-09-13T18:52:20.195999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:45.768218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925140+0000","last_change":"2024-09-13T18:52:17.535536+0000","last_active":"2024-09-13T18:52:18.925140+0000","last_peered":"2024-09-13T18:52:18.925140+0000","last_clean":"2024-09-13T18:52:18.925140+0000","last_became_active":"2024-09-13T18:52:17.534924+0000","last_became_peered":"2024-09-13T18:52:17.534924+0000","last_unstale":"2024-09-13T18:52:18.925140+0000","last_undegraded":"2024-09-13T18:52:18.925140+0000","last_fullsized":"2024-09-13T18:52:18.925140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:55.776670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924931+0000","last_change":"2024-09-13T18:52:17.931585+0000","last_active":"2024-09-13T18:52:18.924931+0000","last_peered":"2024-09-13T18:52:18.924931+0000","last_clean":"2024-09-13T18:52:18.924931+0000","last_became_active":"2024-09-13T18:52:17.931280+0000","last_became_peered":"2024-09-13T18:52:17.931280+0000","last_unstale":"2024-09-13T18:52:18.924931+0000","last_undegraded":"2024-09-13T18:52:18.924931+0000","last_fullsized":"2024-09-13T18:52:18.924931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:20:42.480429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196202+0000","last_change":"2024-09-13T18:52:17.542275+0000","last_active":"2024-09-13T18:52:20.196202+0000","last_peered":"2024-09-13T18:52:20.196202+0000","last_clean":"2024-09-13T18:52:20.196202+0000","last_became_active":"2024-09-13T18:52:17.541964+0000","last_became_peered":"2024-09-13T18:52:17.541964+0000","last_unstale":"2024-09-13T18:52:20.196202+0000","last_undegraded":"2024-09-13T18:52:20.196202+0000","last_fullsized":"2024-09-13T18:52:20.196202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:55.776670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":823296,"data_stored":669740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":80354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":78880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":78916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":237568,"data_stored":80354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.373}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":225280,"data_stored":78790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":233472,"data_stored":78950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":831488,"data_stored":669796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:52:26.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:52:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.221+0000 7f599bfff700 1 Processor -- start 2024-09-13T18:52:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.221+0000 7f599bfff700 1 -- start start 2024-09-13T18:52:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.221+0000 7f599bfff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f599c0b3ae0 0x7f599c06a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.221+0000 7f599bfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f599c13dcb0 con 0x7f599c06bbf0 2024-09-13T18:52:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.221+0000 7f599bfff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f599c15be60 con 0x7f599c06aaf0 2024-09-13T18:52:26.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.221+0000 7f599bfff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f599c13ffd0 con 0x7f599c0b3ae0 2024-09-13T18:52:26.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.221+0000 7f599affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f599c06bbf0 0x7f599c0b33d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35890/0 (socket says 172.21.15.7:35890) 2024-09-13T18:52:26.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.221+0000 7f599affd700 1 -- 172.21.15.7:0/2266957075 learned_addr learned my addr 172.21.15.7:0/2266957075 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:26.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.221+0000 7f5993fff700 1 --2- 172.21.15.7:0/2266957075 >> v2:172.21.15.171:3300/0 conn(0x7f599c0b3ae0 0x7f599c06a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.221+0000 7f599a7fc700 1 -- 172.21.15.7:0/2266957075 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486954499 0 0) 0x7f599c13dcb0 con 0x7f599c06bbf0 2024-09-13T18:52:26.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.221+0000 7f599a7fc700 1 -- 172.21.15.7:0/2266957075 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59880035d0 con 0x7f599c06bbf0 2024-09-13T18:52:26.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.221+0000 7f5993fff700 1 -- 172.21.15.7:0/2266957075 >> v1:172.21.15.171:6789/0 conn(0x7f599c06aaf0 legacy=0x7f599c15e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.221+0000 7f599a7fc700 1 -- 172.21.15.7:0/2266957075 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 418531881 0 0) 0x7f59880035d0 con 0x7f599c06bbf0 2024-09-13T18:52:26.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.221+0000 7f5993fff700 1 -- 172.21.15.7:0/2266957075 >> v1:172.21.15.7:6789/0 conn(0x7f599c06bbf0 legacy=0x7f599c0b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.221+0000 7f5993fff700 1 -- 172.21.15.7:0/2266957075 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599c140220 con 0x7f599c0b3ae0 2024-09-13T18:52:26.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f5993fff700 1 --2- 172.21.15.7:0/2266957075 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f599c0b3ae0 0x7f599c06a5b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5984009850 tx=0x7f5984009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa55f6f4da3c94bd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:26.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599a7fc700 1 -- 172.21.15.7:0/2266957075 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c001670 con 0x7f599c0b3ae0 2024-09-13T18:52:26.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599a7fc700 1 -- 172.21.15.7:0/2266957075 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5984004030 con 0x7f599c0b3ae0 2024-09-13T18:52:26.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599a7fc700 1 -- 172.21.15.7:0/2266957075 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59840043c0 con 0x7f599c0b3ae0 2024-09-13T18:52:26.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599bfff700 1 -- 172.21.15.7:0/2266957075 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f599c0b3ae0 msgr2=0x7f599c06a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599bfff700 1 --2- 172.21.15.7:0/2266957075 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f599c0b3ae0 0x7f599c06a5b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5984009850 tx=0x7f5984009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:26.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599bfff700 1 -- 172.21.15.7:0/2266957075 shutdown_connections 2024-09-13T18:52:26.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599bfff700 1 --2- 172.21.15.7:0/2266957075 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f599c0b3ae0 0x7f599c06a5b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599bfff700 1 -- 172.21.15.7:0/2266957075 >> 172.21.15.7:0/2266957075 conn(0x7f599c0b0f00 msgr2=0x7f599c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:26.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599bfff700 1 -- 172.21.15.7:0/2266957075 shutdown_connections 2024-09-13T18:52:26.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599bfff700 1 -- 172.21.15.7:0/2266957075 wait complete. 2024-09-13T18:52:26.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599bfff700 1 Processor -- start 2024-09-13T18:52:26.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599bfff700 1 -- start start 2024-09-13T18:52:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f599c06aaf0 0x7f599c0ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599c06bbf0 0x7f599c1f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599bfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f599c0b3ae0 0x7f599c153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599bfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c1f3580 con 0x7f599c06bbf0 2024-09-13T18:52:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599bfff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c1f3860 con 0x7f599c06aaf0 2024-09-13T18:52:26.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599bfff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599c1f3b40 con 0x7f599c0b3ae0 2024-09-13T18:52:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f5993fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599c06bbf0 0x7f599c1f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f5993fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599c06bbf0 0x7f599c1f2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49630/0 (socket says 172.21.15.7:49630) 2024-09-13T18:52:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f5993fff700 1 -- 172.21.15.7:0/1077246275 learned_addr learned my addr 172.21.15.7:0/1077246275 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599affd700 1 --2- 172.21.15.7:0/1077246275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f599c06aaf0 0x7f599c0ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599b7fe700 1 --2- 172.21.15.7:0/1077246275 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f599c0b3ae0 0x7f599c153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f5993fff700 1 -- 172.21.15.7:0/1077246275 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f599c0b3ae0 msgr2=0x7f599c153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f5993fff700 1 --2- 172.21.15.7:0/1077246275 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f599c0b3ae0 0x7f599c153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f5993fff700 1 -- 172.21.15.7:0/1077246275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f599c06aaf0 msgr2=0x7f599c0ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f5993fff700 1 --2- 172.21.15.7:0/1077246275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f599c06aaf0 0x7f599c0ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f5993fff700 1 -- 172.21.15.7:0/1077246275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59840094b0 con 0x7f599c06bbf0 2024-09-13T18:52:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f599b7fe700 1 --2- 172.21.15.7:0/1077246275 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f599c0b3ae0 0x7f599c153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f5993fff700 1 --2- 172.21.15.7:0/1077246275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599c06bbf0 0x7f599c1f2b70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f599c06b360 tx=0x7f5984004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:26.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.225+0000 7f59937fe700 1 -- 172.21.15.7:0/1077246275 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984003e10 con 0x7f599c06bbf0 2024-09-13T18:52:26.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.229+0000 7f59937fe700 1 -- 172.21.15.7:0/1077246275 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598401f070 con 0x7f599c06bbf0 2024-09-13T18:52:26.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.229+0000 7f599bfff700 1 -- 172.21.15.7:0/1077246275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599c1f3e20 con 0x7f599c06bbf0 2024-09-13T18:52:26.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.229+0000 7f59937fe700 1 -- 172.21.15.7:0/1077246275 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984011870 con 0x7f599c06bbf0 2024-09-13T18:52:26.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.229+0000 7f599bfff700 1 -- 172.21.15.7:0/1077246275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599c0ade20 con 0x7f599c06bbf0 2024-09-13T18:52:26.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.229+0000 7f599bfff700 1 -- 172.21.15.7:0/1077246275 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f599c0adfb0 con 0x7f599c06bbf0 2024-09-13T18:52:26.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.229+0000 7f59937fe700 1 -- 172.21.15.7:0/1077246275 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5984021030 con 0x7f599c06bbf0 2024-09-13T18:52:26.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.229+0000 7f59937fe700 1 --2- 172.21.15.7:0/1077246275 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f597c066730 0x7f597c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.229+0000 7f59937fe700 1 -- 172.21.15.7:0/1077246275 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59840855c0 con 0x7f599c06bbf0 2024-09-13T18:52:26.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.229+0000 7f599affd700 1 --2- 172.21.15.7:0/1077246275 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f597c066730 0x7f597c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.233+0000 7f599affd700 1 --2- 172.21.15.7:0/1077246275 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f597c066730 0x7f597c068bf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f598c001eb0 tx=0x7f598c001c60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:26.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.237+0000 7f59937fe700 1 -- 172.21.15.7:0/1077246275 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5984052930 con 0x7f599c06bbf0 2024-09-13T18:52:26.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.401+0000 7f599bfff700 1 -- 172.21.15.7:0/1077246275 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f599c140ac0 con 0x7f597c066730 2024-09-13T18:52:26.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.417+0000 7f59937fe700 1 -- 172.21.15.7:0/1077246275 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264496 (secure 0 0 0) 0x7f598c001670 con 0x7f597c066730 2024-09-13T18:52:26.426 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:26.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.421+0000 7f599bfff700 1 -- 172.21.15.7:0/1077246275 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f597c066730 msgr2=0x7f597c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.421+0000 7f599bfff700 1 --2- 172.21.15.7:0/1077246275 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f597c066730 0x7f597c068bf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f598c001eb0 tx=0x7f598c001c60 comp rx=0 tx=0).stop 2024-09-13T18:52:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.421+0000 7f599bfff700 1 -- 172.21.15.7:0/1077246275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599c06bbf0 msgr2=0x7f599c1f2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.421+0000 7f599bfff700 1 --2- 172.21.15.7:0/1077246275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599c06bbf0 0x7f599c1f2b70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f599c06b360 tx=0x7f5984004770 comp rx=0 tx=0).stop 2024-09-13T18:52:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.421+0000 7f599bfff700 1 -- 172.21.15.7:0/1077246275 shutdown_connections 2024-09-13T18:52:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.421+0000 7f599bfff700 1 --2- 172.21.15.7:0/1077246275 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f597c066730 0x7f597c068bf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.421+0000 7f599bfff700 1 --2- 172.21.15.7:0/1077246275 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f599c0b3ae0 0x7f599c153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.421+0000 7f599bfff700 1 --2- 172.21.15.7:0/1077246275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f599c06bbf0 0x7f599c1f2b70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.421+0000 7f599bfff700 1 --2- 172.21.15.7:0/1077246275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f599c06aaf0 0x7f599c0ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.421+0000 7f599bfff700 1 -- 172.21.15.7:0/1077246275 >> 172.21.15.7:0/1077246275 conn(0x7f599c0b0f00 msgr2=0x7f599c0afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.421+0000 7f599bfff700 1 -- 172.21.15.7:0/1077246275 shutdown_connections 2024-09-13T18:52:26.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.421+0000 7f599bfff700 1 -- 172.21.15.7:0/1077246275 wait complete. 2024-09-13T18:52:26.430 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-13T18:52:26.441 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-13T18:52:26.213465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235740,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738980,"statfs":{"total":773094113280,"available":772852715520,"internally_reserved":0,"allocated":3055616,"data_stored":1815780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.296520"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200870+0000","last_change":"2024-09-13T18:52:17.941085+0000","last_active":"2024-09-13T18:52:20.200870+0000","last_peered":"2024-09-13T18:52:20.200870+0000","last_clean":"2024-09-13T18:52:20.200870+0000","last_became_active":"2024-09-13T18:52:17.940741+0000","last_became_peered":"2024-09-13T18:52:17.940741+0000","last_unstale":"2024-09-13T18:52:20.200870+0000","last_undegraded":"2024-09-13T18:52:20.200870+0000","last_fullsized":"2024-09-13T18:52:20.200870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:50.609433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195285+0000","last_change":"2024-09-13T18:52:17.942350+0000","last_active":"2024-09-13T18:52:20.195285+0000","last_peered":"2024-09-13T18:52:20.195285+0000","last_clean":"2024-09-13T18:52:20.195285+0000","last_became_active":"2024-09-13T18:52:17.942062+0000","last_became_peered":"2024-09-13T18:52:17.942062+0000","last_unstale":"2024-09-13T18:52:20.195285+0000","last_undegraded":"2024-09-13T18:52:20.195285+0000","last_fullsized":"2024-09-13T18:52:20.195285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:45.768218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195540+0000","last_change":"2024-09-13T18:52:17.953055+0000","last_active":"2024-09-13T18:52:20.195540+0000","last_peered":"2024-09-13T18:52:20.195540+0000","last_clean":"2024-09-13T18:52:20.195540+0000","last_became_active":"2024-09-13T18:52:17.952806+0000","last_became_peered":"2024-09-13T18:52:17.952806+0000","last_unstale":"2024-09-13T18:52:20.195540+0000","last_undegraded":"2024-09-13T18:52:20.195540+0000","last_fullsized":"2024-09-13T18:52:20.195540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:18.149491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198959+0000","last_change":"2024-09-13T18:52:17.943472+0000","last_active":"2024-09-13T18:52:20.198959+0000","last_peered":"2024-09-13T18:52:20.198959+0000","last_clean":"2024-09-13T18:52:20.198959+0000","last_became_active":"2024-09-13T18:52:17.943070+0000","last_became_peered":"2024-09-13T18:52:17.943070+0000","last_unstale":"2024-09-13T18:52:20.198959+0000","last_undegraded":"2024-09-13T18:52:20.198959+0000","last_fullsized":"2024-09-13T18:52:20.198959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.990929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198942+0000","last_change":"2024-09-13T18:52:17.957081+0000","last_active":"2024-09-13T18:52:20.198942+0000","last_peered":"2024-09-13T18:52:20.198942+0000","last_clean":"2024-09-13T18:52:20.198942+0000","last_became_active":"2024-09-13T18:52:17.956787+0000","last_became_peered":"2024-09-13T18:52:17.956787+0000","last_unstale":"2024-09-13T18:52:20.198942+0000","last_undegraded":"2024-09-13T18:52:20.198942+0000","last_fullsized":"2024-09-13T18:52:20.198942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:31.026983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198843+0000","last_change":"2024-09-13T18:52:17.944754+0000","last_active":"2024-09-13T18:52:20.198843+0000","last_peered":"2024-09-13T18:52:20.198843+0000","last_clean":"2024-09-13T18:52:20.198843+0000","last_became_active":"2024-09-13T18:52:17.944495+0000","last_became_peered":"2024-09-13T18:52:17.944495+0000","last_unstale":"2024-09-13T18:52:20.198843+0000","last_undegraded":"2024-09-13T18:52:20.198843+0000","last_fullsized":"2024-09-13T18:52:20.198843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:40:09.595666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198289+0000","last_change":"2024-09-13T18:52:17.945805+0000","last_active":"2024-09-13T18:52:20.198289+0000","last_peered":"2024-09-13T18:52:20.198289+0000","last_clean":"2024-09-13T18:52:20.198289+0000","last_became_active":"2024-09-13T18:52:17.945364+0000","last_became_peered":"2024-09-13T18:52:17.945364+0000","last_unstale":"2024-09-13T18:52:20.198289+0000","last_undegraded":"2024-09-13T18:52:20.198289+0000","last_fullsized":"2024-09-13T18:52:20.198289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:43.043485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.923839+0000","last_change":"2024-09-13T18:52:17.939184+0000","last_active":"2024-09-13T18:52:18.923839+0000","last_peered":"2024-09-13T18:52:18.923839+0000","last_clean":"2024-09-13T18:52:18.923839+0000","last_became_active":"2024-09-13T18:52:17.938855+0000","last_became_peered":"2024-09-13T18:52:17.938855+0000","last_unstale":"2024-09-13T18:52:18.923839+0000","last_undegraded":"2024-09-13T18:52:18.923839+0000","last_fullsized":"2024-09-13T18:52:18.923839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:45.768218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200942+0000","last_change":"2024-09-13T18:52:17.940244+0000","last_active":"2024-09-13T18:52:20.200942+0000","last_peered":"2024-09-13T18:52:20.200942+0000","last_clean":"2024-09-13T18:52:20.200942+0000","last_became_active":"2024-09-13T18:52:17.939977+0000","last_became_peered":"2024-09-13T18:52:17.939977+0000","last_unstale":"2024-09-13T18:52:20.200942+0000","last_undegraded":"2024-09-13T18:52:20.200942+0000","last_fullsized":"2024-09-13T18:52:20.200942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:21.405599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.928272+0000","last_change":"2024-09-13T18:52:17.940027+0000","last_active":"2024-09-13T18:52:18.928272+0000","last_peered":"2024-09-13T18:52:18.928272+0000","last_clean":"2024-09-13T18:52:18.928272+0000","last_became_active":"2024-09-13T18:52:17.939743+0000","last_became_peered":"2024-09-13T18:52:17.939743+0000","last_unstale":"2024-09-13T18:52:18.928272+0000","last_undegraded":"2024-09-13T18:52:18.928272+0000","last_fullsized":"2024-09-13T18:52:18.928272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:26.228888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927956+0000","last_change":"2024-09-13T18:52:17.940006+0000","last_active":"2024-09-13T18:52:18.927956+0000","last_peered":"2024-09-13T18:52:18.927956+0000","last_clean":"2024-09-13T18:52:18.927956+0000","last_became_active":"2024-09-13T18:52:17.939474+0000","last_became_peered":"2024-09-13T18:52:17.939474+0000","last_unstale":"2024-09-13T18:52:18.927956+0000","last_undegraded":"2024-09-13T18:52:18.927956+0000","last_fullsized":"2024-09-13T18:52:18.927956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:24.025045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198153+0000","last_change":"2024-09-13T18:52:17.942513+0000","last_active":"2024-09-13T18:52:20.198153+0000","last_peered":"2024-09-13T18:52:20.198153+0000","last_clean":"2024-09-13T18:52:20.198153+0000","last_became_active":"2024-09-13T18:52:17.940486+0000","last_became_peered":"2024-09-13T18:52:17.940486+0000","last_unstale":"2024-09-13T18:52:20.198153+0000","last_undegraded":"2024-09-13T18:52:20.198153+0000","last_fullsized":"2024-09-13T18:52:20.198153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:08.839071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924421+0000","last_change":"2024-09-13T18:52:17.935439+0000","last_active":"2024-09-13T18:52:18.924421+0000","last_peered":"2024-09-13T18:52:18.924421+0000","last_clean":"2024-09-13T18:52:18.924421+0000","last_became_active":"2024-09-13T18:52:17.935114+0000","last_became_peered":"2024-09-13T18:52:17.935114+0000","last_unstale":"2024-09-13T18:52:18.924421+0000","last_undegraded":"2024-09-13T18:52:18.924421+0000","last_fullsized":"2024-09-13T18:52:18.924421+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:05:38.704891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195436+0000","last_change":"2024-09-13T18:52:17.942475+0000","last_active":"2024-09-13T18:52:20.195436+0000","last_peered":"2024-09-13T18:52:20.195436+0000","last_clean":"2024-09-13T18:52:20.195436+0000","last_became_active":"2024-09-13T18:52:17.942233+0000","last_became_peered":"2024-09-13T18:52:17.942233+0000","last_unstale":"2024-09-13T18:52:20.195436+0000","last_undegraded":"2024-09-13T18:52:20.195436+0000","last_fullsized":"2024-09-13T18:52:20.195436+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:24.025045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.194709+0000","last_change":"2024-09-13T18:52:17.926146+0000","last_active":"2024-09-13T18:52:20.194709+0000","last_peered":"2024-09-13T18:52:20.194709+0000","last_clean":"2024-09-13T18:52:20.194709+0000","last_became_active":"2024-09-13T18:52:17.925837+0000","last_became_peered":"2024-09-13T18:52:17.925837+0000","last_unstale":"2024-09-13T18:52:20.194709+0000","last_undegraded":"2024-09-13T18:52:20.194709+0000","last_fullsized":"2024-09-13T18:52:20.194709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:30.871944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196715+0000","last_change":"2024-09-13T18:52:17.945764+0000","last_active":"2024-09-13T18:52:20.196715+0000","last_peered":"2024-09-13T18:52:20.196715+0000","last_clean":"2024-09-13T18:52:20.196715+0000","last_became_active":"2024-09-13T18:52:17.944484+0000","last_became_peered":"2024-09-13T18:52:17.944484+0000","last_unstale":"2024-09-13T18:52:20.196715+0000","last_undegraded":"2024-09-13T18:52:20.196715+0000","last_fullsized":"2024-09-13T18:52:20.196715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:45.768218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199740+0000","last_change":"2024-09-13T18:52:17.943317+0000","last_active":"2024-09-13T18:52:20.199740+0000","last_peered":"2024-09-13T18:52:20.199740+0000","last_clean":"2024-09-13T18:52:20.199740+0000","last_became_active":"2024-09-13T18:52:17.943015+0000","last_became_peered":"2024-09-13T18:52:17.943015+0000","last_unstale":"2024-09-13T18:52:20.199740+0000","last_undegraded":"2024-09-13T18:52:20.199740+0000","last_fullsized":"2024-09-13T18:52:20.199740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:50.609433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925962+0000","last_change":"2024-09-13T18:52:17.956474+0000","last_active":"2024-09-13T18:52:18.925962+0000","last_peered":"2024-09-13T18:52:18.925962+0000","last_clean":"2024-09-13T18:52:18.925962+0000","last_became_active":"2024-09-13T18:52:17.956200+0000","last_became_peered":"2024-09-13T18:52:17.956200+0000","last_unstale":"2024-09-13T18:52:18.925962+0000","last_undegraded":"2024-09-13T18:52:18.925962+0000","last_fullsized":"2024-09-13T18:52:18.925962+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:17:56.476262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200865+0000","last_change":"2024-09-13T18:52:17.955849+0000","last_active":"2024-09-13T18:52:20.200865+0000","last_peered":"2024-09-13T18:52:20.200865+0000","last_clean":"2024-09-13T18:52:20.200865+0000","last_became_active":"2024-09-13T18:52:17.955606+0000","last_became_peered":"2024-09-13T18:52:17.955606+0000","last_unstale":"2024-09-13T18:52:20.200865+0000","last_undegraded":"2024-09-13T18:52:20.200865+0000","last_fullsized":"2024-09-13T18:52:20.200865+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:45.768218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927684+0000","last_change":"2024-09-13T18:52:17.947271+0000","last_active":"2024-09-13T18:52:18.927684+0000","last_peered":"2024-09-13T18:52:18.927684+0000","last_clean":"2024-09-13T18:52:18.927684+0000","last_became_active":"2024-09-13T18:52:17.947018+0000","last_became_peered":"2024-09-13T18:52:17.947018+0000","last_unstale":"2024-09-13T18:52:18.927684+0000","last_undegraded":"2024-09-13T18:52:18.927684+0000","last_fullsized":"2024-09-13T18:52:18.927684+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:45.768218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195004+0000","last_change":"2024-09-13T18:52:17.949499+0000","last_active":"2024-09-13T18:52:20.195004+0000","last_peered":"2024-09-13T18:52:20.195004+0000","last_clean":"2024-09-13T18:52:20.195004+0000","last_became_active":"2024-09-13T18:52:17.949233+0000","last_became_peered":"2024-09-13T18:52:17.949233+0000","last_unstale":"2024-09-13T18:52:20.195004+0000","last_undegraded":"2024-09-13T18:52:20.195004+0000","last_fullsized":"2024-09-13T18:52:20.195004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:05:38.704891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197193+0000","last_change":"2024-09-13T18:52:17.941847+0000","last_active":"2024-09-13T18:52:20.197193+0000","last_peered":"2024-09-13T18:52:20.197193+0000","last_clean":"2024-09-13T18:52:20.197193+0000","last_became_active":"2024-09-13T18:52:17.941228+0000","last_became_peered":"2024-09-13T18:52:17.941228+0000","last_unstale":"2024-09-13T18:52:20.197193+0000","last_undegraded":"2024-09-13T18:52:20.197193+0000","last_fullsized":"2024-09-13T18:52:20.197193+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:41.161193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924015+0000","last_change":"2024-09-13T18:52:17.924625+0000","last_active":"2024-09-13T18:52:18.924015+0000","last_peered":"2024-09-13T18:52:18.924015+0000","last_clean":"2024-09-13T18:52:18.924015+0000","last_became_active":"2024-09-13T18:52:17.923776+0000","last_became_peered":"2024-09-13T18:52:17.923776+0000","last_unstale":"2024-09-13T18:52:18.924015+0000","last_undegraded":"2024-09-13T18:52:18.924015+0000","last_fullsized":"2024-09-13T18:52:18.924015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:36.215528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196911+0000","last_change":"2024-09-13T18:52:17.938687+0000","last_active":"2024-09-13T18:52:20.196911+0000","last_peered":"2024-09-13T18:52:20.196911+0000","last_clean":"2024-09-13T18:52:20.196911+0000","last_became_active":"2024-09-13T18:52:17.938426+0000","last_became_peered":"2024-09-13T18:52:17.938426+0000","last_unstale":"2024-09-13T18:52:20.196911+0000","last_undegraded":"2024-09-13T18:52:20.196911+0000","last_fullsized":"2024-09-13T18:52:20.196911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:53.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924463+0000","last_change":"2024-09-13T18:52:17.924256+0000","last_active":"2024-09-13T18:52:18.924463+0000","last_peered":"2024-09-13T18:52:18.924463+0000","last_clean":"2024-09-13T18:52:18.924463+0000","last_became_active":"2024-09-13T18:52:17.923756+0000","last_became_peered":"2024-09-13T18:52:17.923756+0000","last_unstale":"2024-09-13T18:52:18.924463+0000","last_undegraded":"2024-09-13T18:52:18.924463+0000","last_fullsized":"2024-09-13T18:52:18.924463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:26.228888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.928375+0000","last_change":"2024-09-13T18:52:17.931734+0000","last_active":"2024-09-13T18:52:18.928375+0000","last_peered":"2024-09-13T18:52:18.928375+0000","last_clean":"2024-09-13T18:52:18.928375+0000","last_became_active":"2024-09-13T18:52:17.931034+0000","last_became_peered":"2024-09-13T18:52:17.931034+0000","last_unstale":"2024-09-13T18:52:18.928375+0000","last_undegraded":"2024-09-13T18:52:18.928375+0000","last_fullsized":"2024-09-13T18:52:18.928375+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:31.026983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.194568+0000","last_change":"2024-09-13T18:52:17.927130+0000","last_active":"2024-09-13T18:52:20.194568+0000","last_peered":"2024-09-13T18:52:20.194568+0000","last_clean":"2024-09-13T18:52:20.194568+0000","last_became_active":"2024-09-13T18:52:17.926099+0000","last_became_peered":"2024-09-13T18:52:17.926099+0000","last_unstale":"2024-09-13T18:52:20.194568+0000","last_undegraded":"2024-09-13T18:52:20.194568+0000","last_fullsized":"2024-09-13T18:52:20.194568+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:05:38.704891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924857+0000","last_change":"2024-09-13T18:52:17.924220+0000","last_active":"2024-09-13T18:52:18.924857+0000","last_peered":"2024-09-13T18:52:18.924857+0000","last_clean":"2024-09-13T18:52:18.924857+0000","last_became_active":"2024-09-13T18:52:17.923830+0000","last_became_peered":"2024-09-13T18:52:17.923830+0000","last_unstale":"2024-09-13T18:52:18.924857+0000","last_undegraded":"2024-09-13T18:52:18.924857+0000","last_fullsized":"2024-09-13T18:52:18.924857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:24.025045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195245+0000","last_change":"2024-09-13T18:52:17.942480+0000","last_active":"2024-09-13T18:52:20.195245+0000","last_peered":"2024-09-13T18:52:20.195245+0000","last_clean":"2024-09-13T18:52:20.195245+0000","last_became_active":"2024-09-13T18:52:17.942156+0000","last_became_peered":"2024-09-13T18:52:17.942156+0000","last_unstale":"2024-09-13T18:52:20.195245+0000","last_undegraded":"2024-09-13T18:52:20.195245+0000","last_fullsized":"2024-09-13T18:52:20.195245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:30.871944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197425+0000","last_change":"2024-09-13T18:52:17.943608+0000","last_active":"2024-09-13T18:52:20.197425+0000","last_peered":"2024-09-13T18:52:20.197425+0000","last_clean":"2024-09-13T18:52:20.197425+0000","last_became_active":"2024-09-13T18:52:17.943314+0000","last_became_peered":"2024-09-13T18:52:17.943314+0000","last_unstale":"2024-09-13T18:52:20.197425+0000","last_undegraded":"2024-09-13T18:52:20.197425+0000","last_fullsized":"2024-09-13T18:52:20.197425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:22:43.043485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925985+0000","last_change":"2024-09-13T18:52:17.956342+0000","last_active":"2024-09-13T18:52:18.925985+0000","last_peered":"2024-09-13T18:52:18.925985+0000","last_clean":"2024-09-13T18:52:18.925985+0000","last_became_active":"2024-09-13T18:52:17.956066+0000","last_became_peered":"2024-09-13T18:52:17.956066+0000","last_unstale":"2024-09-13T18:52:18.925985+0000","last_undegraded":"2024-09-13T18:52:18.925985+0000","last_fullsized":"2024-09-13T18:52:18.925985+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:24:03.326121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927056+0000","last_change":"2024-09-13T18:52:17.944931+0000","last_active":"2024-09-13T18:52:18.927056+0000","last_peered":"2024-09-13T18:52:18.927056+0000","last_clean":"2024-09-13T18:52:18.927056+0000","last_became_active":"2024-09-13T18:52:17.944522+0000","last_became_peered":"2024-09-13T18:52:17.944522+0000","last_unstale":"2024-09-13T18:52:18.927056+0000","last_undegraded":"2024-09-13T18:52:18.927056+0000","last_fullsized":"2024-09-13T18:52:18.927056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:36.215528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197266+0000","last_change":"2024-09-13T18:52:17.924921+0000","last_active":"2024-09-13T18:52:20.197266+0000","last_peered":"2024-09-13T18:52:20.197266+0000","last_clean":"2024-09-13T18:52:20.197266+0000","last_became_active":"2024-09-13T18:52:17.924708+0000","last_became_peered":"2024-09-13T18:52:17.924708+0000","last_unstale":"2024-09-13T18:52:20.197266+0000","last_undegraded":"2024-09-13T18:52:20.197266+0000","last_fullsized":"2024-09-13T18:52:20.197266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:25.786229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925450+0000","last_change":"2024-09-13T18:52:17.538391+0000","last_active":"2024-09-13T18:52:18.925450+0000","last_peered":"2024-09-13T18:52:18.925450+0000","last_clean":"2024-09-13T18:52:18.925450+0000","last_became_active":"2024-09-13T18:52:17.538114+0000","last_became_peered":"2024-09-13T18:52:17.538114+0000","last_unstale":"2024-09-13T18:52:18.925450+0000","last_undegraded":"2024-09-13T18:52:18.925450+0000","last_fullsized":"2024-09-13T18:52:18.925450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:04.695688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195477+0000","last_change":"2024-09-13T18:52:17.946924+0000","last_active":"2024-09-13T18:52:20.195477+0000","last_peered":"2024-09-13T18:52:20.195477+0000","last_clean":"2024-09-13T18:52:20.195477+0000","last_became_active":"2024-09-13T18:52:17.946599+0000","last_became_peered":"2024-09-13T18:52:17.946599+0000","last_unstale":"2024-09-13T18:52:20.195477+0000","last_undegraded":"2024-09-13T18:52:20.195477+0000","last_fullsized":"2024-09-13T18:52:20.195477+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:26.228888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927670+0000","last_change":"2024-09-13T18:52:17.535479+0000","last_active":"2024-09-13T18:52:18.927670+0000","last_peered":"2024-09-13T18:52:18.927670+0000","last_clean":"2024-09-13T18:52:18.927670+0000","last_became_active":"2024-09-13T18:52:17.535185+0000","last_became_peered":"2024-09-13T18:52:17.535185+0000","last_unstale":"2024-09-13T18:52:18.927670+0000","last_undegraded":"2024-09-13T18:52:18.927670+0000","last_fullsized":"2024-09-13T18:52:18.927670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:47.942952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924218+0000","last_change":"2024-09-13T18:52:17.531093+0000","last_active":"2024-09-13T18:52:18.924218+0000","last_peered":"2024-09-13T18:52:18.924218+0000","last_clean":"2024-09-13T18:52:18.924218+0000","last_became_active":"2024-09-13T18:52:17.530499+0000","last_became_peered":"2024-09-13T18:52:17.530499+0000","last_unstale":"2024-09-13T18:52:18.924218+0000","last_undegraded":"2024-09-13T18:52:18.924218+0000","last_fullsized":"2024-09-13T18:52:18.924218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:33:25.953860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.923892+0000","last_change":"2024-09-13T18:52:17.940517+0000","last_active":"2024-09-13T18:52:18.923892+0000","last_peered":"2024-09-13T18:52:18.923892+0000","last_clean":"2024-09-13T18:52:18.923892+0000","last_became_active":"2024-09-13T18:52:17.940250+0000","last_became_peered":"2024-09-13T18:52:17.940250+0000","last_unstale":"2024-09-13T18:52:18.923892+0000","last_undegraded":"2024-09-13T18:52:18.923892+0000","last_fullsized":"2024-09-13T18:52:18.923892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.990929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196937+0000","last_change":"2024-09-13T18:52:17.539508+0000","last_active":"2024-09-13T18:52:20.196937+0000","last_peered":"2024-09-13T18:52:20.196937+0000","last_clean":"2024-09-13T18:52:20.196937+0000","last_became_active":"2024-09-13T18:52:17.539182+0000","last_became_peered":"2024-09-13T18:52:17.539182+0000","last_unstale":"2024-09-13T18:52:20.196937+0000","last_undegraded":"2024-09-13T18:52:20.196937+0000","last_fullsized":"2024-09-13T18:52:20.196937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:49.151192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.194851+0000","last_change":"2024-09-13T18:52:17.926798+0000","last_active":"2024-09-13T18:52:20.194851+0000","last_peered":"2024-09-13T18:52:20.194851+0000","last_clean":"2024-09-13T18:52:20.194851+0000","last_became_active":"2024-09-13T18:52:17.926182+0000","last_became_peered":"2024-09-13T18:52:17.926182+0000","last_unstale":"2024-09-13T18:52:20.194851+0000","last_undegraded":"2024-09-13T18:52:20.194851+0000","last_fullsized":"2024-09-13T18:52:20.194851+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:36.215528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.928162+0000","last_change":"2024-09-13T18:52:17.539633+0000","last_active":"2024-09-13T18:52:18.928162+0000","last_peered":"2024-09-13T18:52:18.928162+0000","last_clean":"2024-09-13T18:52:18.928162+0000","last_became_active":"2024-09-13T18:52:17.539340+0000","last_became_peered":"2024-09-13T18:52:17.539340+0000","last_unstale":"2024-09-13T18:52:18.928162+0000","last_undegraded":"2024-09-13T18:52:18.928162+0000","last_fullsized":"2024-09-13T18:52:18.928162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:06.694707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199408+0000","last_change":"2024-09-13T18:52:17.941631+0000","last_active":"2024-09-13T18:52:20.199408+0000","last_peered":"2024-09-13T18:52:20.199408+0000","last_clean":"2024-09-13T18:52:20.199408+0000","last_became_active":"2024-09-13T18:52:17.940875+0000","last_became_peered":"2024-09-13T18:52:17.940875+0000","last_unstale":"2024-09-13T18:52:20.199408+0000","last_undegraded":"2024-09-13T18:52:20.199408+0000","last_fullsized":"2024-09-13T18:52:20.199408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:42:22.626590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199507+0000","last_change":"2024-09-13T18:52:17.532300+0000","last_active":"2024-09-13T18:52:20.199507+0000","last_peered":"2024-09-13T18:52:20.199507+0000","last_clean":"2024-09-13T18:52:20.199507+0000","last_became_active":"2024-09-13T18:52:17.532019+0000","last_became_peered":"2024-09-13T18:52:17.532019+0000","last_unstale":"2024-09-13T18:52:20.199507+0000","last_undegraded":"2024-09-13T18:52:20.199507+0000","last_fullsized":"2024-09-13T18:52:20.199507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:05:28.094741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196062+0000","last_change":"2024-09-13T18:52:17.945629+0000","last_active":"2024-09-13T18:52:20.196062+0000","last_peered":"2024-09-13T18:52:20.196062+0000","last_clean":"2024-09-13T18:52:20.196062+0000","last_became_active":"2024-09-13T18:52:17.945372+0000","last_became_peered":"2024-09-13T18:52:17.945372+0000","last_unstale":"2024-09-13T18:52:20.196062+0000","last_undegraded":"2024-09-13T18:52:20.196062+0000","last_fullsized":"2024-09-13T18:52:20.196062+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:36.215528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.194942+0000","last_change":"2024-09-13T18:52:17.538613+0000","last_active":"2024-09-13T18:52:20.194942+0000","last_peered":"2024-09-13T18:52:20.194942+0000","last_clean":"2024-09-13T18:52:20.194942+0000","last_became_active":"2024-09-13T18:52:17.538372+0000","last_became_peered":"2024-09-13T18:52:17.538372+0000","last_unstale":"2024-09-13T18:52:20.194942+0000","last_undegraded":"2024-09-13T18:52:20.194942+0000","last_fullsized":"2024-09-13T18:52:20.194942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:05:28.094741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198119+0000","last_change":"2024-09-13T18:52:17.941036+0000","last_active":"2024-09-13T18:52:20.198119+0000","last_peered":"2024-09-13T18:52:20.198119+0000","last_clean":"2024-09-13T18:52:20.198119+0000","last_became_active":"2024-09-13T18:52:17.940741+0000","last_became_peered":"2024-09-13T18:52:17.940741+0000","last_unstale":"2024-09-13T18:52:20.198119+0000","last_undegraded":"2024-09-13T18:52:20.198119+0000","last_fullsized":"2024-09-13T18:52:20.198119+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:19:17.191755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927753+0000","last_change":"2024-09-13T18:52:17.536588+0000","last_active":"2024-09-13T18:52:18.927753+0000","last_peered":"2024-09-13T18:52:18.927753+0000","last_clean":"2024-09-13T18:52:18.927753+0000","last_became_active":"2024-09-13T18:52:17.536319+0000","last_became_peered":"2024-09-13T18:52:17.536319+0000","last_unstale":"2024-09-13T18:52:18.927753+0000","last_undegraded":"2024-09-13T18:52:18.927753+0000","last_fullsized":"2024-09-13T18:52:18.927753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:14:15.613948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199367+0000","last_change":"2024-09-13T18:52:17.935349+0000","last_active":"2024-09-13T18:52:20.199367+0000","last_peered":"2024-09-13T18:52:20.199367+0000","last_clean":"2024-09-13T18:52:20.199367+0000","last_became_active":"2024-09-13T18:52:17.935064+0000","last_became_peered":"2024-09-13T18:52:17.935064+0000","last_unstale":"2024-09-13T18:52:20.199367+0000","last_undegraded":"2024-09-13T18:52:20.199367+0000","last_fullsized":"2024-09-13T18:52:20.199367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:18.183823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195705+0000","last_change":"2024-09-13T18:52:17.538911+0000","last_active":"2024-09-13T18:52:20.195705+0000","last_peered":"2024-09-13T18:52:20.195705+0000","last_clean":"2024-09-13T18:52:20.195705+0000","last_became_active":"2024-09-13T18:52:17.538512+0000","last_became_peered":"2024-09-13T18:52:17.538512+0000","last_unstale":"2024-09-13T18:52:20.195705+0000","last_undegraded":"2024-09-13T18:52:20.195705+0000","last_fullsized":"2024-09-13T18:52:20.195705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:03:19.727396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195688+0000","last_change":"2024-09-13T18:52:17.949196+0000","last_active":"2024-09-13T18:52:20.195688+0000","last_peered":"2024-09-13T18:52:20.195688+0000","last_clean":"2024-09-13T18:52:20.195688+0000","last_became_active":"2024-09-13T18:52:17.948936+0000","last_became_peered":"2024-09-13T18:52:17.948936+0000","last_unstale":"2024-09-13T18:52:20.195688+0000","last_undegraded":"2024-09-13T18:52:20.195688+0000","last_fullsized":"2024-09-13T18:52:20.195688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:43:49.752608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197616+0000","last_change":"2024-09-13T18:52:17.534190+0000","last_active":"2024-09-13T18:52:20.197616+0000","last_peered":"2024-09-13T18:52:20.197616+0000","last_clean":"2024-09-13T18:52:20.197616+0000","last_became_active":"2024-09-13T18:52:17.532898+0000","last_became_peered":"2024-09-13T18:52:17.532898+0000","last_unstale":"2024-09-13T18:52:20.197616+0000","last_undegraded":"2024-09-13T18:52:20.197616+0000","last_fullsized":"2024-09-13T18:52:20.197616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:14:15.613948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200129+0000","last_change":"2024-09-13T18:52:17.947584+0000","last_active":"2024-09-13T18:52:20.200129+0000","last_peered":"2024-09-13T18:52:20.200129+0000","last_clean":"2024-09-13T18:52:20.200129+0000","last_became_active":"2024-09-13T18:52:17.947333+0000","last_became_peered":"2024-09-13T18:52:17.947333+0000","last_unstale":"2024-09-13T18:52:20.200129+0000","last_undegraded":"2024-09-13T18:52:20.200129+0000","last_fullsized":"2024-09-13T18:52:20.200129+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:53.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927484+0000","last_change":"2024-09-13T18:52:17.539689+0000","last_active":"2024-09-13T18:52:18.927484+0000","last_peered":"2024-09-13T18:52:18.927484+0000","last_clean":"2024-09-13T18:52:18.927484+0000","last_became_active":"2024-09-13T18:52:17.539435+0000","last_became_peered":"2024-09-13T18:52:17.539435+0000","last_unstale":"2024-09-13T18:52:18.927484+0000","last_undegraded":"2024-09-13T18:52:18.927484+0000","last_fullsized":"2024-09-13T18:52:18.927484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:22:26.864392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927431+0000","last_change":"2024-09-13T18:52:17.944767+0000","last_active":"2024-09-13T18:52:18.927431+0000","last_peered":"2024-09-13T18:52:18.927431+0000","last_clean":"2024-09-13T18:52:18.927431+0000","last_became_active":"2024-09-13T18:52:17.944163+0000","last_became_peered":"2024-09-13T18:52:17.944163+0000","last_unstale":"2024-09-13T18:52:18.927431+0000","last_undegraded":"2024-09-13T18:52:18.927431+0000","last_fullsized":"2024-09-13T18:52:18.927431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:30.871944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925915+0000","last_change":"2024-09-13T18:52:17.535230+0000","last_active":"2024-09-13T18:52:18.925915+0000","last_peered":"2024-09-13T18:52:18.925915+0000","last_clean":"2024-09-13T18:52:18.925915+0000","last_became_active":"2024-09-13T18:52:17.534923+0000","last_became_peered":"2024-09-13T18:52:17.534923+0000","last_unstale":"2024-09-13T18:52:18.925915+0000","last_undegraded":"2024-09-13T18:52:18.925915+0000","last_fullsized":"2024-09-13T18:52:18.925915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:47:07.580322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198532+0000","last_change":"2024-09-13T18:52:17.949849+0000","last_active":"2024-09-13T18:52:20.198532+0000","last_peered":"2024-09-13T18:52:20.198532+0000","last_clean":"2024-09-13T18:52:20.198532+0000","last_became_active":"2024-09-13T18:52:17.949503+0000","last_became_peered":"2024-09-13T18:52:17.949503+0000","last_unstale":"2024-09-13T18:52:20.198532+0000","last_undegraded":"2024-09-13T18:52:20.198532+0000","last_fullsized":"2024-09-13T18:52:20.198532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:30.871944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.198430+0000","last_change":"2024-09-13T18:52:17.540059+0000","last_active":"2024-09-13T18:52:20.198430+0000","last_peered":"2024-09-13T18:52:20.198430+0000","last_clean":"2024-09-13T18:52:20.198430+0000","last_became_active":"2024-09-13T18:52:17.539768+0000","last_became_peered":"2024-09-13T18:52:17.539768+0000","last_unstale":"2024-09-13T18:52:20.198430+0000","last_undegraded":"2024-09-13T18:52:20.198430+0000","last_fullsized":"2024-09-13T18:52:20.198430+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:56:33.879872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200908+0000","last_change":"2024-09-13T18:52:17.947339+0000","last_active":"2024-09-13T18:52:20.200908+0000","last_peered":"2024-09-13T18:52:20.200908+0000","last_clean":"2024-09-13T18:52:20.200908+0000","last_became_active":"2024-09-13T18:52:17.947073+0000","last_became_peered":"2024-09-13T18:52:17.947073+0000","last_unstale":"2024-09-13T18:52:20.200908+0000","last_undegraded":"2024-09-13T18:52:20.200908+0000","last_fullsized":"2024-09-13T18:52:20.200908+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:36.215528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925944+0000","last_change":"2024-09-13T18:52:17.534989+0000","last_active":"2024-09-13T18:52:18.925944+0000","last_peered":"2024-09-13T18:52:18.925944+0000","last_clean":"2024-09-13T18:52:18.925944+0000","last_became_active":"2024-09-13T18:52:17.532634+0000","last_became_peered":"2024-09-13T18:52:17.532634+0000","last_unstale":"2024-09-13T18:52:18.925944+0000","last_undegraded":"2024-09-13T18:52:18.925944+0000","last_fullsized":"2024-09-13T18:52:18.925944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:10:12.228931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200958+0000","last_change":"2024-09-13T18:52:17.949744+0000","last_active":"2024-09-13T18:52:20.200958+0000","last_peered":"2024-09-13T18:52:20.200958+0000","last_clean":"2024-09-13T18:52:20.200958+0000","last_became_active":"2024-09-13T18:52:17.949469+0000","last_became_peered":"2024-09-13T18:52:17.949469+0000","last_unstale":"2024-09-13T18:52:20.200958+0000","last_undegraded":"2024-09-13T18:52:20.200958+0000","last_fullsized":"2024-09-13T18:52:20.200958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:42:22.626590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195041+0000","last_change":"2024-09-13T18:52:17.537724+0000","last_active":"2024-09-13T18:52:20.195041+0000","last_peered":"2024-09-13T18:52:20.195041+0000","last_clean":"2024-09-13T18:52:20.195041+0000","last_became_active":"2024-09-13T18:52:17.537474+0000","last_became_peered":"2024-09-13T18:52:17.537474+0000","last_unstale":"2024-09-13T18:52:20.195041+0000","last_undegraded":"2024-09-13T18:52:20.195041+0000","last_fullsized":"2024-09-13T18:52:20.195041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:06.694707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199207+0000","last_change":"2024-09-13T18:52:17.939592+0000","last_active":"2024-09-13T18:52:20.199207+0000","last_peered":"2024-09-13T18:52:20.199207+0000","last_clean":"2024-09-13T18:52:20.199207+0000","last_became_active":"2024-09-13T18:52:17.939331+0000","last_became_peered":"2024-09-13T18:52:17.939331+0000","last_unstale":"2024-09-13T18:52:20.199207+0000","last_undegraded":"2024-09-13T18:52:20.199207+0000","last_fullsized":"2024-09-13T18:52:20.199207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:31.026983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.926147+0000","last_change":"2024-09-13T18:52:17.536715+0000","last_active":"2024-09-13T18:52:18.926147+0000","last_peered":"2024-09-13T18:52:18.926147+0000","last_clean":"2024-09-13T18:52:18.926147+0000","last_became_active":"2024-09-13T18:52:17.536409+0000","last_became_peered":"2024-09-13T18:52:17.536409+0000","last_unstale":"2024-09-13T18:52:18.926147+0000","last_undegraded":"2024-09-13T18:52:18.926147+0000","last_fullsized":"2024-09-13T18:52:18.926147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:24:01.849550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200621+0000","last_change":"2024-09-13T18:52:17.940926+0000","last_active":"2024-09-13T18:52:20.200621+0000","last_peered":"2024-09-13T18:52:20.200621+0000","last_clean":"2024-09-13T18:52:20.200621+0000","last_became_active":"2024-09-13T18:52:17.940572+0000","last_became_peered":"2024-09-13T18:52:17.940572+0000","last_unstale":"2024-09-13T18:52:20.200621+0000","last_undegraded":"2024-09-13T18:52:20.200621+0000","last_fullsized":"2024-09-13T18:52:20.200621+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:09.990929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-13T18:52:17.928742+0000","last_change":"2024-09-13T18:52:15.080080+0000","last_active":"2024-09-13T18:52:17.928742+0000","last_peered":"2024-09-13T18:52:17.928742+0000","last_clean":"2024-09-13T18:52:17.928742+0000","last_became_active":"2024-09-13T18:52:15.078100+0000","last_became_peered":"2024-09-13T18:52:15.078100+0000","last_unstale":"2024-09-13T18:52:17.928742+0000","last_undegraded":"2024-09-13T18:52:17.928742+0000","last_fullsized":"2024-09-13T18:52:17.928742+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_clean_scrub_stamp":"2024-09-13T18:52:13.196473+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:29:59.452757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197092+0000","last_change":"2024-09-13T18:52:17.531117+0000","last_active":"2024-09-13T18:52:20.197092+0000","last_peered":"2024-09-13T18:52:20.197092+0000","last_clean":"2024-09-13T18:52:20.197092+0000","last_became_active":"2024-09-13T18:52:17.530778+0000","last_became_peered":"2024-09-13T18:52:17.530778+0000","last_unstale":"2024-09-13T18:52:20.197092+0000","last_undegraded":"2024-09-13T18:52:20.197092+0000","last_fullsized":"2024-09-13T18:52:20.197092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:06.694707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924347+0000","last_change":"2024-09-13T18:52:17.943655+0000","last_active":"2024-09-13T18:52:18.924347+0000","last_peered":"2024-09-13T18:52:18.924347+0000","last_clean":"2024-09-13T18:52:18.924347+0000","last_became_active":"2024-09-13T18:52:17.943301+0000","last_became_peered":"2024-09-13T18:52:17.943301+0000","last_unstale":"2024-09-13T18:52:18.924347+0000","last_undegraded":"2024-09-13T18:52:18.924347+0000","last_fullsized":"2024-09-13T18:52:18.924347+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:53.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197196+0000","last_change":"2024-09-13T18:52:17.548220+0000","last_active":"2024-09-13T18:52:20.197196+0000","last_peered":"2024-09-13T18:52:20.197196+0000","last_clean":"2024-09-13T18:52:20.197196+0000","last_became_active":"2024-09-13T18:52:17.547957+0000","last_became_peered":"2024-09-13T18:52:17.547957+0000","last_unstale":"2024-09-13T18:52:20.197196+0000","last_undegraded":"2024-09-13T18:52:20.197196+0000","last_fullsized":"2024-09-13T18:52:20.197196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:47.942952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.194587+0000","last_change":"2024-09-13T18:52:17.946587+0000","last_active":"2024-09-13T18:52:20.194587+0000","last_peered":"2024-09-13T18:52:20.194587+0000","last_clean":"2024-09-13T18:52:20.194587+0000","last_became_active":"2024-09-13T18:52:17.946352+0000","last_became_peered":"2024-09-13T18:52:17.946352+0000","last_unstale":"2024-09-13T18:52:20.194587+0000","last_undegraded":"2024-09-13T18:52:20.194587+0000","last_fullsized":"2024-09-13T18:52:20.194587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:37.404109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.926130+0000","last_change":"2024-09-13T18:52:17.532972+0000","last_active":"2024-09-13T18:52:18.926130+0000","last_peered":"2024-09-13T18:52:18.926130+0000","last_clean":"2024-09-13T18:52:18.926130+0000","last_became_active":"2024-09-13T18:52:17.532675+0000","last_became_peered":"2024-09-13T18:52:17.532675+0000","last_unstale":"2024-09-13T18:52:18.926130+0000","last_undegraded":"2024-09-13T18:52:18.926130+0000","last_fullsized":"2024-09-13T18:52:18.926130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:41.232168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.194709+0000","last_change":"2024-09-13T18:52:17.955502+0000","last_active":"2024-09-13T18:52:20.194709+0000","last_peered":"2024-09-13T18:52:20.194709+0000","last_clean":"2024-09-13T18:52:20.194709+0000","last_became_active":"2024-09-13T18:52:17.955061+0000","last_became_peered":"2024-09-13T18:52:17.955061+0000","last_unstale":"2024-09-13T18:52:20.194709+0000","last_undegraded":"2024-09-13T18:52:20.194709+0000","last_fullsized":"2024-09-13T18:52:20.194709+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:50:24.025045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197805+0000","last_change":"2024-09-13T18:52:17.535698+0000","last_active":"2024-09-13T18:52:20.197805+0000","last_peered":"2024-09-13T18:52:20.197805+0000","last_clean":"2024-09-13T18:52:20.197805+0000","last_became_active":"2024-09-13T18:52:17.535439+0000","last_became_peered":"2024-09-13T18:52:17.535439+0000","last_unstale":"2024-09-13T18:52:20.197805+0000","last_undegraded":"2024-09-13T18:52:20.197805+0000","last_fullsized":"2024-09-13T18:52:20.197805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:47.942952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197950+0000","last_change":"2024-09-13T18:52:17.945038+0000","last_active":"2024-09-13T18:52:20.197950+0000","last_peered":"2024-09-13T18:52:20.197950+0000","last_clean":"2024-09-13T18:52:20.197950+0000","last_became_active":"2024-09-13T18:52:17.944770+0000","last_became_peered":"2024-09-13T18:52:17.944770+0000","last_unstale":"2024-09-13T18:52:20.197950+0000","last_undegraded":"2024-09-13T18:52:20.197950+0000","last_fullsized":"2024-09-13T18:52:20.197950+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:24:03.326121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196209+0000","last_change":"2024-09-13T18:52:17.539013+0000","last_active":"2024-09-13T18:52:20.196209+0000","last_peered":"2024-09-13T18:52:20.196209+0000","last_clean":"2024-09-13T18:52:20.196209+0000","last_became_active":"2024-09-13T18:52:17.538720+0000","last_became_peered":"2024-09-13T18:52:17.538720+0000","last_unstale":"2024-09-13T18:52:20.196209+0000","last_undegraded":"2024-09-13T18:52:20.196209+0000","last_fullsized":"2024-09-13T18:52:20.196209+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:49.151192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199583+0000","last_change":"2024-09-13T18:52:17.943875+0000","last_active":"2024-09-13T18:52:20.199583+0000","last_peered":"2024-09-13T18:52:20.199583+0000","last_clean":"2024-09-13T18:52:20.199583+0000","last_became_active":"2024-09-13T18:52:17.943557+0000","last_became_peered":"2024-09-13T18:52:17.943557+0000","last_unstale":"2024-09-13T18:52:20.199583+0000","last_undegraded":"2024-09-13T18:52:20.199583+0000","last_fullsized":"2024-09-13T18:52:20.199583+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:18.183823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200835+0000","last_change":"2024-09-13T18:52:17.548571+0000","last_active":"2024-09-13T18:52:20.200835+0000","last_peered":"2024-09-13T18:52:20.200835+0000","last_clean":"2024-09-13T18:52:20.200835+0000","last_became_active":"2024-09-13T18:52:17.548306+0000","last_became_peered":"2024-09-13T18:52:17.548306+0000","last_unstale":"2024-09-13T18:52:20.200835+0000","last_undegraded":"2024-09-13T18:52:20.200835+0000","last_fullsized":"2024-09-13T18:52:20.200835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:22:26.864392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195209+0000","last_change":"2024-09-13T18:52:17.954517+0000","last_active":"2024-09-13T18:52:20.195209+0000","last_peered":"2024-09-13T18:52:20.195209+0000","last_clean":"2024-09-13T18:52:20.195209+0000","last_became_active":"2024-09-13T18:52:17.954264+0000","last_became_peered":"2024-09-13T18:52:17.954264+0000","last_unstale":"2024-09-13T18:52:20.195209+0000","last_undegraded":"2024-09-13T18:52:20.195209+0000","last_fullsized":"2024-09-13T18:52:20.195209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:57:51.000231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:21.214627+0000","last_change":"2024-09-13T18:52:17.538927+0000","last_active":"2024-09-13T18:52:21.214627+0000","last_peered":"2024-09-13T18:52:21.214627+0000","last_clean":"2024-09-13T18:52:21.214627+0000","last_became_active":"2024-09-13T18:52:17.538334+0000","last_became_peered":"2024-09-13T18:52:17.538334+0000","last_unstale":"2024-09-13T18:52:21.214627+0000","last_undegraded":"2024-09-13T18:52:21.214627+0000","last_fullsized":"2024-09-13T18:52:21.214627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:30:47.942952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924572+0000","last_change":"2024-09-13T18:52:17.929155+0000","last_active":"2024-09-13T18:52:18.924572+0000","last_peered":"2024-09-13T18:52:18.924572+0000","last_clean":"2024-09-13T18:52:18.924572+0000","last_became_active":"2024-09-13T18:52:17.928876+0000","last_became_peered":"2024-09-13T18:52:17.928876+0000","last_unstale":"2024-09-13T18:52:18.924572+0000","last_undegraded":"2024-09-13T18:52:18.924572+0000","last_fullsized":"2024-09-13T18:52:18.924572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:01:39.422836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.928355+0000","last_change":"2024-09-13T18:52:17.543500+0000","last_active":"2024-09-13T18:52:18.928355+0000","last_peered":"2024-09-13T18:52:18.928355+0000","last_clean":"2024-09-13T18:52:18.928355+0000","last_became_active":"2024-09-13T18:52:17.543184+0000","last_became_peered":"2024-09-13T18:52:17.543184+0000","last_unstale":"2024-09-13T18:52:18.928355+0000","last_undegraded":"2024-09-13T18:52:18.928355+0000","last_fullsized":"2024-09-13T18:52:18.928355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:05:28.094741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195839+0000","last_change":"2024-09-13T18:52:17.942277+0000","last_active":"2024-09-13T18:52:20.195839+0000","last_peered":"2024-09-13T18:52:20.195839+0000","last_clean":"2024-09-13T18:52:20.195839+0000","last_became_active":"2024-09-13T18:52:17.941984+0000","last_became_peered":"2024-09-13T18:52:17.941984+0000","last_unstale":"2024-09-13T18:52:20.195839+0000","last_undegraded":"2024-09-13T18:52:20.195839+0000","last_fullsized":"2024-09-13T18:52:20.195839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:20:42.480429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.197001+0000","last_change":"2024-09-13T18:52:17.545742+0000","last_active":"2024-09-13T18:52:20.197001+0000","last_peered":"2024-09-13T18:52:20.197001+0000","last_clean":"2024-09-13T18:52:20.197001+0000","last_became_active":"2024-09-13T18:52:17.545448+0000","last_became_peered":"2024-09-13T18:52:17.545448+0000","last_unstale":"2024-09-13T18:52:20.197001+0000","last_undegraded":"2024-09-13T18:52:20.197001+0000","last_fullsized":"2024-09-13T18:52:20.197001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:19:49.151192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196848+0000","last_change":"2024-09-13T18:52:17.955942+0000","last_active":"2024-09-13T18:52:20.196848+0000","last_peered":"2024-09-13T18:52:20.196848+0000","last_clean":"2024-09-13T18:52:20.196848+0000","last_became_active":"2024-09-13T18:52:17.955598+0000","last_became_peered":"2024-09-13T18:52:17.955598+0000","last_unstale":"2024-09-13T18:52:20.196848+0000","last_undegraded":"2024-09-13T18:52:20.196848+0000","last_fullsized":"2024-09-13T18:52:20.196848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:36:42.236037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196212+0000","last_change":"2024-09-13T18:52:17.541045+0000","last_active":"2024-09-13T18:52:20.196212+0000","last_peered":"2024-09-13T18:52:20.196212+0000","last_clean":"2024-09-13T18:52:20.196212+0000","last_became_active":"2024-09-13T18:52:17.540627+0000","last_became_peered":"2024-09-13T18:52:17.540627+0000","last_unstale":"2024-09-13T18:52:20.196212+0000","last_undegraded":"2024-09-13T18:52:20.196212+0000","last_fullsized":"2024-09-13T18:52:20.196212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:14:15.613948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924137+0000","last_change":"2024-09-13T18:52:17.924064+0000","last_active":"2024-09-13T18:52:18.924137+0000","last_peered":"2024-09-13T18:52:18.924137+0000","last_clean":"2024-09-13T18:52:18.924137+0000","last_became_active":"2024-09-13T18:52:17.921690+0000","last_became_peered":"2024-09-13T18:52:17.921690+0000","last_unstale":"2024-09-13T18:52:18.924137+0000","last_undegraded":"2024-09-13T18:52:18.924137+0000","last_fullsized":"2024-09-13T18:52:18.924137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:30.871944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195867+0000","last_change":"2024-09-13T18:52:17.533845+0000","last_active":"2024-09-13T18:52:20.195867+0000","last_peered":"2024-09-13T18:52:20.195867+0000","last_clean":"2024-09-13T18:52:20.195867+0000","last_became_active":"2024-09-13T18:52:17.533579+0000","last_became_peered":"2024-09-13T18:52:17.533579+0000","last_unstale":"2024-09-13T18:52:20.195867+0000","last_undegraded":"2024-09-13T18:52:20.195867+0000","last_fullsized":"2024-09-13T18:52:20.195867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:44:04.695688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925441+0000","last_change":"2024-09-13T18:52:17.944835+0000","last_active":"2024-09-13T18:52:18.925441+0000","last_peered":"2024-09-13T18:52:18.925441+0000","last_clean":"2024-09-13T18:52:18.925441+0000","last_became_active":"2024-09-13T18:52:17.944469+0000","last_became_peered":"2024-09-13T18:52:17.944469+0000","last_unstale":"2024-09-13T18:52:18.925441+0000","last_undegraded":"2024-09-13T18:52:18.925441+0000","last_fullsized":"2024-09-13T18:52:18.925441+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:31.026983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200263+0000","last_change":"2024-09-13T18:52:17.545810+0000","last_active":"2024-09-13T18:52:20.200263+0000","last_peered":"2024-09-13T18:52:20.200263+0000","last_clean":"2024-09-13T18:52:20.200263+0000","last_became_active":"2024-09-13T18:52:17.545523+0000","last_became_peered":"2024-09-13T18:52:17.545523+0000","last_unstale":"2024-09-13T18:52:20.200263+0000","last_undegraded":"2024-09-13T18:52:20.200263+0000","last_fullsized":"2024-09-13T18:52:20.200263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:05:28.094741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.927471+0000","last_change":"2024-09-13T18:52:17.940857+0000","last_active":"2024-09-13T18:52:18.927471+0000","last_peered":"2024-09-13T18:52:18.927471+0000","last_clean":"2024-09-13T18:52:18.927471+0000","last_became_active":"2024-09-13T18:52:17.940675+0000","last_became_peered":"2024-09-13T18:52:17.940675+0000","last_unstale":"2024-09-13T18:52:18.927471+0000","last_undegraded":"2024-09-13T18:52:18.927471+0000","last_fullsized":"2024-09-13T18:52:18.927471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:53.166551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199367+0000","last_change":"2024-09-13T18:52:17.947933+0000","last_active":"2024-09-13T18:52:20.199367+0000","last_peered":"2024-09-13T18:52:20.199367+0000","last_clean":"2024-09-13T18:52:20.199367+0000","last_became_active":"2024-09-13T18:52:17.947665+0000","last_became_peered":"2024-09-13T18:52:17.947665+0000","last_unstale":"2024-09-13T18:52:20.199367+0000","last_undegraded":"2024-09-13T18:52:20.199367+0000","last_fullsized":"2024-09-13T18:52:20.199367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:21.405599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199849+0000","last_change":"2024-09-13T18:52:17.548211+0000","last_active":"2024-09-13T18:52:20.199849+0000","last_peered":"2024-09-13T18:52:20.199849+0000","last_clean":"2024-09-13T18:52:20.199849+0000","last_became_active":"2024-09-13T18:52:17.547997+0000","last_became_peered":"2024-09-13T18:52:17.547997+0000","last_unstale":"2024-09-13T18:52:20.199849+0000","last_undegraded":"2024-09-13T18:52:20.199849+0000","last_fullsized":"2024-09-13T18:52:20.199849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:06.694707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.199118+0000","last_change":"2024-09-13T18:52:17.948133+0000","last_active":"2024-09-13T18:52:20.199118+0000","last_peered":"2024-09-13T18:52:20.199118+0000","last_clean":"2024-09-13T18:52:20.199118+0000","last_became_active":"2024-09-13T18:52:17.947793+0000","last_became_peered":"2024-09-13T18:52:17.947793+0000","last_unstale":"2024-09-13T18:52:20.199118+0000","last_undegraded":"2024-09-13T18:52:20.199118+0000","last_fullsized":"2024-09-13T18:52:20.199118+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:20:43.355560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.200592+0000","last_change":"2024-09-13T18:52:17.548922+0000","last_active":"2024-09-13T18:52:20.200592+0000","last_peered":"2024-09-13T18:52:20.200592+0000","last_clean":"2024-09-13T18:52:20.200592+0000","last_became_active":"2024-09-13T18:52:17.548775+0000","last_became_peered":"2024-09-13T18:52:17.548775+0000","last_unstale":"2024-09-13T18:52:20.200592+0000","last_undegraded":"2024-09-13T18:52:20.200592+0000","last_fullsized":"2024-09-13T18:52:20.200592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:14:15.613948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.195999+0000","last_change":"2024-09-13T18:52:17.950416+0000","last_active":"2024-09-13T18:52:20.195999+0000","last_peered":"2024-09-13T18:52:20.195999+0000","last_clean":"2024-09-13T18:52:20.195999+0000","last_became_active":"2024-09-13T18:52:17.949988+0000","last_became_peered":"2024-09-13T18:52:17.949988+0000","last_unstale":"2024-09-13T18:52:20.195999+0000","last_undegraded":"2024-09-13T18:52:20.195999+0000","last_fullsized":"2024-09-13T18:52:20.195999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:10:45.768218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.925140+0000","last_change":"2024-09-13T18:52:17.535536+0000","last_active":"2024-09-13T18:52:18.925140+0000","last_peered":"2024-09-13T18:52:18.925140+0000","last_clean":"2024-09-13T18:52:18.925140+0000","last_became_active":"2024-09-13T18:52:17.534924+0000","last_became_peered":"2024-09-13T18:52:17.534924+0000","last_unstale":"2024-09-13T18:52:18.925140+0000","last_undegraded":"2024-09-13T18:52:18.925140+0000","last_fullsized":"2024-09-13T18:52:18.925140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:55.776670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:18.924931+0000","last_change":"2024-09-13T18:52:17.931585+0000","last_active":"2024-09-13T18:52:18.924931+0000","last_peered":"2024-09-13T18:52:18.924931+0000","last_clean":"2024-09-13T18:52:18.924931+0000","last_became_active":"2024-09-13T18:52:17.931280+0000","last_became_peered":"2024-09-13T18:52:17.931280+0000","last_unstale":"2024-09-13T18:52:18.924931+0000","last_undegraded":"2024-09-13T18:52:18.924931+0000","last_fullsized":"2024-09-13T18:52:18.924931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:17.509490+0000","last_clean_scrub_stamp":"2024-09-13T18:52:17.509490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:20:42.480429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T18:52:20.196202+0000","last_change":"2024-09-13T18:52:17.542275+0000","last_active":"2024-09-13T18:52:20.196202+0000","last_peered":"2024-09-13T18:52:20.196202+0000","last_clean":"2024-09-13T18:52:20.196202+0000","last_became_active":"2024-09-13T18:52:17.541964+0000","last_became_peered":"2024-09-13T18:52:17.541964+0000","last_unstale":"2024-09-13T18:52:20.196202+0000","last_undegraded":"2024-09-13T18:52:20.196202+0000","last_fullsized":"2024-09-13T18:52:20.196202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:16.505621+0000","last_clean_scrub_stamp":"2024-09-13T18:52:16.505621+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:55.776670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":823296,"data_stored":669740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":80354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":78880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":78916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":237568,"data_stored":80354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93799999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.373}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":225280,"data_stored":78790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.145}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1739999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":233472,"data_stored":78950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2270000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":831488,"data_stored":669796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.268}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:52:26.445 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T18:52:26.445 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T18:52:26.445 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T18:52:26.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T18:52:26.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.545+0000 7faa574ee700 1 Processor -- start 2024-09-13T18:52:26.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.545+0000 7faa574ee700 1 -- start start 2024-09-13T18:52:26.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.545+0000 7faa574ee700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7faa50159720 0x7faa5006cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.545+0000 7faa574ee700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5006af00 con 0x7faa5006aa90 2024-09-13T18:52:26.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.545+0000 7faa574ee700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5006d690 con 0x7faa500b3ae0 2024-09-13T18:52:26.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.545+0000 7faa574ee700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa500cbfe0 con 0x7faa50159720 2024-09-13T18:52:26.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.545+0000 7faa55ceb700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7faa5006aa90 0x7faa500b33d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35902/0 (socket says 172.21.15.7:35902) 2024-09-13T18:52:26.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.545+0000 7faa55ceb700 1 -- 172.21.15.7:0/1311617932 learned_addr learned my addr 172.21.15.7:0/1311617932 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:26.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa564ec700 1 --2- 172.21.15.7:0/1311617932 >> v2:172.21.15.171:3300/0 conn(0x7faa50159720 0x7faa5006cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 -- 172.21.15.7:0/1311617932 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316672308 0 0) 0x7faa5006af00 con 0x7faa5006aa90 2024-09-13T18:52:26.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 -- 172.21.15.7:0/1311617932 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa380035d0 con 0x7faa5006aa90 2024-09-13T18:52:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 -- 172.21.15.7:0/1311617932 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 569485583 0 0) 0x7faa5006d690 con 0x7faa500b3ae0 2024-09-13T18:52:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 -- 172.21.15.7:0/1311617932 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa5006af00 con 0x7faa500b3ae0 2024-09-13T18:52:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 -- 172.21.15.7:0/1311617932 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3712787722 0 0) 0x7faa380035d0 con 0x7faa5006aa90 2024-09-13T18:52:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 -- 172.21.15.7:0/1311617932 >> v2:172.21.15.171:3300/0 conn(0x7faa50159720 msgr2=0x7faa5006cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 --2- 172.21.15.7:0/1311617932 >> v2:172.21.15.171:3300/0 conn(0x7faa50159720 0x7faa5006cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 -- 172.21.15.7:0/1311617932 >> v1:172.21.15.171:6789/0 conn(0x7faa500b3ae0 legacy=0x7faa500ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 -- 172.21.15.7:0/1311617932 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa500cc2c0 con 0x7faa5006aa90 2024-09-13T18:52:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 -- 172.21.15.7:0/1311617932 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7faa4c0019f0 con 0x7faa5006aa90 2024-09-13T18:52:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 -- 172.21.15.7:0/1311617932 >> v1:172.21.15.7:6789/0 conn(0x7faa5006aa90 legacy=0x7faa500b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 --2- 172.21.15.7:0/1311617932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa38003bb0 0x7faa38003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 --2- 172.21.15.7:0/1311617932 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faa38004870 0x7faa38004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 --2- 172.21.15.7:0/1311617932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa38005230 0x7faa3800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 -- 172.21.15.7:0/1311617932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa500cbfe0 con 0x7faa38005230 2024-09-13T18:52:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 -- 172.21.15.7:0/1311617932 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3800e370 con 0x7faa38003bb0 2024-09-13T18:52:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 -- 172.21.15.7:0/1311617932 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3800e530 con 0x7faa38004870 2024-09-13T18:52:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa56ced700 1 --2- 172.21.15.7:0/1311617932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa38005230 0x7faa3800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa564ec700 1 --2- 172.21.15.7:0/1311617932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa38003bb0 0x7faa38003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa56ced700 1 -- 172.21.15.7:0/1311617932 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faa38004870 msgr2=0x7faa38004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa55ceb700 1 --2- 172.21.15.7:0/1311617932 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faa38004870 0x7faa38004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa56ced700 1 --2- 172.21.15.7:0/1311617932 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faa38004870 0x7faa38004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa56ced700 1 -- 172.21.15.7:0/1311617932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa38003bb0 msgr2=0x7faa38003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa56ced700 1 --2- 172.21.15.7:0/1311617932 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa38003bb0 0x7faa38003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa56ced700 1 -- 172.21.15.7:0/1311617932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa380035d0 con 0x7faa38005230 2024-09-13T18:52:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa55ceb700 1 --2- 172.21.15.7:0/1311617932 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faa38004870 0x7faa38004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:26.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa56ced700 1 -- 172.21.15.7:0/1311617932 reap_dead start 2024-09-13T18:52:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa56ced700 1 --2- 172.21.15.7:0/1311617932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa38005230 0x7faa3800ba30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faa48001960 tx=0x7faa380058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb864379e997487 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 -- 172.21.15.7:0/1311617932 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa48001450 con 0x7faa38005230 2024-09-13T18:52:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 -- 172.21.15.7:0/1311617932 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa480052e0 con 0x7faa38005230 2024-09-13T18:52:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa554ea700 1 -- 172.21.15.7:0/1311617932 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa480036a0 con 0x7faa38005230 2024-09-13T18:52:26.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa574ee700 1 -- 172.21.15.7:0/1311617932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa38005230 msgr2=0x7faa3800ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa574ee700 1 --2- 172.21.15.7:0/1311617932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa38005230 0x7faa3800ba30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faa48001960 tx=0x7faa380058a0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa574ee700 1 -- 172.21.15.7:0/1311617932 shutdown_connections 2024-09-13T18:52:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa574ee700 1 --2- 172.21.15.7:0/1311617932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa38005230 0x7faa3800ba30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa574ee700 1 -- 172.21.15.7:0/1311617932 >> 172.21.15.7:0/1311617932 conn(0x7faa500b0f00 msgr2=0x7faa500b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa574ee700 1 -- 172.21.15.7:0/1311617932 shutdown_connections 2024-09-13T18:52:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.549+0000 7faa574ee700 1 -- 172.21.15.7:0/1311617932 wait complete. 2024-09-13T18:52:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.553+0000 7faa574ee700 1 Processor -- start 2024-09-13T18:52:26.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.553+0000 7faa574ee700 1 -- start start 2024-09-13T18:52:26.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa574ee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa38005230 0x7faa50151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa574ee700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faa501f1ad0 0x7faa5015c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa574ee700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa501f1f50 0x7faa5015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa574ee700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5013da90 con 0x7faa38005230 2024-09-13T18:52:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa574ee700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa50151d80 con 0x7faa501f1f50 2024-09-13T18:52:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa574ee700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa501507e0 con 0x7faa501f1ad0 2024-09-13T18:52:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa564ec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa38005230 0x7faa50151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa564ec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa38005230 0x7faa50151840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49650/0 (socket says 172.21.15.7:49650) 2024-09-13T18:52:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa564ec700 1 -- 172.21.15.7:0/1228384804 learned_addr learned my addr 172.21.15.7:0/1228384804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa564ec700 1 -- 172.21.15.7:0/1228384804 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faa501f1ad0 msgr2=0x7faa5015c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa55ceb700 1 --2- 172.21.15.7:0/1228384804 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faa501f1ad0 0x7faa5015c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa56ced700 1 --2- 172.21.15.7:0/1228384804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa501f1f50 0x7faa5015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa564ec700 1 --2- 172.21.15.7:0/1228384804 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faa501f1ad0 0x7faa5015c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa564ec700 1 -- 172.21.15.7:0/1228384804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa501f1f50 msgr2=0x7faa5015ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa564ec700 1 --2- 172.21.15.7:0/1228384804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa501f1f50 0x7faa5015ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa564ec700 1 -- 172.21.15.7:0/1228384804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa50150a80 con 0x7faa38005230 2024-09-13T18:52:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa56ced700 1 --2- 172.21.15.7:0/1228384804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa501f1f50 0x7faa5015ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa564ec700 1 --2- 172.21.15.7:0/1228384804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa38005230 0x7faa50151840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7faa400108f0 tx=0x7faa40010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa46ffd700 1 -- 172.21.15.7:0/1228384804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa40010e90 con 0x7faa38005230 2024-09-13T18:52:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa46ffd700 1 -- 172.21.15.7:0/1228384804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4000e8d0 con 0x7faa38005230 2024-09-13T18:52:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa46ffd700 1 -- 172.21.15.7:0/1228384804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4000ec40 con 0x7faa38005230 2024-09-13T18:52:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa574ee700 1 -- 172.21.15.7:0/1228384804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa50150d60 con 0x7faa38005230 2024-09-13T18:52:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa574ee700 1 -- 172.21.15.7:0/1228384804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa50151320 con 0x7faa38005230 2024-09-13T18:52:26.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.557+0000 7faa574ee700 1 -- 172.21.15.7:0/1228384804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa500b33d0 con 0x7faa38005230 2024-09-13T18:52:26.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.561+0000 7faa46ffd700 1 -- 172.21.15.7:0/1228384804 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faa40016460 con 0x7faa38005230 2024-09-13T18:52:26.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.561+0000 7faa46ffd700 1 --2- 172.21.15.7:0/1228384804 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7faa3c066730 0x7faa3c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.561+0000 7faa46ffd700 1 -- 172.21.15.7:0/1228384804 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa400844f0 con 0x7faa38005230 2024-09-13T18:52:26.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.561+0000 7faa55ceb700 1 --2- 172.21.15.7:0/1228384804 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7faa3c066730 0x7faa3c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.561+0000 7faa55ceb700 1 --2- 172.21.15.7:0/1228384804 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7faa3c066730 0x7faa3c068bf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faa4c001580 tx=0x7faa4c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:26.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.569+0000 7faa46ffd700 1 -- 172.21.15.7:0/1228384804 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faa400517c0 con 0x7faa38005230 2024-09-13T18:52:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.765+0000 7faa574ee700 1 -- 172.21.15.7:0/1228384804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7faa500cb550 con 0x7faa38005230 2024-09-13T18:52:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.765+0000 7faa46ffd700 1 -- 172.21.15.7:0/1228384804 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7faa40054de0 con 0x7faa38005230 2024-09-13T18:52:26.773 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:26.773 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T18:52:26.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.769+0000 7faa574ee700 1 -- 172.21.15.7:0/1228384804 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7faa3c066730 msgr2=0x7faa3c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.769+0000 7faa574ee700 1 --2- 172.21.15.7:0/1228384804 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7faa3c066730 0x7faa3c068bf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faa4c001580 tx=0x7faa4c009040 comp rx=0 tx=0).stop 2024-09-13T18:52:26.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.769+0000 7faa574ee700 1 -- 172.21.15.7:0/1228384804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa38005230 msgr2=0x7faa50151840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.769+0000 7faa574ee700 1 --2- 172.21.15.7:0/1228384804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa38005230 0x7faa50151840 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7faa400108f0 tx=0x7faa40010cb0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.769+0000 7faa574ee700 1 -- 172.21.15.7:0/1228384804 shutdown_connections 2024-09-13T18:52:26.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.769+0000 7faa574ee700 1 --2- 172.21.15.7:0/1228384804 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7faa3c066730 0x7faa3c068bf0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.769+0000 7faa574ee700 1 --2- 172.21.15.7:0/1228384804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faa501f1f50 0x7faa5015ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.769+0000 7faa574ee700 1 --2- 172.21.15.7:0/1228384804 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faa501f1ad0 0x7faa5015c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.769+0000 7faa574ee700 1 --2- 172.21.15.7:0/1228384804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa38005230 0x7faa50151840 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.769+0000 7faa574ee700 1 -- 172.21.15.7:0/1228384804 >> 172.21.15.7:0/1228384804 conn(0x7faa500b0f00 msgr2=0x7faa500af740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:26.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.769+0000 7faa574ee700 1 -- 172.21.15.7:0/1228384804 shutdown_connections 2024-09-13T18:52:26.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.769+0000 7faa574ee700 1 -- 172.21.15.7:0/1228384804 wait complete. 2024-09-13T18:52:26.787 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T18:52:26.787 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:26.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.885+0000 7f3a87fff700 1 Processor -- start 2024-09-13T18:52:26.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.885+0000 7f3a87fff700 1 -- start start 2024-09-13T18:52:26.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.885+0000 7f3a87fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3a881433c0 0x7f3a8806cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.885+0000 7f3a87fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3a8806d4d0 0x7f3a880b3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.885+0000 7f3a87fff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f3a880b39a0 0x7f3a88070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.885+0000 7f3a87fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a880b3e50 con 0x7f3a881433c0 2024-09-13T18:52:26.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.885+0000 7f3a87fff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a8806b0b0 con 0x7f3a880b39a0 2024-09-13T18:52:26.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.885+0000 7f3a87fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a880cbe00 con 0x7f3a8806d4d0 2024-09-13T18:52:26.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.885+0000 7f3a86ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3a881433c0 0x7f3a8806cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.885+0000 7f3a86ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3a881433c0 0x7f3a8806cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49666/0 (socket says 172.21.15.7:49666) 2024-09-13T18:52:26.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.885+0000 7f3a86ffd700 1 -- 172.21.15.7:0/109914619 learned_addr learned my addr 172.21.15.7:0/109914619 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:26.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a877fe700 1 --2- 172.21.15.7:0/109914619 >> v2:172.21.15.171:3300/0 conn(0x7f3a880b39a0 0x7f3a88070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a867fc700 1 --2- 172.21.15.7:0/109914619 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3a8806d4d0 0x7f3a880b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a86ffd700 1 -- 172.21.15.7:0/109914619 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3a8806d4d0 msgr2=0x7f3a880b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:26.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a86ffd700 1 --2- 172.21.15.7:0/109914619 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3a8806d4d0 0x7f3a880b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a86ffd700 1 -- 172.21.15.7:0/109914619 >> v2:172.21.15.171:3300/0 conn(0x7f3a880b39a0 msgr2=0x7f3a88070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:26.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a86ffd700 1 --2- 172.21.15.7:0/109914619 >> v2:172.21.15.171:3300/0 conn(0x7f3a880b39a0 0x7f3a88070c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a86ffd700 1 -- 172.21.15.7:0/109914619 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8806da30 con 0x7f3a881433c0 2024-09-13T18:52:26.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a867fc700 1 --2- 172.21.15.7:0/109914619 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3a8806d4d0 0x7f3a880b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:26.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a877fe700 1 --2- 172.21.15.7:0/109914619 >> v2:172.21.15.171:3300/0 conn(0x7f3a880b39a0 0x7f3a88070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:26.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a86ffd700 1 --2- 172.21.15.7:0/109914619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a881433c0 0x7f3a8806cf90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3a880cb500 tx=0x7f3a78009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ea9d6ad26256a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:26.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a85ffb700 1 -- 172.21.15.7:0/109914619 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a78011830 con 0x7f3a881433c0 2024-09-13T18:52:26.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a85ffb700 1 -- 172.21.15.7:0/109914619 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a78011990 con 0x7f3a881433c0 2024-09-13T18:52:26.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a85ffb700 1 -- 172.21.15.7:0/109914619 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a78011d20 con 0x7f3a881433c0 2024-09-13T18:52:26.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a87fff700 1 -- 172.21.15.7:0/109914619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a881433c0 msgr2=0x7f3a8806cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a87fff700 1 --2- 172.21.15.7:0/109914619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a881433c0 0x7f3a8806cf90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3a880cb500 tx=0x7f3a78009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a87fff700 1 -- 172.21.15.7:0/109914619 shutdown_connections 2024-09-13T18:52:26.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a87fff700 1 --2- 172.21.15.7:0/109914619 >> v2:172.21.15.171:3300/0 conn(0x7f3a880b39a0 0x7f3a88070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a87fff700 1 --2- 172.21.15.7:0/109914619 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3a8806d4d0 0x7f3a880b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a87fff700 1 --2- 172.21.15.7:0/109914619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a881433c0 0x7f3a8806cf90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a87fff700 1 -- 172.21.15.7:0/109914619 >> 172.21.15.7:0/109914619 conn(0x7f3a880b0f00 msgr2=0x7f3a880b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:26.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a87fff700 1 -- 172.21.15.7:0/109914619 shutdown_connections 2024-09-13T18:52:26.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.889+0000 7f3a87fff700 1 -- 172.21.15.7:0/109914619 wait complete. 2024-09-13T18:52:26.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a87fff700 1 Processor -- start 2024-09-13T18:52:26.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a87fff700 1 -- start start 2024-09-13T18:52:26.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a87fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3a8806d4d0 0x7f3a88137aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a87fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a880b39a0 0x7f3a881573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a87fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a881433c0 0x7f3a88159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a87fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a88136fe0 con 0x7f3a881433c0 2024-09-13T18:52:26.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a87fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a88157920 con 0x7f3a880b39a0 2024-09-13T18:52:26.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a87fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a88136a20 con 0x7f3a8806d4d0 2024-09-13T18:52:26.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a881433c0 0x7f3a88159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a881433c0 0x7f3a88159e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49670/0 (socket says 172.21.15.7:49670) 2024-09-13T18:52:26.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a877fe700 1 -- 172.21.15.7:0/541176494 learned_addr learned my addr 172.21.15.7:0/541176494 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:26.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a867fc700 1 --2- 172.21.15.7:0/541176494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a880b39a0 0x7f3a881573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a86ffd700 1 --2- 172.21.15.7:0/541176494 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3a8806d4d0 0x7f3a88137aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a877fe700 1 -- 172.21.15.7:0/541176494 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3a8806d4d0 msgr2=0x7f3a88137aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a877fe700 1 --2- 172.21.15.7:0/541176494 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3a8806d4d0 0x7f3a88137aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a877fe700 1 -- 172.21.15.7:0/541176494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a880b39a0 msgr2=0x7f3a881573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:26.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a877fe700 1 --2- 172.21.15.7:0/541176494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a880b39a0 0x7f3a881573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:26.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a877fe700 1 -- 172.21.15.7:0/541176494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a700094b0 con 0x7f3a881433c0 2024-09-13T18:52:26.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a877fe700 1 --2- 172.21.15.7:0/541176494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a881433c0 0x7f3a88159e90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000efc0 tx=0x7f3a8000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:26.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a7f7fe700 1 -- 172.21.15.7:0/541176494 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80009d70 con 0x7f3a881433c0 2024-09-13T18:52:26.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a7f7fe700 1 -- 172.21.15.7:0/541176494 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8000f040 con 0x7f3a881433c0 2024-09-13T18:52:26.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a87fff700 1 -- 172.21.15.7:0/541176494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a80009a70 con 0x7f3a881433c0 2024-09-13T18:52:26.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a7f7fe700 1 -- 172.21.15.7:0/541176494 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8000ce00 con 0x7f3a881433c0 2024-09-13T18:52:26.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.893+0000 7f3a87fff700 1 -- 172.21.15.7:0/541176494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a78009570 con 0x7f3a881433c0 2024-09-13T18:52:26.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.897+0000 7f3a87fff700 1 -- 172.21.15.7:0/541176494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a880b3460 con 0x7f3a881433c0 2024-09-13T18:52:26.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.897+0000 7f3a7f7fe700 1 -- 172.21.15.7:0/541176494 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3a80014480 con 0x7f3a881433c0 2024-09-13T18:52:26.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.897+0000 7f3a7f7fe700 1 --2- 172.21.15.7:0/541176494 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3a6c066730 0x7f3a6c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:26.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.897+0000 7f3a7f7fe700 1 -- 172.21.15.7:0/541176494 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a80084ca0 con 0x7f3a881433c0 2024-09-13T18:52:26.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.897+0000 7f3a86ffd700 1 --2- 172.21.15.7:0/541176494 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3a6c066730 0x7f3a6c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:26.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.897+0000 7f3a86ffd700 1 --2- 172.21.15.7:0/541176494 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3a6c066730 0x7f3a6c068bf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3a78014040 tx=0x7f3a78008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:26.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:26.901+0000 7f3a7f7fe700 1 -- 172.21.15.7:0/541176494 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3a80051ee0 con 0x7f3a881433c0 2024-09-13T18:52:27.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.073+0000 7f3a87fff700 1 -- 172.21.15.7:0/541176494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a880ae630 con 0x7f3a881433c0 2024-09-13T18:52:27.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.073+0000 7f3a7f7fe700 1 -- 172.21.15.7:0/541176494 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f3a80018020 con 0x7f3a881433c0 2024-09-13T18:52:27.080 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:27.080 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:27.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.077+0000 7f3a87fff700 1 -- 172.21.15.7:0/541176494 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3a6c066730 msgr2=0x7f3a6c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.077+0000 7f3a87fff700 1 --2- 172.21.15.7:0/541176494 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3a6c066730 0x7f3a6c068bf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3a78014040 tx=0x7f3a78008030 comp rx=0 tx=0).stop 2024-09-13T18:52:27.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.077+0000 7f3a87fff700 1 -- 172.21.15.7:0/541176494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a881433c0 msgr2=0x7f3a88159e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.077+0000 7f3a87fff700 1 --2- 172.21.15.7:0/541176494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a881433c0 0x7f3a88159e90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000efc0 tx=0x7f3a8000c3a0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.077+0000 7f3a87fff700 1 -- 172.21.15.7:0/541176494 shutdown_connections 2024-09-13T18:52:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.077+0000 7f3a87fff700 1 --2- 172.21.15.7:0/541176494 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3a6c066730 0x7f3a6c068bf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.077+0000 7f3a87fff700 1 --2- 172.21.15.7:0/541176494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a881433c0 0x7f3a88159e90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.077+0000 7f3a87fff700 1 --2- 172.21.15.7:0/541176494 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3a880b39a0 0x7f3a881573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.077+0000 7f3a87fff700 1 --2- 172.21.15.7:0/541176494 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3a8806d4d0 0x7f3a88137aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.077+0000 7f3a87fff700 1 -- 172.21.15.7:0/541176494 >> 172.21.15.7:0/541176494 conn(0x7f3a880b0f00 msgr2=0x7f3a880b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.077+0000 7f3a87fff700 1 -- 172.21.15.7:0/541176494 shutdown_connections 2024-09-13T18:52:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.077+0000 7f3a87fff700 1 -- 172.21.15.7:0/541176494 wait complete. 2024-09-13T18:52:27.094 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:52:15.554683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:52:16.896879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:52:27.094 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:52:27.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.205+0000 7fadebfff700 1 Processor -- start 2024-09-13T18:52:27.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.205+0000 7fadebfff700 1 -- start start 2024-09-13T18:52:27.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.205+0000 7fadebfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fadec06a520 0x7fadec06a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:27.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.205+0000 7fadebfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadec0b8430 con 0x7fadec15cda0 2024-09-13T18:52:27.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.205+0000 7fadebfff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadec06ae70 con 0x7fadec06b900 2024-09-13T18:52:27.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.205+0000 7fadebfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadec0b64f0 con 0x7fadec06a520 2024-09-13T18:52:27.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.205+0000 7fadeb7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fadec06a520 0x7fadec06a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:27.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.205+0000 7fadeaffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fadec15cda0 0x7fadec0ceb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35908/0 (socket says 172.21.15.7:35908) 2024-09-13T18:52:27.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.205+0000 7fadeaffd700 1 -- 172.21.15.7:0/1634230767 learned_addr learned my addr 172.21.15.7:0/1634230767 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fade9ffb700 1 -- 172.21.15.7:0/1634230767 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830783935 0 0) 0x7fadec0b8430 con 0x7fadec15cda0 2024-09-13T18:52:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadeb7fe700 1 -- 172.21.15.7:0/1634230767 >> v1:172.21.15.171:6789/0 conn(0x7fadec06b900 legacy=0x7fadec0b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadeb7fe700 1 -- 172.21.15.7:0/1634230767 >> v1:172.21.15.7:6789/0 conn(0x7fadec15cda0 legacy=0x7fadec0ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadeb7fe700 1 -- 172.21.15.7:0/1634230767 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec06bd60 con 0x7fadec06a520 2024-09-13T18:52:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadeb7fe700 1 --2- 172.21.15.7:0/1634230767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec06a520 0x7fadec06a930 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fade4009d90 tx=0x7fade400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db84d372ec9c69e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fade9ffb700 1 -- 172.21.15.7:0/1634230767 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade400e070 con 0x7fadec06a520 2024-09-13T18:52:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fade9ffb700 1 -- 172.21.15.7:0/1634230767 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade400bbd0 con 0x7fadec06a520 2024-09-13T18:52:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fade9ffb700 1 -- 172.21.15.7:0/1634230767 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4012430 con 0x7fadec06a520 2024-09-13T18:52:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadebfff700 1 -- 172.21.15.7:0/1634230767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec06a520 msgr2=0x7fadec06a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadebfff700 1 --2- 172.21.15.7:0/1634230767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec06a520 0x7fadec06a930 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fade4009d90 tx=0x7fade400b330 comp rx=0 tx=0).stop 2024-09-13T18:52:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadebfff700 1 -- 172.21.15.7:0/1634230767 shutdown_connections 2024-09-13T18:52:27.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadebfff700 1 --2- 172.21.15.7:0/1634230767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec06a520 0x7fadec06a930 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadebfff700 1 -- 172.21.15.7:0/1634230767 >> 172.21.15.7:0/1634230767 conn(0x7fadec0d2cb0 msgr2=0x7fadec0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:27.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadebfff700 1 -- 172.21.15.7:0/1634230767 shutdown_connections 2024-09-13T18:52:27.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadebfff700 1 -- 172.21.15.7:0/1634230767 wait complete. 2024-09-13T18:52:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadebfff700 1 Processor -- start 2024-09-13T18:52:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadebfff700 1 -- start start 2024-09-13T18:52:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadebfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fadec06a520 0x7fadec1efc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadebfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fadec06b900 0x7fadec1f3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec15cda0 0x7fadec0c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadebfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec0c6580 con 0x7fadec15cda0 2024-09-13T18:52:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadebfff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec0c8f00 con 0x7fadec06a520 2024-09-13T18:52:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadebfff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec1f3540 con 0x7fadec06b900 2024-09-13T18:52:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadeb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec15cda0 0x7fadec0c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadeb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec15cda0 0x7fadec0c9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49690/0 (socket says 172.21.15.7:49690) 2024-09-13T18:52:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.209+0000 7fadeb7fe700 1 -- 172.21.15.7:0/3616401237 learned_addr learned my addr 172.21.15.7:0/3616401237 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.213+0000 7fadea7fc700 1 --2- 172.21.15.7:0/3616401237 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fadec06b900 0x7fadec1f3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.213+0000 7fadeaffd700 1 --2- 172.21.15.7:0/3616401237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fadec06a520 0x7fadec1efc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.213+0000 7fadeb7fe700 1 -- 172.21.15.7:0/3616401237 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fadec06b900 msgr2=0x7fadec1f3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.213+0000 7fadeb7fe700 1 --2- 172.21.15.7:0/3616401237 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fadec06b900 0x7fadec1f3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.213+0000 7fadeb7fe700 1 -- 172.21.15.7:0/3616401237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fadec06a520 msgr2=0x7fadec1efc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.213+0000 7fadeb7fe700 1 --2- 172.21.15.7:0/3616401237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fadec06a520 0x7fadec1efc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.213+0000 7fadeb7fe700 1 -- 172.21.15.7:0/3616401237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade4009a70 con 0x7fadec15cda0 2024-09-13T18:52:27.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.213+0000 7fadeaffd700 1 --2- 172.21.15.7:0/3616401237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fadec06a520 0x7fadec1efc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:27.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.213+0000 7fadeb7fe700 1 --2- 172.21.15.7:0/3616401237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec15cda0 0x7fadec0c9390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fadec0c67b0 tx=0x7fade4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:27.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.213+0000 7fade3fff700 1 -- 172.21.15.7:0/3616401237 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade401b810 con 0x7fadec15cda0 2024-09-13T18:52:27.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.213+0000 7fade3fff700 1 -- 172.21.15.7:0/3616401237 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade4020070 con 0x7fadec15cda0 2024-09-13T18:52:27.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.213+0000 7fade3fff700 1 -- 172.21.15.7:0/3616401237 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade401bbf0 con 0x7fadec15cda0 2024-09-13T18:52:27.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.213+0000 7fadebfff700 1 -- 172.21.15.7:0/3616401237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadec0c82f0 con 0x7fadec15cda0 2024-09-13T18:52:27.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.213+0000 7fadebfff700 1 -- 172.21.15.7:0/3616401237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadec0c88b0 con 0x7fadec15cda0 2024-09-13T18:52:27.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.213+0000 7fadebfff700 1 -- 172.21.15.7:0/3616401237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadec0c90b0 con 0x7fadec15cda0 2024-09-13T18:52:27.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.217+0000 7fade3fff700 1 -- 172.21.15.7:0/3616401237 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fade401bd50 con 0x7fadec15cda0 2024-09-13T18:52:27.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.217+0000 7fade3fff700 1 --2- 172.21.15.7:0/3616401237 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fadcc066760 0x7fadcc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:27.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.217+0000 7fade3fff700 1 -- 172.21.15.7:0/3616401237 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fade4022070 con 0x7fadec15cda0 2024-09-13T18:52:27.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.217+0000 7fadeaffd700 1 --2- 172.21.15.7:0/3616401237 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fadcc066760 0x7fadcc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:27.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.217+0000 7fadeaffd700 1 --2- 172.21.15.7:0/3616401237 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fadcc066760 0x7fadcc068c20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faddc000f60 tx=0x7faddc007e70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:27.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.221+0000 7fade3fff700 1 -- 172.21.15.7:0/3616401237 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fade4057060 con 0x7fadec15cda0 2024-09-13T18:52:27.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.385+0000 7fadebfff700 1 -- 172.21.15.7:0/3616401237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fadec0d1120 con 0x7fadec15cda0 2024-09-13T18:52:27.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.385+0000 7fade3fff700 1 -- 172.21.15.7:0/3616401237 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fade4056bf0 con 0x7fadec15cda0 2024-09-13T18:52:27.393 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:52:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.385+0000 7fadebfff700 1 -- 172.21.15.7:0/3616401237 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fadcc066760 msgr2=0x7fadcc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.385+0000 7fadebfff700 1 --2- 172.21.15.7:0/3616401237 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fadcc066760 0x7fadcc068c20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faddc000f60 tx=0x7faddc007e70 comp rx=0 tx=0).stop 2024-09-13T18:52:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.385+0000 7fadebfff700 1 -- 172.21.15.7:0/3616401237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec15cda0 msgr2=0x7fadec0c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.385+0000 7fadebfff700 1 --2- 172.21.15.7:0/3616401237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec15cda0 0x7fadec0c9390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fadec0c67b0 tx=0x7fade4004af0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.389+0000 7fadebfff700 1 -- 172.21.15.7:0/3616401237 shutdown_connections 2024-09-13T18:52:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.389+0000 7fadebfff700 1 --2- 172.21.15.7:0/3616401237 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fadcc066760 0x7fadcc068c20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.389+0000 7fadebfff700 1 --2- 172.21.15.7:0/3616401237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadec15cda0 0x7fadec0c9390 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.389+0000 7fadebfff700 1 --2- 172.21.15.7:0/3616401237 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fadec06b900 0x7fadec1f3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.389+0000 7fadebfff700 1 --2- 172.21.15.7:0/3616401237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fadec06a520 0x7fadec1efc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.389+0000 7fadebfff700 1 -- 172.21.15.7:0/3616401237 >> 172.21.15.7:0/3616401237 conn(0x7fadec0d2cb0 msgr2=0x7fadec0ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.389+0000 7fadebfff700 1 -- 172.21.15.7:0/3616401237 shutdown_connections 2024-09-13T18:52:27.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.389+0000 7fadebfff700 1 -- 172.21.15.7:0/3616401237 wait complete. 2024-09-13T18:52:27.405 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:52:27.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b879db700 1 Processor -- start 2024-09-13T18:52:27.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b879db700 1 -- start start 2024-09-13T18:52:27.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b879db700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b8015cda0 0x7f4b800d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:27.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b879db700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b800b5e00 0x7f4b800b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:27.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b879db700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b800d0880 con 0x7f4b800b6780 2024-09-13T18:52:27.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b879db700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b800cde50 con 0x7f4b800b5e00 2024-09-13T18:52:27.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b879db700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b800d16e0 con 0x7f4b8015cda0 2024-09-13T18:52:27.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b84f76700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b800b5e00 0x7f4b800b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:27.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b84f76700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b800b5e00 0x7f4b800b6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49692/0 (socket says 172.21.15.7:49692) 2024-09-13T18:52:27.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b84f76700 1 -- 172.21.15.7:0/2670924189 learned_addr learned my addr 172.21.15.7:0/2670924189 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:27.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b85777700 1 --2- 172.21.15.7:0/2670924189 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b8015cda0 0x7f4b800d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:27.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b84f76700 1 -- 172.21.15.7:0/2670924189 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b8015cda0 msgr2=0x7f4b800d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:27.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b84f76700 1 --2- 172.21.15.7:0/2670924189 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b8015cda0 0x7f4b800d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b84f76700 1 -- 172.21.15.7:0/2670924189 >> v1:172.21.15.7:6789/0 conn(0x7f4b800b6780 legacy=0x7f4b8006a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b84f76700 1 -- 172.21.15.7:0/2670924189 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b800d0ad0 con 0x7f4b800b5e00 2024-09-13T18:52:27.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b84f76700 1 --2- 172.21.15.7:0/2670924189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b800b5e00 0x7f4b800b6240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009940 tx=0x7f4b70009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974df4a7a0bcd5a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:27.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b77fff700 1 -- 172.21.15.7:0/2670924189 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7000ba20 con 0x7f4b800b5e00 2024-09-13T18:52:27.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b77fff700 1 -- 172.21.15.7:0/2670924189 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b70009e90 con 0x7f4b800b5e00 2024-09-13T18:52:27.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b77fff700 1 -- 172.21.15.7:0/2670924189 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7000bda0 con 0x7f4b800b5e00 2024-09-13T18:52:27.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b879db700 1 -- 172.21.15.7:0/2670924189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b800b5e00 msgr2=0x7f4b800b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.513+0000 7f4b879db700 1 --2- 172.21.15.7:0/2670924189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b800b5e00 0x7f4b800b6240 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009940 tx=0x7f4b70009c50 comp rx=0 tx=0).stop 2024-09-13T18:52:27.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.517+0000 7f4b879db700 1 -- 172.21.15.7:0/2670924189 shutdown_connections 2024-09-13T18:52:27.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.517+0000 7f4b879db700 1 --2- 172.21.15.7:0/2670924189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b800b5e00 0x7f4b800b6240 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.517+0000 7f4b879db700 1 --2- 172.21.15.7:0/2670924189 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b8015cda0 0x7f4b800d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.517+0000 7f4b879db700 1 -- 172.21.15.7:0/2670924189 >> 172.21.15.7:0/2670924189 conn(0x7f4b800d2cb0 msgr2=0x7f4b800d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:27.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.517+0000 7f4b879db700 1 -- 172.21.15.7:0/2670924189 shutdown_connections 2024-09-13T18:52:27.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.517+0000 7f4b879db700 1 -- 172.21.15.7:0/2670924189 wait complete. 2024-09-13T18:52:27.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.517+0000 7f4b879db700 1 Processor -- start 2024-09-13T18:52:27.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.517+0000 7f4b879db700 1 -- start start 2024-09-13T18:52:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b879db700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b800b5e00 0x7f4b80156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b879db700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b800b6780 0x7f4b80153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b879db700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b8015cda0 0x7f4b800c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b879db700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b801567d0 con 0x7f4b800b5e00 2024-09-13T18:52:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b879db700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b80153f40 con 0x7f4b800b6780 2024-09-13T18:52:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b879db700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b801541e0 con 0x7f4b8015cda0 2024-09-13T18:52:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b85777700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b800b5e00 0x7f4b80156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b85777700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b800b5e00 0x7f4b80156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49696/0 (socket says 172.21.15.7:49696) 2024-09-13T18:52:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b85777700 1 -- 172.21.15.7:0/2311109627 learned_addr learned my addr 172.21.15.7:0/2311109627 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b84f76700 1 --2- 172.21.15.7:0/2311109627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b800b6780 0x7f4b80153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b85f78700 1 --2- 172.21.15.7:0/2311109627 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b8015cda0 0x7f4b800c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b85777700 1 -- 172.21.15.7:0/2311109627 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b8015cda0 msgr2=0x7f4b800c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b85777700 1 --2- 172.21.15.7:0/2311109627 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b8015cda0 0x7f4b800c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b85777700 1 -- 172.21.15.7:0/2311109627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b800b6780 msgr2=0x7f4b80153a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b85777700 1 --2- 172.21.15.7:0/2311109627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b800b6780 0x7f4b80153a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b85777700 1 -- 172.21.15.7:0/2311109627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b70009570 con 0x7f4b800b5e00 2024-09-13T18:52:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b85f78700 1 --2- 172.21.15.7:0/2311109627 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b8015cda0 0x7f4b800c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:27.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b85777700 1 --2- 172.21.15.7:0/2311109627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b800b5e00 0x7f4b80156290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00d800 tx=0x7f4b7c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:27.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b75ffb700 1 -- 172.21.15.7:0/2311109627 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0125f0 con 0x7f4b800b5e00 2024-09-13T18:52:27.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b75ffb700 1 -- 172.21.15.7:0/2311109627 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c012750 con 0x7f4b800b5e00 2024-09-13T18:52:27.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b879db700 1 -- 172.21.15.7:0/2311109627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b801544c0 con 0x7f4b800b5e00 2024-09-13T18:52:27.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b75ffb700 1 -- 172.21.15.7:0/2311109627 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c012ac0 con 0x7f4b800b5e00 2024-09-13T18:52:27.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.521+0000 7f4b879db700 1 -- 172.21.15.7:0/2311109627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b80154a80 con 0x7f4b800b5e00 2024-09-13T18:52:27.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.525+0000 7f4b879db700 1 -- 172.21.15.7:0/2311109627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b800b6240 con 0x7f4b800b5e00 2024-09-13T18:52:27.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.525+0000 7f4b75ffb700 1 -- 172.21.15.7:0/2311109627 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4b7c012c20 con 0x7f4b800b5e00 2024-09-13T18:52:27.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.525+0000 7f4b75ffb700 1 --2- 172.21.15.7:0/2311109627 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4b64066730 0x7f4b64068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:27.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.525+0000 7f4b75ffb700 1 -- 172.21.15.7:0/2311109627 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b7c0844f0 con 0x7f4b800b5e00 2024-09-13T18:52:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.525+0000 7f4b84f76700 1 --2- 172.21.15.7:0/2311109627 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4b64066730 0x7f4b64068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.525+0000 7f4b84f76700 1 --2- 172.21.15.7:0/2311109627 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4b64066730 0x7f4b64068bf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009940 tx=0x7f4b70008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:27.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.533+0000 7f4b75ffb700 1 -- 172.21.15.7:0/2311109627 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4b7c051860 con 0x7f4b800b5e00 2024-09-13T18:52:27.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.705+0000 7f4b879db700 1 -- 172.21.15.7:0/2311109627 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4b8006b040 con 0x7f4b800b5e00 2024-09-13T18:52:27.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.705+0000 7f4b75ffb700 1 -- 172.21.15.7:0/2311109627 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4b7c012ed0 con 0x7f4b800b5e00 2024-09-13T18:52:27.711 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:52:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.709+0000 7f4b879db700 1 -- 172.21.15.7:0/2311109627 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4b64066730 msgr2=0x7f4b64068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.709+0000 7f4b879db700 1 --2- 172.21.15.7:0/2311109627 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4b64066730 0x7f4b64068bf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4b70009940 tx=0x7f4b70008030 comp rx=0 tx=0).stop 2024-09-13T18:52:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.709+0000 7f4b879db700 1 -- 172.21.15.7:0/2311109627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b800b5e00 msgr2=0x7f4b80156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.709+0000 7f4b879db700 1 --2- 172.21.15.7:0/2311109627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b800b5e00 0x7f4b80156290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00d800 tx=0x7f4b7c00dbc0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.709+0000 7f4b879db700 1 -- 172.21.15.7:0/2311109627 shutdown_connections 2024-09-13T18:52:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.709+0000 7f4b879db700 1 --2- 172.21.15.7:0/2311109627 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4b64066730 0x7f4b64068bf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.709+0000 7f4b879db700 1 --2- 172.21.15.7:0/2311109627 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b8015cda0 0x7f4b800c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.709+0000 7f4b879db700 1 --2- 172.21.15.7:0/2311109627 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b800b6780 0x7f4b80153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.709+0000 7f4b879db700 1 --2- 172.21.15.7:0/2311109627 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b800b5e00 0x7f4b80156290 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.709+0000 7f4b879db700 1 -- 172.21.15.7:0/2311109627 >> 172.21.15.7:0/2311109627 conn(0x7f4b800d2cb0 msgr2=0x7f4b800d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.709+0000 7f4b879db700 1 -- 172.21.15.7:0/2311109627 shutdown_connections 2024-09-13T18:52:27.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.709+0000 7f4b879db700 1 -- 172.21.15.7:0/2311109627 wait complete. 2024-09-13T18:52:27.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:52:27.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e859e2700 1 Processor -- start 2024-09-13T18:52:27.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e859e2700 1 -- start start 2024-09-13T18:52:27.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e859e2700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e8015cda0 0x7f0e8006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:27.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e859e2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e800ceb70 0x7f0e800b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e859e2700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f0e8006a520 0x7f0e8006a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e859e2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e8006be40 con 0x7f0e800ceb70 2024-09-13T18:52:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e859e2700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e8006aeb0 con 0x7f0e8006a520 2024-09-13T18:52:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e859e2700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800b8430 con 0x7f0e8015cda0 2024-09-13T18:52:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e7ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e800ceb70 0x7f0e800b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e7ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e800ceb70 0x7f0e800b5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49698/0 (socket says 172.21.15.7:49698) 2024-09-13T18:52:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e7ffff700 1 -- 172.21.15.7:0/1164767071 learned_addr learned my addr 172.21.15.7:0/1164767071 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e849e0700 1 --2- 172.21.15.7:0/1164767071 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e8015cda0 0x7f0e8006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e851e1700 1 --2- 172.21.15.7:0/1164767071 >> v2:172.21.15.171:3300/0 conn(0x7f0e8006a520 0x7f0e8006a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e7ffff700 1 -- 172.21.15.7:0/1164767071 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e8015cda0 msgr2=0x7f0e8006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e7ffff700 1 --2- 172.21.15.7:0/1164767071 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e8015cda0 0x7f0e8006b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e7ffff700 1 -- 172.21.15.7:0/1164767071 >> v2:172.21.15.171:3300/0 conn(0x7f0e8006a520 msgr2=0x7f0e8006a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e7ffff700 1 --2- 172.21.15.7:0/1164767071 >> v2:172.21.15.171:3300/0 conn(0x7f0e8006a520 0x7f0e8006a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e7ffff700 1 -- 172.21.15.7:0/1164767071 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e800cf350 con 0x7f0e800ceb70 2024-09-13T18:52:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e849e0700 1 --2- 172.21.15.7:0/1164767071 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e8015cda0 0x7f0e8006b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e851e1700 1 --2- 172.21.15.7:0/1164767071 >> v2:172.21.15.171:3300/0 conn(0x7f0e8006a520 0x7f0e8006a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e7ffff700 1 --2- 172.21.15.7:0/1164767071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e800ceb70 0x7f0e800b5de0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0e801617d0 tx=0x7f0e70009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6555021dcad580c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e7f7fe700 1 -- 172.21.15.7:0/1164767071 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7000b8a0 con 0x7f0e800ceb70 2024-09-13T18:52:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e7f7fe700 1 -- 172.21.15.7:0/1164767071 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7000ba00 con 0x7f0e800ceb70 2024-09-13T18:52:27.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e7f7fe700 1 -- 172.21.15.7:0/1164767071 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7000bd90 con 0x7f0e800ceb70 2024-09-13T18:52:27.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e859e2700 1 -- 172.21.15.7:0/1164767071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e800ceb70 msgr2=0x7f0e800b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e859e2700 1 --2- 172.21.15.7:0/1164767071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e800ceb70 0x7f0e800b5de0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0e801617d0 tx=0x7f0e70009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e859e2700 1 -- 172.21.15.7:0/1164767071 shutdown_connections 2024-09-13T18:52:27.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e859e2700 1 --2- 172.21.15.7:0/1164767071 >> v2:172.21.15.171:3300/0 conn(0x7f0e8006a520 0x7f0e8006a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e859e2700 1 --2- 172.21.15.7:0/1164767071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e800ceb70 0x7f0e800b5de0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e859e2700 1 --2- 172.21.15.7:0/1164767071 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e8015cda0 0x7f0e8006b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e859e2700 1 -- 172.21.15.7:0/1164767071 >> 172.21.15.7:0/1164767071 conn(0x7f0e800d2cb0 msgr2=0x7f0e800d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:27.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e859e2700 1 -- 172.21.15.7:0/1164767071 shutdown_connections 2024-09-13T18:52:27.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.841+0000 7f0e859e2700 1 -- 172.21.15.7:0/1164767071 wait complete. 2024-09-13T18:52:27.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e859e2700 1 Processor -- start 2024-09-13T18:52:27.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e859e2700 1 -- start start 2024-09-13T18:52:27.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e859e2700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e8006a520 0x7f0e8015a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:27.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e859e2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e800ceb70 0x7f0e8006aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:27.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e859e2700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e8015cda0 0x7f0e8006b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:27.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e859e2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800c7270 con 0x7f0e800ceb70 2024-09-13T18:52:27.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e859e2700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800cc7b0 con 0x7f0e8015cda0 2024-09-13T18:52:27.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e859e2700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800c6910 con 0x7f0e8006a520 2024-09-13T18:52:27.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e7ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e800ceb70 0x7f0e8006aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:27.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e7ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e800ceb70 0x7f0e8006aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49702/0 (socket says 172.21.15.7:49702) 2024-09-13T18:52:27.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e7ffff700 1 -- 172.21.15.7:0/1451447458 learned_addr learned my addr 172.21.15.7:0/1451447458 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:27.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e849e0700 1 --2- 172.21.15.7:0/1451447458 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e8006a520 0x7f0e8015a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:27.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e851e1700 1 --2- 172.21.15.7:0/1451447458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e8015cda0 0x7f0e8006b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:27.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e7ffff700 1 -- 172.21.15.7:0/1451447458 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e8006a520 msgr2=0x7f0e8015a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e7ffff700 1 --2- 172.21.15.7:0/1451447458 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e8006a520 0x7f0e8015a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e7ffff700 1 -- 172.21.15.7:0/1451447458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e8015cda0 msgr2=0x7f0e8006b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:27.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e7ffff700 1 --2- 172.21.15.7:0/1451447458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e8015cda0 0x7f0e8006b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:27.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e7ffff700 1 -- 172.21.15.7:0/1451447458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e70009570 con 0x7f0e800ceb70 2024-09-13T18:52:27.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.845+0000 7f0e849e0700 1 --2- 172.21.15.7:0/1451447458 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e8006a520 0x7f0e8015a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:27.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.849+0000 7f0e7ffff700 1 --2- 172.21.15.7:0/1451447458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e800ceb70 0x7f0e8006aff0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0e70013000 tx=0x7f0e70004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:27.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.849+0000 7f0e7d7fa700 1 -- 172.21.15.7:0/1451447458 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e700038c0 con 0x7f0e800ceb70 2024-09-13T18:52:27.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.849+0000 7f0e7d7fa700 1 -- 172.21.15.7:0/1451447458 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7001f070 con 0x7f0e800ceb70 2024-09-13T18:52:27.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.849+0000 7f0e859e2700 1 -- 172.21.15.7:0/1451447458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e800c6bb0 con 0x7f0e800ceb70 2024-09-13T18:52:27.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.849+0000 7f0e7d7fa700 1 -- 172.21.15.7:0/1451447458 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70003c80 con 0x7f0e800ceb70 2024-09-13T18:52:27.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.849+0000 7f0e859e2700 1 -- 172.21.15.7:0/1451447458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8015a920 con 0x7f0e800ceb70 2024-09-13T18:52:27.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.849+0000 7f0e859e2700 1 -- 172.21.15.7:0/1451447458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e800c73b0 con 0x7f0e800ceb70 2024-09-13T18:52:27.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.849+0000 7f0e7d7fa700 1 -- 172.21.15.7:0/1451447458 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0e7001a430 con 0x7f0e800ceb70 2024-09-13T18:52:27.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.849+0000 7f0e7d7fa700 1 --2- 172.21.15.7:0/1451447458 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0e60066730 0x7f0e60068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:27.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.849+0000 7f0e7d7fa700 1 -- 172.21.15.7:0/1451447458 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e70021070 con 0x7f0e800ceb70 2024-09-13T18:52:27.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.849+0000 7f0e849e0700 1 --2- 172.21.15.7:0/1451447458 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0e60066730 0x7f0e60068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:27.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.853+0000 7f0e849e0700 1 --2- 172.21.15.7:0/1451447458 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0e60066730 0x7f0e60068bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0e68006fd0 tx=0x7f0e68008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:27.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:27.857+0000 7f0e7d7fa700 1 -- 172.21.15.7:0/1451447458 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0e800c73b0 con 0x7f0e800ceb70 2024-09-13T18:52:28.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.029+0000 7f0e859e2700 1 -- 172.21.15.7:0/1451447458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0e800c73b0 con 0x7f0e800ceb70 2024-09-13T18:52:28.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.029+0000 7f0e7d7fa700 1 -- 172.21.15.7:0/1451447458 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0e70015020 con 0x7f0e800ceb70 2024-09-13T18:52:28.035 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:52:28.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.029+0000 7f0e859e2700 1 -- 172.21.15.7:0/1451447458 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0e60066730 msgr2=0x7f0e60068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:28.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.029+0000 7f0e859e2700 1 --2- 172.21.15.7:0/1451447458 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0e60066730 0x7f0e60068bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0e68006fd0 tx=0x7f0e68008040 comp rx=0 tx=0).stop 2024-09-13T18:52:28.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.029+0000 7f0e859e2700 1 -- 172.21.15.7:0/1451447458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e800ceb70 msgr2=0x7f0e8006aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:28.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.029+0000 7f0e859e2700 1 --2- 172.21.15.7:0/1451447458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e800ceb70 0x7f0e8006aff0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0e70013000 tx=0x7f0e70004770 comp rx=0 tx=0).stop 2024-09-13T18:52:28.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.029+0000 7f0e859e2700 1 -- 172.21.15.7:0/1451447458 shutdown_connections 2024-09-13T18:52:28.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.029+0000 7f0e859e2700 1 --2- 172.21.15.7:0/1451447458 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0e60066730 0x7f0e60068bf0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.029+0000 7f0e859e2700 1 --2- 172.21.15.7:0/1451447458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e8015cda0 0x7f0e8006b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.029+0000 7f0e859e2700 1 --2- 172.21.15.7:0/1451447458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e800ceb70 0x7f0e8006aff0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.029+0000 7f0e859e2700 1 --2- 172.21.15.7:0/1451447458 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0e8006a520 0x7f0e8015a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.029+0000 7f0e859e2700 1 -- 172.21.15.7:0/1451447458 >> 172.21.15.7:0/1451447458 conn(0x7f0e800d2cb0 msgr2=0x7f0e800b5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:28.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.033+0000 7f0e859e2700 1 -- 172.21.15.7:0/1451447458 shutdown_connections 2024-09-13T18:52:28.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.033+0000 7f0e859e2700 1 -- 172.21.15.7:0/1451447458 wait complete. 2024-09-13T18:52:28.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:28.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f5365dc2700 1 Processor -- start 2024-09-13T18:52:28.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f5365dc2700 1 -- start start 2024-09-13T18:52:28.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f5365dc2700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f536015a7e0 0x7f53600cd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:28.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f5365dc2700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f536015b960 0x7f53600b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:28.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f5365dc2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f53600b2d30 0x7f536015d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:28.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f5365dc2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53600b3190 con 0x7f53600b2d30 2024-09-13T18:52:28.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f5365dc2700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f536015c4f0 con 0x7f536015a7e0 2024-09-13T18:52:28.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f5365dc2700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536015c7d0 con 0x7f536015b960 2024-09-13T18:52:28.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f53655c1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f53600b2d30 0x7f536015d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:28.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f53655c1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f53600b2d30 0x7f536015d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49710/0 (socket says 172.21.15.7:49710) 2024-09-13T18:52:28.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f53655c1700 1 -- 172.21.15.7:0/4290835162 learned_addr learned my addr 172.21.15.7:0/4290835162 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:28.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f535ffff700 1 --2- 172.21.15.7:0/4290835162 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f536015b960 0x7f53600b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:28.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f53655c1700 1 -- 172.21.15.7:0/4290835162 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f536015b960 msgr2=0x7f53600b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:28.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f5364dc0700 1 --2- 172.21.15.7:0/4290835162 >> v2:172.21.15.171:3300/0 conn(0x7f536015a7e0 0x7f53600cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:28.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f53655c1700 1 --2- 172.21.15.7:0/4290835162 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f536015b960 0x7f53600b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f53655c1700 1 -- 172.21.15.7:0/4290835162 >> v2:172.21.15.171:3300/0 conn(0x7f536015a7e0 msgr2=0x7f53600cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:28.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f53655c1700 1 --2- 172.21.15.7:0/4290835162 >> v2:172.21.15.171:3300/0 conn(0x7f536015a7e0 0x7f53600cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f53655c1700 1 -- 172.21.15.7:0/4290835162 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536015cab0 con 0x7f53600b2d30 2024-09-13T18:52:28.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f5364dc0700 1 --2- 172.21.15.7:0/4290835162 >> v2:172.21.15.171:3300/0 conn(0x7f536015a7e0 0x7f53600cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f535ffff700 1 --2- 172.21.15.7:0/4290835162 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f536015b960 0x7f53600b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f53655c1700 1 --2- 172.21.15.7:0/4290835162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53600b2d30 0x7f536015d590 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f53600b6ca0 tx=0x7f535800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d640298443ce3a0f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f535f7fe700 1 -- 172.21.15.7:0/4290835162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535800e070 con 0x7f53600b2d30 2024-09-13T18:52:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f535f7fe700 1 -- 172.21.15.7:0/4290835162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535800b9c0 con 0x7f53600b2d30 2024-09-13T18:52:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f535f7fe700 1 -- 172.21.15.7:0/4290835162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535800bd70 con 0x7f53600b2d30 2024-09-13T18:52:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f5365dc2700 1 -- 172.21.15.7:0/4290835162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53600b2d30 msgr2=0x7f536015d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f5365dc2700 1 --2- 172.21.15.7:0/4290835162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53600b2d30 0x7f536015d590 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f53600b6ca0 tx=0x7f535800b330 comp rx=0 tx=0).stop 2024-09-13T18:52:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f5365dc2700 1 -- 172.21.15.7:0/4290835162 shutdown_connections 2024-09-13T18:52:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f5365dc2700 1 --2- 172.21.15.7:0/4290835162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53600b2d30 0x7f536015d590 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f5365dc2700 1 --2- 172.21.15.7:0/4290835162 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f536015b960 0x7f53600b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f5365dc2700 1 --2- 172.21.15.7:0/4290835162 >> v2:172.21.15.171:3300/0 conn(0x7f536015a7e0 0x7f53600cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f5365dc2700 1 -- 172.21.15.7:0/4290835162 >> 172.21.15.7:0/4290835162 conn(0x7f536006ffb0 msgr2=0x7f5360071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.157+0000 7f5365dc2700 1 -- 172.21.15.7:0/4290835162 shutdown_connections 2024-09-13T18:52:28.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f5365dc2700 1 -- 172.21.15.7:0/4290835162 wait complete. 2024-09-13T18:52:28.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f5365dc2700 1 Processor -- start 2024-09-13T18:52:28.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f5365dc2700 1 -- start start 2024-09-13T18:52:28.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f5365dc2700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f53600b2d30 0x7f536014ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:28.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f5365dc2700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f536015a7e0 0x7f53600b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:28.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f5365dc2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f536015b960 0x7f53600b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:28.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f5365dc2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53600ce1c0 con 0x7f536015b960 2024-09-13T18:52:28.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f5365dc2700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53600b6a80 con 0x7f536015a7e0 2024-09-13T18:52:28.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f5365dc2700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5360149c80 con 0x7f53600b2d30 2024-09-13T18:52:28.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f53655c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f536015b960 0x7f53600b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:28.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f53655c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f536015b960 0x7f53600b4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49712/0 (socket says 172.21.15.7:49712) 2024-09-13T18:52:28.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f53655c1700 1 -- 172.21.15.7:0/3869704616 learned_addr learned my addr 172.21.15.7:0/3869704616 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:28.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f535ffff700 1 --2- 172.21.15.7:0/3869704616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f536015a7e0 0x7f53600b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:28.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f5364dc0700 1 --2- 172.21.15.7:0/3869704616 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f53600b2d30 0x7f536014ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:28.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f53655c1700 1 -- 172.21.15.7:0/3869704616 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f53600b2d30 msgr2=0x7f536014ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:28.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f53655c1700 1 --2- 172.21.15.7:0/3869704616 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f53600b2d30 0x7f536014ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f53655c1700 1 -- 172.21.15.7:0/3869704616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f536015a7e0 msgr2=0x7f53600b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:28.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f53655c1700 1 --2- 172.21.15.7:0/3869704616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f536015a7e0 0x7f53600b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f53655c1700 1 -- 172.21.15.7:0/3869704616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5358009b30 con 0x7f536015b960 2024-09-13T18:52:28.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f535ffff700 1 --2- 172.21.15.7:0/3869704616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f536015a7e0 0x7f53600b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:28.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f53655c1700 1 --2- 172.21.15.7:0/3869704616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f536015b960 0x7f53600b4540 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f535800b370 tx=0x7f5358004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:28.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f535d7fa700 1 -- 172.21.15.7:0/3869704616 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53580092e0 con 0x7f536015b960 2024-09-13T18:52:28.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f535d7fa700 1 -- 172.21.15.7:0/3869704616 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535801c440 con 0x7f536015b960 2024-09-13T18:52:28.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f5365dc2700 1 -- 172.21.15.7:0/3869704616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5360149f60 con 0x7f536015b960 2024-09-13T18:52:28.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f535d7fa700 1 -- 172.21.15.7:0/3869704616 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535801c7f0 con 0x7f536015b960 2024-09-13T18:52:28.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.161+0000 7f5365dc2700 1 -- 172.21.15.7:0/3869704616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536014a520 con 0x7f536015b960 2024-09-13T18:52:28.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.165+0000 7f535d7fa700 1 -- 172.21.15.7:0/3869704616 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f535801c950 con 0x7f536015b960 2024-09-13T18:52:28.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.165+0000 7f5365dc2700 1 -- 172.21.15.7:0/3869704616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536015abc0 con 0x7f536015b960 2024-09-13T18:52:28.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.165+0000 7f535d7fa700 1 --2- 172.21.15.7:0/3869704616 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5340066730 0x7f5340068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:28.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.165+0000 7f535d7fa700 1 -- 172.21.15.7:0/3869704616 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5358085be0 con 0x7f536015b960 2024-09-13T18:52:28.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.165+0000 7f5364dc0700 1 --2- 172.21.15.7:0/3869704616 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5340066730 0x7f5340068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:28.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.169+0000 7f5364dc0700 1 --2- 172.21.15.7:0/3869704616 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5340066730 0x7f5340068bf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5348006fd0 tx=0x7f5348008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:28.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.173+0000 7f535d7fa700 1 -- 172.21.15.7:0/3869704616 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f53580530a0 con 0x7f536015b960 2024-09-13T18:52:28.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.381+0000 7f5365dc2700 1 -- 172.21.15.7:0/3869704616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5360156230 con 0x7f536015b960 2024-09-13T18:52:28.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.381+0000 7f535d7fa700 1 -- 172.21.15.7:0/3869704616 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4725 (secure 0 0 0) 0x7f53580566c0 con 0x7f536015b960 2024-09-13T18:52:28.386 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:28.386 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1309419125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1309419125},{"type":"v1","addr":"172.21.15.7:6835","nonce":1309419125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/951828843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":951828843},{"type":"v1","addr":"172.21.15.7:6833","nonce":951828843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4226627580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4226627580},{"type":"v1","addr":"172.21.15.171:6835","nonce":4226627580}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:52:17.908937+0000","modified":"2024-09-13T18:52:20.930515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6837/3658296803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3658296803},{"type":"v1","addr":"172.21.15.171:6837","nonce":3658296803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.381+0000 7f5365dc2700 1 -- 172.21.15.7:0/3869704616 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5340066730 msgr2=0x7f5340068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.381+0000 7f5365dc2700 1 --2- 172.21.15.7:0/3869704616 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5340066730 0x7f5340068bf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5348006fd0 tx=0x7f5348008040 comp rx=0 tx=0).stop 2024-09-13T18:52:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.381+0000 7f5365dc2700 1 -- 172.21.15.7:0/3869704616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f536015b960 msgr2=0x7f53600b4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.381+0000 7f5365dc2700 1 --2- 172.21.15.7:0/3869704616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f536015b960 0x7f53600b4540 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f535800b370 tx=0x7f5358004af0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.381+0000 7f5365dc2700 1 -- 172.21.15.7:0/3869704616 shutdown_connections 2024-09-13T18:52:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.381+0000 7f5365dc2700 1 --2- 172.21.15.7:0/3869704616 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5340066730 0x7f5340068bf0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.381+0000 7f5365dc2700 1 --2- 172.21.15.7:0/3869704616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f536015b960 0x7f53600b4540 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.381+0000 7f5365dc2700 1 --2- 172.21.15.7:0/3869704616 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f536015a7e0 0x7f53600b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.381+0000 7f5365dc2700 1 --2- 172.21.15.7:0/3869704616 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f53600b2d30 0x7f536014ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.381+0000 7f5365dc2700 1 -- 172.21.15.7:0/3869704616 >> 172.21.15.7:0/3869704616 conn(0x7f536006ffb0 msgr2=0x7f536006d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:28.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.385+0000 7f5365dc2700 1 -- 172.21.15.7:0/3869704616 shutdown_connections 2024-09-13T18:52:28.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.385+0000 7f5365dc2700 1 -- 172.21.15.7:0/3869704616 wait complete. 2024-09-13T18:52:28.390 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-13T18:52:28.403 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:28.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd10cf82700 1 Processor -- start 2024-09-13T18:52:28.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd10cf82700 1 -- start start 2024-09-13T18:52:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd10cf82700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1080cdcc0 0x7fd1080b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd10cf82700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd1080b2930 0x7fd10806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd10cf82700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1080b2dc0 con 0x7fd10815b890 2024-09-13T18:52:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd10cf82700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd10815bc70 con 0x7fd1080b2930 2024-09-13T18:52:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd10cf82700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1080ccc20 con 0x7fd1080cdcc0 2024-09-13T18:52:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd10659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd10815b890 0x7fd108162220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35916/0 (socket says 172.21.15.7:35916) 2024-09-13T18:52:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd10659c700 1 -- 172.21.15.7:0/1550890463 learned_addr learned my addr 172.21.15.7:0/1550890463 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd106d9d700 1 --2- 172.21.15.7:0/1550890463 >> v2:172.21.15.7:3300/0 conn(0x7fd1080b2930 0x7fd10806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd105d9b700 1 --2- 172.21.15.7:0/1550890463 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1080cdcc0 0x7fd1080b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:28.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd10559a700 1 -- 172.21.15.7:0/1550890463 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 579842812 0 0) 0x7fd0f0001110 con 0x7fd10815b890 2024-09-13T18:52:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd10559a700 1 -- 172.21.15.7:0/1550890463 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0f40035d0 con 0x7fd10815b890 2024-09-13T18:52:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd106d9d700 1 -- 172.21.15.7:0/1550890463 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1080cdcc0 msgr2=0x7fd1080b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:28.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd106d9d700 1 --2- 172.21.15.7:0/1550890463 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1080cdcc0 0x7fd1080b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd106d9d700 1 -- 172.21.15.7:0/1550890463 >> v1:172.21.15.7:6789/0 conn(0x7fd10815b890 legacy=0x7fd108162220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd106d9d700 1 -- 172.21.15.7:0/1550890463 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1080ccf00 con 0x7fd1080b2930 2024-09-13T18:52:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd106d9d700 1 --2- 172.21.15.7:0/1550890463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1080b2930 0x7fd10806ffe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009850 tx=0x7fd0f8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1434afb54d250158 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd10559a700 1 -- 172.21.15.7:0/1550890463 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8011830 con 0x7fd1080b2930 2024-09-13T18:52:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd10559a700 1 -- 172.21.15.7:0/1550890463 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f8011990 con 0x7fd1080b2930 2024-09-13T18:52:28.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd10559a700 1 -- 172.21.15.7:0/1550890463 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8011d40 con 0x7fd1080b2930 2024-09-13T18:52:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd10cf82700 1 -- 172.21.15.7:0/1550890463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1080b2930 msgr2=0x7fd10806ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.513+0000 7fd10cf82700 1 --2- 172.21.15.7:0/1550890463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1080b2930 0x7fd10806ffe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009850 tx=0x7fd0f8009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.517+0000 7fd10cf82700 1 -- 172.21.15.7:0/1550890463 shutdown_connections 2024-09-13T18:52:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.517+0000 7fd10cf82700 1 --2- 172.21.15.7:0/1550890463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1080b2930 0x7fd10806ffe0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.517+0000 7fd10cf82700 1 --2- 172.21.15.7:0/1550890463 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1080cdcc0 0x7fd1080b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.517+0000 7fd10cf82700 1 -- 172.21.15.7:0/1550890463 >> 172.21.15.7:0/1550890463 conn(0x7fd1080ce980 msgr2=0x7fd1080ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.517+0000 7fd10cf82700 1 -- 172.21.15.7:0/1550890463 shutdown_connections 2024-09-13T18:52:28.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.517+0000 7fd10cf82700 1 -- 172.21.15.7:0/1550890463 wait complete. 2024-09-13T18:52:28.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.517+0000 7fd10cf82700 1 Processor -- start 2024-09-13T18:52:28.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.517+0000 7fd10cf82700 1 -- start start 2024-09-13T18:52:28.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd10cf82700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1080b2930 0x7fd1081f7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:28.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd10cf82700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1080cdcc0 0x7fd1081581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:28.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd10cf82700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10815b890 0x7fd108154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:28.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd10cf82700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1081559a0 con 0x7fd10815b890 2024-09-13T18:52:28.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd10cf82700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd108155c40 con 0x7fd1080b2930 2024-09-13T18:52:28.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd10cf82700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd108155f20 con 0x7fd1080cdcc0 2024-09-13T18:52:28.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd106d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10815b890 0x7fd108154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:28.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd106d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10815b890 0x7fd108154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49736/0 (socket says 172.21.15.7:49736) 2024-09-13T18:52:28.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd106d9d700 1 -- 172.21.15.7:0/3115327310 learned_addr learned my addr 172.21.15.7:0/3115327310 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:28.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd10659c700 1 --2- 172.21.15.7:0/3115327310 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1080b2930 0x7fd1081f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:28.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd105d9b700 1 --2- 172.21.15.7:0/3115327310 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1080cdcc0 0x7fd1081581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:28.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd106d9d700 1 -- 172.21.15.7:0/3115327310 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1080cdcc0 msgr2=0x7fd1081581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:28.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd106d9d700 1 --2- 172.21.15.7:0/3115327310 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1080cdcc0 0x7fd1081581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd106d9d700 1 -- 172.21.15.7:0/3115327310 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1080b2930 msgr2=0x7fd1081f7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:28.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd106d9d700 1 --2- 172.21.15.7:0/3115327310 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1080b2930 0x7fd1081f7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd106d9d700 1 -- 172.21.15.7:0/3115327310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f80094b0 con 0x7fd10815b890 2024-09-13T18:52:28.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd106d9d700 1 --2- 172.21.15.7:0/3115327310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10815b890 0x7fd108154ad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd1080cc560 tx=0x7fd0f8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:28.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd0feffd700 1 -- 172.21.15.7:0/3115327310 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8004240 con 0x7fd10815b890 2024-09-13T18:52:28.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd0feffd700 1 -- 172.21.15.7:0/3115327310 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f801f070 con 0x7fd10815b890 2024-09-13T18:52:28.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd10cf82700 1 -- 172.21.15.7:0/3115327310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd108156200 con 0x7fd10815b890 2024-09-13T18:52:28.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd0feffd700 1 -- 172.21.15.7:0/3115327310 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8004600 con 0x7fd10815b890 2024-09-13T18:52:28.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.521+0000 7fd10cf82700 1 -- 172.21.15.7:0/3115327310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd108155010 con 0x7fd10815b890 2024-09-13T18:52:28.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.525+0000 7fd10cf82700 1 -- 172.21.15.7:0/3115327310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd108156720 con 0x7fd10815b890 2024-09-13T18:52:28.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.525+0000 7fd0feffd700 1 -- 172.21.15.7:0/3115327310 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd0f8021030 con 0x7fd10815b890 2024-09-13T18:52:28.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.525+0000 7fd0feffd700 1 --2- 172.21.15.7:0/3115327310 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd0e8066730 0x7fd0e8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:28.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.525+0000 7fd0feffd700 1 -- 172.21.15.7:0/3115327310 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0f80853e0 con 0x7fd10815b890 2024-09-13T18:52:28.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.525+0000 7fd10659c700 1 --2- 172.21.15.7:0/3115327310 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd0e8066730 0x7fd0e8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:28.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.525+0000 7fd10659c700 1 --2- 172.21.15.7:0/3115327310 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd0e8066730 0x7fd0e8068bf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0007fd0 tx=0x7fd0f0007c60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:28.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.533+0000 7fd0feffd700 1 -- 172.21.15.7:0/3115327310 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd108156720 con 0x7fd10815b890 2024-09-13T18:52:28.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.705+0000 7fd10cf82700 1 -- 172.21.15.7:0/3115327310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd108156720 con 0x7fd10815b890 2024-09-13T18:52:28.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.709+0000 7fd0feffd700 1 -- 172.21.15.7:0/3115327310 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7fd108156720 con 0x7fd10815b890 2024-09-13T18:52:28.715 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:28.715 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:28.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.709+0000 7fd10cf82700 1 -- 172.21.15.7:0/3115327310 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd0e8066730 msgr2=0x7fd0e8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:28.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.709+0000 7fd10cf82700 1 --2- 172.21.15.7:0/3115327310 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd0e8066730 0x7fd0e8068bf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0007fd0 tx=0x7fd0f0007c60 comp rx=0 tx=0).stop 2024-09-13T18:52:28.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.709+0000 7fd10cf82700 1 -- 172.21.15.7:0/3115327310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10815b890 msgr2=0x7fd108154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:28.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.709+0000 7fd10cf82700 1 --2- 172.21.15.7:0/3115327310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10815b890 0x7fd108154ad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd1080cc560 tx=0x7fd0f8004770 comp rx=0 tx=0).stop 2024-09-13T18:52:28.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.713+0000 7fd10cf82700 1 -- 172.21.15.7:0/3115327310 shutdown_connections 2024-09-13T18:52:28.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.713+0000 7fd10cf82700 1 --2- 172.21.15.7:0/3115327310 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd0e8066730 0x7fd0e8068bf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.713+0000 7fd10cf82700 1 --2- 172.21.15.7:0/3115327310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd10815b890 0x7fd108154ad0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.713+0000 7fd10cf82700 1 --2- 172.21.15.7:0/3115327310 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1080cdcc0 0x7fd1081581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.713+0000 7fd10cf82700 1 --2- 172.21.15.7:0/3115327310 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1080b2930 0x7fd1081f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.713+0000 7fd10cf82700 1 -- 172.21.15.7:0/3115327310 >> 172.21.15.7:0/3115327310 conn(0x7fd1080ce980 msgr2=0x7fd1080aeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:28.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.713+0000 7fd10cf82700 1 -- 172.21.15.7:0/3115327310 shutdown_connections 2024-09-13T18:52:28.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.713+0000 7fd10cf82700 1 -- 172.21.15.7:0/3115327310 wait complete. 2024-09-13T18:52:28.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:52:15.554683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:52:16.896879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:52:28.731 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:52:28.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.849+0000 7fc04ffff700 1 Processor -- start 2024-09-13T18:52:28.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.849+0000 7fc04ffff700 1 -- start start 2024-09-13T18:52:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.849+0000 7fc04ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc0500b3b30 0x7fc0500b0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.849+0000 7fc04ffff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc05006b7b0 con 0x7fc0500b36c0 2024-09-13T18:52:28.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.849+0000 7fc04ffff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc05006aba0 con 0x7fc05006d070 2024-09-13T18:52:28.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.849+0000 7fc04ffff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc05006ae80 con 0x7fc0500b3b30 2024-09-13T18:52:28.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.849+0000 7fc04e7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc0500b36c0 0x7fc0500b0110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35924/0 (socket says 172.21.15.7:35924) 2024-09-13T18:52:28.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.849+0000 7fc04f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc0500b3b30 0x7fc0500b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:28.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.849+0000 7fc04e7fc700 1 -- 172.21.15.7:0/3824996831 learned_addr learned my addr 172.21.15.7:0/3824996831 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:28.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.849+0000 7fc04f7fe700 1 -- 172.21.15.7:0/3824996831 >> v1:172.21.15.171:6789/0 conn(0x7fc05006d070 legacy=0x7fc05006d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:28.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.849+0000 7fc04f7fe700 1 -- 172.21.15.7:0/3824996831 >> v1:172.21.15.7:6789/0 conn(0x7fc0500b36c0 legacy=0x7fc0500b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:28.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.849+0000 7fc04f7fe700 1 -- 172.21.15.7:0/3824996831 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05006b160 con 0x7fc0500b3b30 2024-09-13T18:52:28.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.849+0000 7fc04f7fe700 1 --2- 172.21.15.7:0/3824996831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0500b3b30 0x7fc0500b0820 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc048009850 tx=0x7fc048009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16376906c6f9e8a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:28.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.849+0000 7fc04dffb700 1 -- 172.21.15.7:0/3824996831 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048011830 con 0x7fc0500b3b30 2024-09-13T18:52:28.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.849+0000 7fc04dffb700 1 -- 172.21.15.7:0/3824996831 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc048011990 con 0x7fc0500b3b30 2024-09-13T18:52:28.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.849+0000 7fc04dffb700 1 -- 172.21.15.7:0/3824996831 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048011d20 con 0x7fc0500b3b30 2024-09-13T18:52:28.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.853+0000 7fc04ffff700 1 -- 172.21.15.7:0/3824996831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0500b3b30 msgr2=0x7fc0500b0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:28.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.853+0000 7fc04ffff700 1 --2- 172.21.15.7:0/3824996831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0500b3b30 0x7fc0500b0820 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc048009850 tx=0x7fc048009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:28.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.853+0000 7fc04ffff700 1 -- 172.21.15.7:0/3824996831 shutdown_connections 2024-09-13T18:52:28.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.853+0000 7fc04ffff700 1 --2- 172.21.15.7:0/3824996831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0500b3b30 0x7fc0500b0820 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.853+0000 7fc04ffff700 1 -- 172.21.15.7:0/3824996831 >> 172.21.15.7:0/3824996831 conn(0x7fc0500b2880 msgr2=0x7fc0500b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:28.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.853+0000 7fc04ffff700 1 -- 172.21.15.7:0/3824996831 shutdown_connections 2024-09-13T18:52:28.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.853+0000 7fc04ffff700 1 -- 172.21.15.7:0/3824996831 wait complete. 2024-09-13T18:52:28.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.853+0000 7fc04ffff700 1 Processor -- start 2024-09-13T18:52:28.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.853+0000 7fc04ffff700 1 -- start start 2024-09-13T18:52:28.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.853+0000 7fc04ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc05006d070 0x7fc050138f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:28.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.853+0000 7fc04ffff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0500b36c0 0x7fc050137d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:28.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.853+0000 7fc04ffff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc0500b3b30 0x7fc050138270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:28.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.853+0000 7fc04ffff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0501387b0 con 0x7fc05006d070 2024-09-13T18:52:28.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.853+0000 7fc04ffff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05013aea0 con 0x7fc0500b36c0 2024-09-13T18:52:28.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.853+0000 7fc04ffff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05013b180 con 0x7fc0500b3b30 2024-09-13T18:52:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.853+0000 7fc04effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc05006d070 0x7fc050138f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.857+0000 7fc04f7fe700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc0500b3b30 0x7fc050138270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.857+0000 7fc04effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc05006d070 0x7fc050138f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49740/0 (socket says 172.21.15.7:49740) 2024-09-13T18:52:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.857+0000 7fc04effd700 1 -- 172.21.15.7:0/3850547504 learned_addr learned my addr 172.21.15.7:0/3850547504 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:28.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.857+0000 7fc04e7fc700 1 --2- 172.21.15.7:0/3850547504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0500b36c0 0x7fc050137d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:28.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.857+0000 7fc04f7fe700 1 -- 172.21.15.7:0/3850547504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0500b36c0 msgr2=0x7fc050137d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:28.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.857+0000 7fc04f7fe700 1 --2- 172.21.15.7:0/3850547504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0500b36c0 0x7fc050137d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.857+0000 7fc04f7fe700 1 -- 172.21.15.7:0/3850547504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc05006d070 msgr2=0x7fc050138f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:28.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.857+0000 7fc04f7fe700 1 --2- 172.21.15.7:0/3850547504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc05006d070 0x7fc050138f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:28.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.857+0000 7fc04f7fe700 1 -- 172.21.15.7:0/3850547504 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0480094b0 con 0x7fc0500b3b30 2024-09-13T18:52:28.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.857+0000 7fc04f7fe700 1 --2- 172.21.15.7:0/3850547504 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc0500b3b30 0x7fc050138270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc048016000 tx=0x7fc048004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:28.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.857+0000 7fc047fff700 1 -- 172.21.15.7:0/3850547504 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048004550 con 0x7fc0500b3b30 2024-09-13T18:52:28.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.857+0000 7fc047fff700 1 -- 172.21.15.7:0/3850547504 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04801e070 con 0x7fc0500b3b30 2024-09-13T18:52:28.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.857+0000 7fc04ffff700 1 -- 172.21.15.7:0/3850547504 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05013b460 con 0x7fc0500b3b30 2024-09-13T18:52:28.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.857+0000 7fc047fff700 1 -- 172.21.15.7:0/3850547504 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048003900 con 0x7fc0500b3b30 2024-09-13T18:52:28.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.857+0000 7fc04ffff700 1 -- 172.21.15.7:0/3850547504 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05013ba20 con 0x7fc0500b3b30 2024-09-13T18:52:28.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.857+0000 7fc04ffff700 1 -- 172.21.15.7:0/3850547504 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05013bc10 con 0x7fc0500b3b30 2024-09-13T18:52:28.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.861+0000 7fc047fff700 1 -- 172.21.15.7:0/3850547504 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc048003a60 con 0x7fc0500b3b30 2024-09-13T18:52:28.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.861+0000 7fc047fff700 1 --2- 172.21.15.7:0/3850547504 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc030066730 0x7fc030068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:28.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.861+0000 7fc047fff700 1 -- 172.21.15.7:0/3850547504 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc048085300 con 0x7fc0500b3b30 2024-09-13T18:52:28.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.861+0000 7fc04effd700 1 --2- 172.21.15.7:0/3850547504 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc030066730 0x7fc030068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:28.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.861+0000 7fc04effd700 1 --2- 172.21.15.7:0/3850547504 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc030066730 0x7fc030068bf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc040001940 tx=0x7fc040009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:28.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:28.873+0000 7fc047fff700 1 -- 172.21.15.7:0/3850547504 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc0480527e0 con 0x7fc0500b3b30 2024-09-13T18:52:29.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.053+0000 7fc04ffff700 1 -- 172.21.15.7:0/3850547504 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0500b10a0 con 0x7fc0500b3b30 2024-09-13T18:52:29.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.057+0000 7fc047fff700 1 -- 172.21.15.7:0/3850547504 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc048055e00 con 0x7fc0500b3b30 2024-09-13T18:52:29.062 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:52:29.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.057+0000 7fc04ffff700 1 -- 172.21.15.7:0/3850547504 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc030066730 msgr2=0x7fc030068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:29.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.057+0000 7fc04ffff700 1 --2- 172.21.15.7:0/3850547504 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc030066730 0x7fc030068bf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc040001940 tx=0x7fc040009040 comp rx=0 tx=0).stop 2024-09-13T18:52:29.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.057+0000 7fc04ffff700 1 -- 172.21.15.7:0/3850547504 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc0500b3b30 msgr2=0x7fc050138270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:29.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.057+0000 7fc04ffff700 1 --2- 172.21.15.7:0/3850547504 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc0500b3b30 0x7fc050138270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc048016000 tx=0x7fc048004770 comp rx=0 tx=0).stop 2024-09-13T18:52:29.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.057+0000 7fc04ffff700 1 -- 172.21.15.7:0/3850547504 shutdown_connections 2024-09-13T18:52:29.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.057+0000 7fc04ffff700 1 --2- 172.21.15.7:0/3850547504 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc030066730 0x7fc030068bf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.057+0000 7fc04ffff700 1 --2- 172.21.15.7:0/3850547504 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc0500b3b30 0x7fc050138270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.057+0000 7fc04ffff700 1 --2- 172.21.15.7:0/3850547504 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc0500b36c0 0x7fc050137d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.057+0000 7fc04ffff700 1 --2- 172.21.15.7:0/3850547504 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc05006d070 0x7fc050138f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.057+0000 7fc04ffff700 1 -- 172.21.15.7:0/3850547504 >> 172.21.15.7:0/3850547504 conn(0x7fc0500b2880 msgr2=0x7fc050070080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:29.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.057+0000 7fc04ffff700 1 -- 172.21.15.7:0/3850547504 shutdown_connections 2024-09-13T18:52:29.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.057+0000 7fc04ffff700 1 -- 172.21.15.7:0/3850547504 wait complete. 2024-09-13T18:52:29.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:52:29.120 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:52:29.122 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:52:29.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.181+0000 7f3be8dbf700 1 Processor -- start 2024-09-13T18:52:29.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.181+0000 7f3be8dbf700 1 -- start start 2024-09-13T18:52:29.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.181+0000 7f3be8dbf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3be415cda0 0x7f3be40d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.181+0000 7f3be8dbf700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3be40b5e00 0x7f3be40b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.181+0000 7f3be8dbf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be40d0880 con 0x7f3be40b6780 2024-09-13T18:52:29.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.181+0000 7f3be8dbf700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be40cde50 con 0x7f3be415cda0 2024-09-13T18:52:29.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.181+0000 7f3be8dbf700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be40d16e0 con 0x7f3be40b5e00 2024-09-13T18:52:29.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.181+0000 7f3be259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3be415cda0 0x7f3be40d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.181+0000 7f3be2d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3be40b6780 0x7f3be406a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35938/0 (socket says 172.21.15.7:35938) 2024-09-13T18:52:29.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.181+0000 7f3be2d9d700 1 -- 172.21.15.7:0/2051498453 learned_addr learned my addr 172.21.15.7:0/2051498453 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:29.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.181+0000 7f3be1d9b700 1 --2- 172.21.15.7:0/2051498453 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3be40b5e00 0x7f3be40b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.181+0000 7f3be259c700 1 -- 172.21.15.7:0/2051498453 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3be40b5e00 msgr2=0x7f3be40b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:29.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.181+0000 7f3be259c700 1 --2- 172.21.15.7:0/2051498453 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3be40b5e00 0x7f3be40b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.185+0000 7f3be259c700 1 -- 172.21.15.7:0/2051498453 >> v1:172.21.15.7:6789/0 conn(0x7f3be40b6780 legacy=0x7f3be406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:29.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.185+0000 7f3be159a700 1 -- 172.21.15.7:0/2051498453 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963950541 0 0) 0x7f3be40d0880 con 0x7f3be40b6780 2024-09-13T18:52:29.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.185+0000 7f3be259c700 1 -- 172.21.15.7:0/2051498453 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be40d0ad0 con 0x7f3be415cda0 2024-09-13T18:52:29.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.185+0000 7f3be1d9b700 1 --2- 172.21.15.7:0/2051498453 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3be40b5e00 0x7f3be40b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:29.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.185+0000 7f3be259c700 1 --2- 172.21.15.7:0/2051498453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be415cda0 0x7f3be40d1b70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009880 tx=0x7f3bcc009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2169fb62e446b78b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:29.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.185+0000 7f3be159a700 1 -- 172.21.15.7:0/2051498453 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc011830 con 0x7f3be415cda0 2024-09-13T18:52:29.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.185+0000 7f3be159a700 1 -- 172.21.15.7:0/2051498453 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bcc011990 con 0x7f3be415cda0 2024-09-13T18:52:29.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.185+0000 7f3be159a700 1 -- 172.21.15.7:0/2051498453 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc011d20 con 0x7f3be415cda0 2024-09-13T18:52:29.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.185+0000 7f3be8dbf700 1 -- 172.21.15.7:0/2051498453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be415cda0 msgr2=0x7f3be40d1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:29.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.185+0000 7f3be8dbf700 1 --2- 172.21.15.7:0/2051498453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be415cda0 0x7f3be40d1b70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009880 tx=0x7f3bcc009b90 comp rx=0 tx=0).stop 2024-09-13T18:52:29.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.185+0000 7f3be8dbf700 1 -- 172.21.15.7:0/2051498453 shutdown_connections 2024-09-13T18:52:29.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.185+0000 7f3be8dbf700 1 --2- 172.21.15.7:0/2051498453 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3be40b5e00 0x7f3be40b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.185+0000 7f3be8dbf700 1 --2- 172.21.15.7:0/2051498453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be415cda0 0x7f3be40d1b70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.185+0000 7f3be8dbf700 1 -- 172.21.15.7:0/2051498453 >> 172.21.15.7:0/2051498453 conn(0x7f3be40d2cb0 msgr2=0x7f3be40d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:29.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.185+0000 7f3be8dbf700 1 -- 172.21.15.7:0/2051498453 shutdown_connections 2024-09-13T18:52:29.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.185+0000 7f3be8dbf700 1 -- 172.21.15.7:0/2051498453 wait complete. 2024-09-13T18:52:29.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.185+0000 7f3be8dbf700 1 Processor -- start 2024-09-13T18:52:29.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.185+0000 7f3be8dbf700 1 -- start start 2024-09-13T18:52:29.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be8dbf700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3be40b5e00 0x7f3be414d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be8dbf700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3be40b6780 0x7f3be415b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be8dbf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be415cda0 0x7f3be415bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be8dbf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be414d8c0 con 0x7f3be415cda0 2024-09-13T18:52:29.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be8dbf700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be4154530 con 0x7f3be40b5e00 2024-09-13T18:52:29.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be8dbf700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be4158390 con 0x7f3be40b6780 2024-09-13T18:52:29.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be415cda0 0x7f3be415bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be1d9b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3be40b6780 0x7f3be415b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be259c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3be40b5e00 0x7f3be414d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be415cda0 0x7f3be415bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49756/0 (socket says 172.21.15.7:49756) 2024-09-13T18:52:29.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be2d9d700 1 -- 172.21.15.7:0/1354585908 learned_addr learned my addr 172.21.15.7:0/1354585908 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:29.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be259c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3be40b5e00 0x7f3be414d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:41090/0 (socket says 172.21.15.7:41090) 2024-09-13T18:52:29.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be2d9d700 1 -- 172.21.15.7:0/1354585908 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3be40b6780 msgr2=0x7f3be415b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:29.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be2d9d700 1 --2- 172.21.15.7:0/1354585908 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3be40b6780 0x7f3be415b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be2d9d700 1 -- 172.21.15.7:0/1354585908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3be40b5e00 msgr2=0x7f3be414d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be2d9d700 1 --2- 172.21.15.7:0/1354585908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3be40b5e00 0x7f3be414d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be2d9d700 1 -- 172.21.15.7:0/1354585908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc0094b0 con 0x7f3be415cda0 2024-09-13T18:52:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be1d9b700 1 --2- 172.21.15.7:0/1354585908 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3be40b6780 0x7f3be415b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be2d9d700 1 --2- 172.21.15.7:0/1354585908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be415cda0 0x7f3be415bb90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00c4b0 tx=0x7f3bdc00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3bdaffd700 1 -- 172.21.15.7:0/1354585908 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc004ca0 con 0x7f3be415cda0 2024-09-13T18:52:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3bdaffd700 1 -- 172.21.15.7:0/1354585908 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bdc004e20 con 0x7f3be415cda0 2024-09-13T18:52:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be8dbf700 1 -- 172.21.15.7:0/1354585908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be4158670 con 0x7f3be415cda0 2024-09-13T18:52:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3bdaffd700 1 -- 172.21.15.7:0/1354585908 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc007310 con 0x7f3be415cda0 2024-09-13T18:52:29.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.189+0000 7f3be8dbf700 1 -- 172.21.15.7:0/1354585908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be4158c30 con 0x7f3be415cda0 2024-09-13T18:52:29.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.193+0000 7f3bdaffd700 1 -- 172.21.15.7:0/1354585908 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3bdc007470 con 0x7f3be415cda0 2024-09-13T18:52:29.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.193+0000 7f3be8dbf700 1 -- 172.21.15.7:0/1354585908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be4158e50 con 0x7f3be415cda0 2024-09-13T18:52:29.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.193+0000 7f3bdaffd700 1 --2- 172.21.15.7:0/1354585908 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3bc4066730 0x7f3bc4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.193+0000 7f3bdaffd700 1 -- 172.21.15.7:0/1354585908 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bdc083c60 con 0x7f3be415cda0 2024-09-13T18:52:29.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.193+0000 7f3be259c700 1 --2- 172.21.15.7:0/1354585908 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3bc4066730 0x7f3bc4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.193+0000 7f3be259c700 1 --2- 172.21.15.7:0/1354585908 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3bc4066730 0x7f3bc4068bf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0090b0 tx=0x7f3bcc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:29.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.205+0000 7f3bdaffd700 1 -- 172.21.15.7:0/1354585908 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3bdc051120 con 0x7f3be415cda0 2024-09-13T18:52:29.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.373+0000 7f3be8dbf700 1 -- 172.21.15.7:0/1354585908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3be40b6240 con 0x7f3be415cda0 2024-09-13T18:52:29.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.377+0000 7f3bdaffd700 1 -- 172.21.15.7:0/1354585908 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3bdc007720 con 0x7f3be415cda0 2024-09-13T18:52:29.381 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:52:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.377+0000 7f3be8dbf700 1 -- 172.21.15.7:0/1354585908 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3bc4066730 msgr2=0x7f3bc4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.377+0000 7f3be8dbf700 1 --2- 172.21.15.7:0/1354585908 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3bc4066730 0x7f3bc4068bf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0090b0 tx=0x7f3bcc008030 comp rx=0 tx=0).stop 2024-09-13T18:52:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.377+0000 7f3be8dbf700 1 -- 172.21.15.7:0/1354585908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be415cda0 msgr2=0x7f3be415bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.377+0000 7f3be8dbf700 1 --2- 172.21.15.7:0/1354585908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be415cda0 0x7f3be415bb90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00c4b0 tx=0x7f3bdc00c7c0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.377+0000 7f3be8dbf700 1 -- 172.21.15.7:0/1354585908 shutdown_connections 2024-09-13T18:52:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.377+0000 7f3be8dbf700 1 --2- 172.21.15.7:0/1354585908 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3bc4066730 0x7f3bc4068bf0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.377+0000 7f3be8dbf700 1 --2- 172.21.15.7:0/1354585908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be415cda0 0x7f3be415bb90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.377+0000 7f3be8dbf700 1 --2- 172.21.15.7:0/1354585908 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3be40b6780 0x7f3be415b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.377+0000 7f3be8dbf700 1 --2- 172.21.15.7:0/1354585908 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3be40b5e00 0x7f3be414d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.377+0000 7f3be8dbf700 1 -- 172.21.15.7:0/1354585908 >> 172.21.15.7:0/1354585908 conn(0x7f3be40d2cb0 msgr2=0x7f3be40c94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:29.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.377+0000 7f3be8dbf700 1 -- 172.21.15.7:0/1354585908 shutdown_connections 2024-09-13T18:52:29.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.377+0000 7f3be8dbf700 1 -- 172.21.15.7:0/1354585908 wait complete. 2024-09-13T18:52:29.396 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:52:29.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b346e6700 1 Processor -- start 2024-09-13T18:52:29.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b346e6700 1 -- start start 2024-09-13T18:52:29.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b346e6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b2c15bb70 0x7f4b2c15e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b346e6700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b2c06add0 0x7f4b2c06b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b346e6700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b2c155580 con 0x7f4b2c15b790 2024-09-13T18:52:29.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b346e6700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b2c071430 con 0x7f4b2c15bb70 2024-09-13T18:52:29.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b346e6700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c15eb80 con 0x7f4b2c06add0 2024-09-13T18:52:29.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b31c81700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b2c15bb70 0x7f4b2c15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b31c81700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b2c15bb70 0x7f4b2c15e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49764/0 (socket says 172.21.15.7:49764) 2024-09-13T18:52:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b31c81700 1 -- 172.21.15.7:0/3189040072 learned_addr learned my addr 172.21.15.7:0/3189040072 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b32c83700 1 --2- 172.21.15.7:0/3189040072 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b2c06add0 0x7f4b2c06b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b31c81700 1 -- 172.21.15.7:0/3189040072 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b2c06add0 msgr2=0x7f4b2c06b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b31c81700 1 --2- 172.21.15.7:0/3189040072 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b2c06add0 0x7f4b2c06b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b31c81700 1 -- 172.21.15.7:0/3189040072 >> v1:172.21.15.171:6789/0 conn(0x7f4b2c15b790 legacy=0x7f4b2c071b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b31c81700 1 -- 172.21.15.7:0/3189040072 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c0716a0 con 0x7f4b2c15bb70 2024-09-13T18:52:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b31c81700 1 --2- 172.21.15.7:0/3189040072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c15bb70 0x7f4b2c15e640 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009910 tx=0x7f4b20009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a8858d2b0e7a0b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b31480700 1 -- 172.21.15.7:0/3189040072 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b20011830 con 0x7f4b2c15bb70 2024-09-13T18:52:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b31480700 1 -- 172.21.15.7:0/3189040072 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b20011990 con 0x7f4b2c15bb70 2024-09-13T18:52:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.517+0000 7f4b31480700 1 -- 172.21.15.7:0/3189040072 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b20011d40 con 0x7f4b2c15bb70 2024-09-13T18:52:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.521+0000 7f4b346e6700 1 -- 172.21.15.7:0/3189040072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c15bb70 msgr2=0x7f4b2c15e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.521+0000 7f4b346e6700 1 --2- 172.21.15.7:0/3189040072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c15bb70 0x7f4b2c15e640 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009910 tx=0x7f4b20009c20 comp rx=0 tx=0).stop 2024-09-13T18:52:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.521+0000 7f4b346e6700 1 -- 172.21.15.7:0/3189040072 shutdown_connections 2024-09-13T18:52:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.521+0000 7f4b346e6700 1 --2- 172.21.15.7:0/3189040072 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b2c06add0 0x7f4b2c06b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.521+0000 7f4b346e6700 1 --2- 172.21.15.7:0/3189040072 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c15bb70 0x7f4b2c15e640 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.521+0000 7f4b346e6700 1 -- 172.21.15.7:0/3189040072 >> 172.21.15.7:0/3189040072 conn(0x7f4b2c06d900 msgr2=0x7f4b2c06dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:29.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.521+0000 7f4b346e6700 1 -- 172.21.15.7:0/3189040072 shutdown_connections 2024-09-13T18:52:29.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.521+0000 7f4b346e6700 1 -- 172.21.15.7:0/3189040072 wait complete. 2024-09-13T18:52:29.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.521+0000 7f4b346e6700 1 Processor -- start 2024-09-13T18:52:29.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.521+0000 7f4b346e6700 1 -- start start 2024-09-13T18:52:29.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b346e6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c06add0 0x7f4b2c15db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b346e6700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b2c15b790 0x7f4b2c0cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b346e6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b2c15bb70 0x7f4b2c0ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b346e6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c15e760 con 0x7f4b2c06add0 2024-09-13T18:52:29.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b346e6700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c15e480 con 0x7f4b2c15bb70 2024-09-13T18:52:29.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b346e6700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2c15c390 con 0x7f4b2c15b790 2024-09-13T18:52:29.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b32482700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c06add0 0x7f4b2c15db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b32482700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c06add0 0x7f4b2c15db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49770/0 (socket says 172.21.15.7:49770) 2024-09-13T18:52:29.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b32482700 1 -- 172.21.15.7:0/158936713 learned_addr learned my addr 172.21.15.7:0/158936713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:29.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b31c81700 1 --2- 172.21.15.7:0/158936713 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b2c15b790 0x7f4b2c0cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b32c83700 1 --2- 172.21.15.7:0/158936713 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b2c15bb70 0x7f4b2c0ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b32482700 1 -- 172.21.15.7:0/158936713 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b2c15b790 msgr2=0x7f4b2c0cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b32482700 1 --2- 172.21.15.7:0/158936713 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b2c15b790 0x7f4b2c0cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b32482700 1 -- 172.21.15.7:0/158936713 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b2c15bb70 msgr2=0x7f4b2c0ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b32482700 1 --2- 172.21.15.7:0/158936713 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b2c15bb70 0x7f4b2c0ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b32482700 1 -- 172.21.15.7:0/158936713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b20009570 con 0x7f4b2c06add0 2024-09-13T18:52:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b32c83700 1 --2- 172.21.15.7:0/158936713 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b2c15bb70 0x7f4b2c0ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b32482700 1 --2- 172.21.15.7:0/158936713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c06add0 0x7f4b2c15db80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4b28001ed0 tx=0x7f4b280021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b1effd700 1 -- 172.21.15.7:0/158936713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b28003c20 con 0x7f4b2c06add0 2024-09-13T18:52:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b1effd700 1 -- 172.21.15.7:0/158936713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b28003d80 con 0x7f4b2c06add0 2024-09-13T18:52:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b1effd700 1 -- 172.21.15.7:0/158936713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2800abf0 con 0x7f4b2c06add0 2024-09-13T18:52:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b346e6700 1 -- 172.21.15.7:0/158936713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2c15c670 con 0x7f4b2c06add0 2024-09-13T18:52:29.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b346e6700 1 -- 172.21.15.7:0/158936713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2c15ce70 con 0x7f4b2c06add0 2024-09-13T18:52:29.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.525+0000 7f4b346e6700 1 -- 172.21.15.7:0/158936713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b2c15c1f0 con 0x7f4b2c06add0 2024-09-13T18:52:29.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.529+0000 7f4b1effd700 1 -- 172.21.15.7:0/158936713 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4b2800ad50 con 0x7f4b2c06add0 2024-09-13T18:52:29.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.529+0000 7f4b1effd700 1 --2- 172.21.15.7:0/158936713 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4b10066730 0x7f4b10068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.529+0000 7f4b1effd700 1 -- 172.21.15.7:0/158936713 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b28084270 con 0x7f4b2c06add0 2024-09-13T18:52:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.529+0000 7f4b31c81700 1 --2- 172.21.15.7:0/158936713 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4b10066730 0x7f4b10068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.529+0000 7f4b31c81700 1 --2- 172.21.15.7:0/158936713 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4b10066730 0x7f4b10068bf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4b20000c00 tx=0x7f4b20008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:29.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.537+0000 7f4b1effd700 1 -- 172.21.15.7:0/158936713 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4b2c15c1f0 con 0x7f4b2c06add0 2024-09-13T18:52:29.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.705+0000 7f4b346e6700 1 -- 172.21.15.7:0/158936713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4b2c15c1f0 con 0x7f4b2c06add0 2024-09-13T18:52:29.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.705+0000 7f4b1effd700 1 -- 172.21.15.7:0/158936713 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4b2c15c1f0 con 0x7f4b2c06add0 2024-09-13T18:52:29.713 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:52:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.709+0000 7f4b346e6700 1 -- 172.21.15.7:0/158936713 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4b10066730 msgr2=0x7f4b10068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.709+0000 7f4b346e6700 1 --2- 172.21.15.7:0/158936713 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4b10066730 0x7f4b10068bf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4b20000c00 tx=0x7f4b20008030 comp rx=0 tx=0).stop 2024-09-13T18:52:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.709+0000 7f4b346e6700 1 -- 172.21.15.7:0/158936713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c06add0 msgr2=0x7f4b2c15db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.709+0000 7f4b346e6700 1 --2- 172.21.15.7:0/158936713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c06add0 0x7f4b2c15db80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4b28001ed0 tx=0x7f4b280021e0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.709+0000 7f4b346e6700 1 -- 172.21.15.7:0/158936713 shutdown_connections 2024-09-13T18:52:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.709+0000 7f4b346e6700 1 --2- 172.21.15.7:0/158936713 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4b10066730 0x7f4b10068bf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.709+0000 7f4b346e6700 1 --2- 172.21.15.7:0/158936713 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b2c15bb70 0x7f4b2c0ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.709+0000 7f4b346e6700 1 --2- 172.21.15.7:0/158936713 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b2c15b790 0x7f4b2c0cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.709+0000 7f4b346e6700 1 --2- 172.21.15.7:0/158936713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b2c06add0 0x7f4b2c15db80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.709+0000 7f4b346e6700 1 -- 172.21.15.7:0/158936713 >> 172.21.15.7:0/158936713 conn(0x7f4b2c06d900 msgr2=0x7f4b2c071b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.709+0000 7f4b346e6700 1 -- 172.21.15.7:0/158936713 shutdown_connections 2024-09-13T18:52:29.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.709+0000 7f4b346e6700 1 -- 172.21.15.7:0/158936713 wait complete. 2024-09-13T18:52:29.728 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:29.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.837+0000 7fccddb05700 1 Processor -- start 2024-09-13T18:52:29.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.837+0000 7fccddb05700 1 -- start start 2024-09-13T18:52:29.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.837+0000 7fccddb05700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccd8070fb0 0x7fccd8071390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.837+0000 7fccddb05700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fccd8159e30 0x7fccd815a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.837+0000 7fccddb05700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd815b840 con 0x7fccd815bcd0 2024-09-13T18:52:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.837+0000 7fccddb05700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd80718d0 con 0x7fccd8159e30 2024-09-13T18:52:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.837+0000 7fccddb05700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd815ac30 con 0x7fccd8070fb0 2024-09-13T18:52:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccdd304700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fccd815bcd0 0x7fccd8156d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35950/0 (socket says 172.21.15.7:35950) 2024-09-13T18:52:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccdd304700 1 -- 172.21.15.7:0/669474264 learned_addr learned my addr 172.21.15.7:0/669474264 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd7fff700 1 --2- 172.21.15.7:0/669474264 >> v2:172.21.15.171:3300/0 conn(0x7fccd8159e30 0x7fccd815a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccdcb03700 1 --2- 172.21.15.7:0/669474264 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccd8070fb0 0x7fccd8071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 -- 172.21.15.7:0/669474264 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2767369638 0 0) 0x7fccd815b840 con 0x7fccd815bcd0 2024-09-13T18:52:29.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 -- 172.21.15.7:0/669474264 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccc40035d0 con 0x7fccd815bcd0 2024-09-13T18:52:29.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 -- 172.21.15.7:0/669474264 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3070178170 0 0) 0x7fccc40035d0 con 0x7fccd815bcd0 2024-09-13T18:52:29.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 -- 172.21.15.7:0/669474264 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccd8070fb0 msgr2=0x7fccd8071390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:29.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 --2- 172.21.15.7:0/669474264 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccd8070fb0 0x7fccd8071390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 -- 172.21.15.7:0/669474264 >> v2:172.21.15.171:3300/0 conn(0x7fccd8159e30 msgr2=0x7fccd815a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:29.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 --2- 172.21.15.7:0/669474264 >> v2:172.21.15.171:3300/0 conn(0x7fccd8159e30 0x7fccd815a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 -- 172.21.15.7:0/669474264 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd815af10 con 0x7fccd815bcd0 2024-09-13T18:52:29.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 -- 172.21.15.7:0/669474264 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7fccd0001670 con 0x7fccd815bcd0 2024-09-13T18:52:29.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 -- 172.21.15.7:0/669474264 >> v1:172.21.15.7:6789/0 conn(0x7fccd815bcd0 legacy=0x7fccd8156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:29.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 --2- 172.21.15.7:0/669474264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccc4003a20 0x7fccc4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 --2- 172.21.15.7:0/669474264 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccd8070fb0 0x7fccc4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 --2- 172.21.15.7:0/669474264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccc4004db0 0x7fccc4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 -- 172.21.15.7:0/669474264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd80718d0 con 0x7fccc4004db0 2024-09-13T18:52:29.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 -- 172.21.15.7:0/669474264 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd815ac30 con 0x7fccc4003a20 2024-09-13T18:52:29.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 -- 172.21.15.7:0/669474264 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc400ba30 con 0x7fccd8070fb0 2024-09-13T18:52:29.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccdd304700 1 --2- 172.21.15.7:0/669474264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccc4004db0 0x7fccc4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccdcb03700 1 --2- 172.21.15.7:0/669474264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccc4003a20 0x7fccc4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd7fff700 1 --2- 172.21.15.7:0/669474264 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccd8070fb0 0x7fccc4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccdd304700 1 -- 172.21.15.7:0/669474264 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccd8070fb0 msgr2=0x7fccc4004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:29.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccdd304700 1 --2- 172.21.15.7:0/669474264 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccd8070fb0 0x7fccc4004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccdd304700 1 -- 172.21.15.7:0/669474264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccc4003a20 msgr2=0x7fccc4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:29.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccdd304700 1 --2- 172.21.15.7:0/669474264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccc4003a20 0x7fccc4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccdd304700 1 -- 172.21.15.7:0/669474264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc40035d0 con 0x7fccc4004db0 2024-09-13T18:52:29.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccdd304700 1 --2- 172.21.15.7:0/669474264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccc4004db0 0x7fccc4005230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fccd000c320 tx=0x7fccc40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764063e8a1be9c3f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:29.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 -- 172.21.15.7:0/669474264 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd000ce60 con 0x7fccc4004db0 2024-09-13T18:52:29.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 -- 172.21.15.7:0/669474264 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd0005980 con 0x7fccc4004db0 2024-09-13T18:52:29.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccd77fe700 1 -- 172.21.15.7:0/669474264 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd0005d30 con 0x7fccc4004db0 2024-09-13T18:52:29.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccddb05700 1 -- 172.21.15.7:0/669474264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccc4004db0 msgr2=0x7fccc4005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:29.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccddb05700 1 --2- 172.21.15.7:0/669474264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccc4004db0 0x7fccc4005230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fccd000c320 tx=0x7fccc40058a0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccdd304700 1 -- 172.21.15.7:0/669474264 reap_dead start 2024-09-13T18:52:29.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccddb05700 1 -- 172.21.15.7:0/669474264 shutdown_connections 2024-09-13T18:52:29.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccddb05700 1 -- 172.21.15.7:0/669474264 >> 172.21.15.7:0/669474264 conn(0x7fccd806a4f0 msgr2=0x7fccd806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:29.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccddb05700 1 -- 172.21.15.7:0/669474264 shutdown_connections 2024-09-13T18:52:29.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.841+0000 7fccddb05700 1 -- 172.21.15.7:0/669474264 wait complete. 2024-09-13T18:52:29.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccddb05700 1 Processor -- start 2024-09-13T18:52:29.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccddb05700 1 -- start start 2024-09-13T18:52:29.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccddb05700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccc4004db0 0x7fccd814aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccddb05700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccd81f2260 0x7fccd8150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccddb05700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccd8150ea0 0x7fccd814b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccddb05700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd8151970 con 0x7fccd8150ea0 2024-09-13T18:52:29.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccddb05700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd80acdd0 con 0x7fccc4004db0 2024-09-13T18:52:29.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccddb05700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd81513b0 con 0x7fccd81f2260 2024-09-13T18:52:29.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccdd304700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccd8150ea0 0x7fccd814b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccdcb03700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccc4004db0 0x7fccd814aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccd7fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccd81f2260 0x7fccd8150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccdd304700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccd8150ea0 0x7fccd814b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49776/0 (socket says 172.21.15.7:49776) 2024-09-13T18:52:29.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccdd304700 1 -- 172.21.15.7:0/848147634 learned_addr learned my addr 172.21.15.7:0/848147634 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:29.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccd7fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccd81f2260 0x7fccd8150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:53466/0 (socket says 172.21.15.7:53466) 2024-09-13T18:52:29.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccdcb03700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccc4004db0 0x7fccd814aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:41120/0 (socket says 172.21.15.7:41120) 2024-09-13T18:52:29.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccdd304700 1 -- 172.21.15.7:0/848147634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccd81f2260 msgr2=0x7fccd8150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:29.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccdd304700 1 --2- 172.21.15.7:0/848147634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccd81f2260 0x7fccd8150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccdd304700 1 -- 172.21.15.7:0/848147634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccc4004db0 msgr2=0x7fccd814aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:29.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccdd304700 1 --2- 172.21.15.7:0/848147634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccc4004db0 0x7fccd814aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:29.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccdd304700 1 -- 172.21.15.7:0/848147634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd8151690 con 0x7fccd8150ea0 2024-09-13T18:52:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccdd304700 1 --2- 172.21.15.7:0/848147634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccd8150ea0 0x7fccd814b880 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fccd0011000 tx=0x7fccd0001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccd57fa700 1 -- 172.21.15.7:0/848147634 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd0005e20 con 0x7fccd8150ea0 2024-09-13T18:52:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccd57fa700 1 -- 172.21.15.7:0/848147634 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd0004aa0 con 0x7fccd8150ea0 2024-09-13T18:52:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccd57fa700 1 -- 172.21.15.7:0/848147634 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd0004e30 con 0x7fccd8150ea0 2024-09-13T18:52:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccddb05700 1 -- 172.21.15.7:0/848147634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd80aca60 con 0x7fccd8150ea0 2024-09-13T18:52:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.845+0000 7fccddb05700 1 -- 172.21.15.7:0/848147634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd814bdc0 con 0x7fccd8150ea0 2024-09-13T18:52:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.849+0000 7fccddb05700 1 -- 172.21.15.7:0/848147634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd80ac8c0 con 0x7fccd8150ea0 2024-09-13T18:52:29.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.849+0000 7fccd57fa700 1 -- 172.21.15.7:0/848147634 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fccd0005070 con 0x7fccd8150ea0 2024-09-13T18:52:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.849+0000 7fccd57fa700 1 --2- 172.21.15.7:0/848147634 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fccb8066760 0x7fccb8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.849+0000 7fccd57fa700 1 -- 172.21.15.7:0/848147634 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fccd00848f0 con 0x7fccd8150ea0 2024-09-13T18:52:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.849+0000 7fccdcb03700 1 --2- 172.21.15.7:0/848147634 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fccb8066760 0x7fccb8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:29.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.849+0000 7fccdcb03700 1 --2- 172.21.15.7:0/848147634 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fccb8066760 0x7fccb8068c20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fccd80716e0 tx=0x7fccc8009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:29.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:29.857+0000 7fccd57fa700 1 -- 172.21.15.7:0/848147634 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fccd0051ef0 con 0x7fccd8150ea0 2024-09-13T18:52:30.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.069+0000 7fccddb05700 1 -- 172.21.15.7:0/848147634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fccd80ac8c0 con 0x7fccd8150ea0 2024-09-13T18:52:30.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.073+0000 7fccd57fa700 1 -- 172.21.15.7:0/848147634 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4725 (secure 0 0 0) 0x7fccd80ac8c0 con 0x7fccd8150ea0 2024-09-13T18:52:30.078 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:30.078 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1309419125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1309419125},{"type":"v1","addr":"172.21.15.7:6835","nonce":1309419125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/951828843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":951828843},{"type":"v1","addr":"172.21.15.7:6833","nonce":951828843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4226627580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4226627580},{"type":"v1","addr":"172.21.15.171:6835","nonce":4226627580}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:52:17.908937+0000","modified":"2024-09-13T18:52:20.930515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6837/3658296803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3658296803},{"type":"v1","addr":"172.21.15.171:6837","nonce":3658296803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:30.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.073+0000 7fccddb05700 1 -- 172.21.15.7:0/848147634 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fccb8066760 msgr2=0x7fccb8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.073+0000 7fccddb05700 1 --2- 172.21.15.7:0/848147634 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fccb8066760 0x7fccb8068c20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fccd80716e0 tx=0x7fccc8009240 comp rx=0 tx=0).stop 2024-09-13T18:52:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.073+0000 7fccddb05700 1 -- 172.21.15.7:0/848147634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccd8150ea0 msgr2=0x7fccd814b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.073+0000 7fccddb05700 1 --2- 172.21.15.7:0/848147634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccd8150ea0 0x7fccd814b880 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fccd0011000 tx=0x7fccd0001a00 comp rx=0 tx=0).stop 2024-09-13T18:52:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.073+0000 7fccddb05700 1 -- 172.21.15.7:0/848147634 shutdown_connections 2024-09-13T18:52:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.073+0000 7fccddb05700 1 --2- 172.21.15.7:0/848147634 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fccb8066760 0x7fccb8068c20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.073+0000 7fccddb05700 1 --2- 172.21.15.7:0/848147634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccd8150ea0 0x7fccd814b880 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.073+0000 7fccddb05700 1 --2- 172.21.15.7:0/848147634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccd81f2260 0x7fccd8150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.073+0000 7fccddb05700 1 --2- 172.21.15.7:0/848147634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccc4004db0 0x7fccd814aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.073+0000 7fccddb05700 1 -- 172.21.15.7:0/848147634 >> 172.21.15.7:0/848147634 conn(0x7fccd806a4f0 msgr2=0x7fccd806de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.073+0000 7fccddb05700 1 -- 172.21.15.7:0/848147634 shutdown_connections 2024-09-13T18:52:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.073+0000 7fccddb05700 1 -- 172.21.15.7:0/848147634 wait complete. 2024-09-13T18:52:30.082 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-13T18:52:30.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7bb7296700 1 Processor -- start 2024-09-13T18:52:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7bb7296700 1 -- start start 2024-09-13T18:52:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7bb7296700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f7bb015d060 0x7f7bb006bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7bb7296700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb00cfaa0 con 0x7f7bb00b0060 2024-09-13T18:52:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7bb7296700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb00b11f0 con 0x7f7bb00cdf30 2024-09-13T18:52:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7bb7296700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb00b04a0 con 0x7f7bb015d060 2024-09-13T18:52:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7bb4831700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7bb00b0060 0x7f7bb00cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35958/0 (socket says 172.21.15.7:35958) 2024-09-13T18:52:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7bb4831700 1 -- 172.21.15.7:0/2756121995 learned_addr learned my addr 172.21.15.7:0/2756121995 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:30.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7bb5032700 1 --2- 172.21.15.7:0/2756121995 >> v2:172.21.15.171:3300/0 conn(0x7f7bb015d060 0x7f7bb006bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7baffff700 1 -- 172.21.15.7:0/2756121995 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217208187 0 0) 0x7f7bb00cfaa0 con 0x7f7bb00b0060 2024-09-13T18:52:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7baffff700 1 -- 172.21.15.7:0/2756121995 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b980035d0 con 0x7f7bb00b0060 2024-09-13T18:52:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7baffff700 1 -- 172.21.15.7:0/2756121995 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3400209478 0 0) 0x7f7bb00b11f0 con 0x7f7bb00cdf30 2024-09-13T18:52:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7baffff700 1 -- 172.21.15.7:0/2756121995 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb00cfaa0 con 0x7f7bb00cdf30 2024-09-13T18:52:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7baffff700 1 -- 172.21.15.7:0/2756121995 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3093484237 0 0) 0x7f7b980035d0 con 0x7f7bb00b0060 2024-09-13T18:52:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7baffff700 1 -- 172.21.15.7:0/2756121995 >> v2:172.21.15.171:3300/0 conn(0x7f7bb015d060 msgr2=0x7f7bb006bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7baffff700 1 --2- 172.21.15.7:0/2756121995 >> v2:172.21.15.171:3300/0 conn(0x7f7bb015d060 0x7f7bb006bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7baffff700 1 -- 172.21.15.7:0/2756121995 >> v1:172.21.15.171:6789/0 conn(0x7f7bb00cdf30 legacy=0x7f7bb00b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7baffff700 1 -- 172.21.15.7:0/2756121995 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb00cf200 con 0x7f7bb00b0060 2024-09-13T18:52:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7baffff700 1 -- 172.21.15.7:0/2756121995 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7f7ba00019f0 con 0x7f7bb00b0060 2024-09-13T18:52:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7baffff700 1 -- 172.21.15.7:0/2756121995 >> v1:172.21.15.7:6789/0 conn(0x7f7bb00b0060 legacy=0x7f7bb00cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7baffff700 1 --2- 172.21.15.7:0/2756121995 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7b98003c00 0x7f7b98004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7baffff700 1 --2- 172.21.15.7:0/2756121995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b98004870 0x7f7b98004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7baffff700 1 --2- 172.21.15.7:0/2756121995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b98005280 0x7f7b9800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7baffff700 1 -- 172.21.15.7:0/2756121995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb00b04a0 con 0x7f7b98004870 2024-09-13T18:52:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7baffff700 1 -- 172.21.15.7:0/2756121995 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9800c1b0 con 0x7f7b98005280 2024-09-13T18:52:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7baffff700 1 -- 172.21.15.7:0/2756121995 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9800c370 con 0x7f7b98003c00 2024-09-13T18:52:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7bb4831700 1 --2- 172.21.15.7:0/2756121995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b98004870 0x7f7b98004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7bb5032700 1 --2- 172.21.15.7:0/2756121995 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7b98003c00 0x7f7b98004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7bb5833700 1 --2- 172.21.15.7:0/2756121995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b98005280 0x7f7b9800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7bb4831700 1 -- 172.21.15.7:0/2756121995 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7b98003c00 msgr2=0x7f7b98004000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7bb4831700 1 --2- 172.21.15.7:0/2756121995 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7b98003c00 0x7f7b98004000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7bb4831700 1 -- 172.21.15.7:0/2756121995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b98005280 msgr2=0x7f7b9800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7bb4831700 1 --2- 172.21.15.7:0/2756121995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b98005280 0x7f7b9800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7bb4831700 1 -- 172.21.15.7:0/2756121995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b980035d0 con 0x7f7b98004870 2024-09-13T18:52:30.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.205+0000 7f7bb5833700 1 -- 172.21.15.7:0/2756121995 reap_dead start 2024-09-13T18:52:30.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.209+0000 7f7bb4831700 1 --2- 172.21.15.7:0/2756121995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b98004870 0x7f7b98004cf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000a700 tx=0x7f7b980058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6cce33d0faa0fa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:30.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.209+0000 7f7baffff700 1 -- 172.21.15.7:0/2756121995 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0001db0 con 0x7f7b98004870 2024-09-13T18:52:30.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.209+0000 7f7baffff700 1 -- 172.21.15.7:0/2756121995 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba00070d0 con 0x7f7b98004870 2024-09-13T18:52:30.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.209+0000 7f7baffff700 1 -- 172.21.15.7:0/2756121995 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0007460 con 0x7f7b98004870 2024-09-13T18:52:30.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.209+0000 7f7bb7296700 1 -- 172.21.15.7:0/2756121995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b98004870 msgr2=0x7f7b98004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.209+0000 7f7bb7296700 1 --2- 172.21.15.7:0/2756121995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b98004870 0x7f7b98004cf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000a700 tx=0x7f7b980058a0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.209+0000 7f7bb7296700 1 -- 172.21.15.7:0/2756121995 shutdown_connections 2024-09-13T18:52:30.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.209+0000 7f7bb7296700 1 --2- 172.21.15.7:0/2756121995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b98004870 0x7f7b98004cf0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.209+0000 7f7bb7296700 1 -- 172.21.15.7:0/2756121995 >> 172.21.15.7:0/2756121995 conn(0x7f7bb006a310 msgr2=0x7f7bb006a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:30.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.209+0000 7f7bb7296700 1 -- 172.21.15.7:0/2756121995 shutdown_connections 2024-09-13T18:52:30.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.209+0000 7f7bb7296700 1 -- 172.21.15.7:0/2756121995 wait complete. 2024-09-13T18:52:30.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.209+0000 7f7bb7296700 1 Processor -- start 2024-09-13T18:52:30.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb7296700 1 -- start start 2024-09-13T18:52:30.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb7296700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b98004870 0x7f7bb01f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb7296700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7bb00b4a30 0x7f7bb015fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb7296700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bb01585f0 0x7f7bb0158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb7296700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb0158fe0 con 0x7f7b98004870 2024-09-13T18:52:30.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb7296700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb00ac8d0 con 0x7f7bb00b4a30 2024-09-13T18:52:30.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb7296700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb01f3480 con 0x7f7bb01585f0 2024-09-13T18:52:30.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb5032700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b98004870 0x7f7bb01f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb5032700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b98004870 0x7f7bb01f2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49784/0 (socket says 172.21.15.7:49784) 2024-09-13T18:52:30.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb5032700 1 -- 172.21.15.7:0/3809586155 learned_addr learned my addr 172.21.15.7:0/3809586155 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:30.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb4831700 1 --2- 172.21.15.7:0/3809586155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7bb00b4a30 0x7f7bb015fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb5833700 1 --2- 172.21.15.7:0/3809586155 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bb01585f0 0x7f7bb0158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb5032700 1 -- 172.21.15.7:0/3809586155 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bb01585f0 msgr2=0x7f7bb0158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb5032700 1 --2- 172.21.15.7:0/3809586155 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bb01585f0 0x7f7bb0158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb5032700 1 -- 172.21.15.7:0/3809586155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7bb00b4a30 msgr2=0x7f7bb015fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb5032700 1 --2- 172.21.15.7:0/3809586155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7bb00b4a30 0x7f7bb015fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb5032700 1 -- 172.21.15.7:0/3809586155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb015d5c0 con 0x7f7b98004870 2024-09-13T18:52:30.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb5833700 1 --2- 172.21.15.7:0/3809586155 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bb01585f0 0x7f7bb0158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:30.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb5032700 1 --2- 172.21.15.7:0/3809586155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b98004870 0x7f7bb01f2f40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4011a20 tx=0x7f7ba4011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:30.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7badffb700 1 -- 172.21.15.7:0/3809586155 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400f9a0 con 0x7f7b98004870 2024-09-13T18:52:30.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7badffb700 1 -- 172.21.15.7:0/3809586155 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba400fb00 con 0x7f7b98004870 2024-09-13T18:52:30.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb7296700 1 -- 172.21.15.7:0/3809586155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb00b4f10 con 0x7f7b98004870 2024-09-13T18:52:30.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7badffb700 1 -- 172.21.15.7:0/3809586155 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400fe90 con 0x7f7b98004870 2024-09-13T18:52:30.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb7296700 1 -- 172.21.15.7:0/3809586155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb00b54d0 con 0x7f7b98004870 2024-09-13T18:52:30.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.213+0000 7f7bb7296700 1 -- 172.21.15.7:0/3809586155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb006bcb0 con 0x7f7b98004870 2024-09-13T18:52:30.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.217+0000 7f7badffb700 1 -- 172.21.15.7:0/3809586155 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7ba4014440 con 0x7f7b98004870 2024-09-13T18:52:30.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.217+0000 7f7badffb700 1 --2- 172.21.15.7:0/3809586155 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7b9c066760 0x7f7b9c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.217+0000 7f7badffb700 1 -- 172.21.15.7:0/3809586155 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ba40848e0 con 0x7f7b98004870 2024-09-13T18:52:30.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.217+0000 7f7bb4831700 1 --2- 172.21.15.7:0/3809586155 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7b9c066760 0x7f7b9c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.217+0000 7f7bb4831700 1 --2- 172.21.15.7:0/3809586155 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7b9c066760 0x7f7b9c068c20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0000f80 tx=0x7f7ba0009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:30.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.225+0000 7f7badffb700 1 -- 172.21.15.7:0/3809586155 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7ba4051bf0 con 0x7f7b98004870 2024-09-13T18:52:30.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.401+0000 7f7bb7296700 1 -- 172.21.15.7:0/3809586155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bb006a9d0 con 0x7f7b98004870 2024-09-13T18:52:30.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.401+0000 7f7badffb700 1 -- 172.21.15.7:0/3809586155 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f7ba4022090 con 0x7f7b98004870 2024-09-13T18:52:30.409 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:30.409 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:30.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.405+0000 7f7bb7296700 1 -- 172.21.15.7:0/3809586155 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7b9c066760 msgr2=0x7f7b9c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.405+0000 7f7bb7296700 1 --2- 172.21.15.7:0/3809586155 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7b9c066760 0x7f7b9c068c20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0000f80 tx=0x7f7ba0009000 comp rx=0 tx=0).stop 2024-09-13T18:52:30.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.405+0000 7f7bb7296700 1 -- 172.21.15.7:0/3809586155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b98004870 msgr2=0x7f7bb01f2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.405+0000 7f7bb7296700 1 --2- 172.21.15.7:0/3809586155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b98004870 0x7f7bb01f2f40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4011a20 tx=0x7f7ba4011de0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.405+0000 7f7bb7296700 1 -- 172.21.15.7:0/3809586155 shutdown_connections 2024-09-13T18:52:30.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.405+0000 7f7bb7296700 1 --2- 172.21.15.7:0/3809586155 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7b9c066760 0x7f7b9c068c20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.405+0000 7f7bb7296700 1 --2- 172.21.15.7:0/3809586155 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7bb01585f0 0x7f7bb0158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.405+0000 7f7bb7296700 1 --2- 172.21.15.7:0/3809586155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7bb00b4a30 0x7f7bb015fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.405+0000 7f7bb7296700 1 --2- 172.21.15.7:0/3809586155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b98004870 0x7f7bb01f2f40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.405+0000 7f7bb7296700 1 -- 172.21.15.7:0/3809586155 >> 172.21.15.7:0/3809586155 conn(0x7f7bb006a310 msgr2=0x7f7bb00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:30.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.405+0000 7f7bb7296700 1 -- 172.21.15.7:0/3809586155 shutdown_connections 2024-09-13T18:52:30.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.405+0000 7f7bb7296700 1 -- 172.21.15.7:0/3809586155 wait complete. 2024-09-13T18:52:30.425 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:52:30.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.533+0000 7f0131b12700 1 Processor -- start 2024-09-13T18:52:30.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.533+0000 7f0131b12700 1 -- start start 2024-09-13T18:52:30.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.533+0000 7f0131b12700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f012c0d1fa0 0x7f012c15e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.533+0000 7f0131b12700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f012c06b480 0x7f012c06b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.533+0000 7f0131b12700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f012c15e960 con 0x7f012c15cb70 2024-09-13T18:52:30.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.533+0000 7f0131b12700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f012c0cd480 con 0x7f012c06b480 2024-09-13T18:52:30.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.533+0000 7f0131b12700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012c0cd760 con 0x7f012c0d1fa0 2024-09-13T18:52:30.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f012bfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f012c0d1fa0 0x7f012c15e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f0131311700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f012c06b480 0x7f012c06b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f012bfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f012c0d1fa0 0x7f012c15e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:53500/0 (socket says 172.21.15.7:53500) 2024-09-13T18:52:30.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f012bfff700 1 -- 172.21.15.7:0/666542197 learned_addr learned my addr 172.21.15.7:0/666542197 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:30.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f0130b10700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f012c15cb70 0x7f012c0d1890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:51738/0 (socket says 172.21.15.7:51738) 2024-09-13T18:52:30.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f012bfff700 1 -- 172.21.15.7:0/666542197 >> v2:172.21.15.171:3300/0 conn(0x7f012c06b480 msgr2=0x7f012c06b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:30.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f012bfff700 1 --2- 172.21.15.7:0/666542197 >> v2:172.21.15.171:3300/0 conn(0x7f012c06b480 0x7f012c06b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f012bfff700 1 -- 172.21.15.7:0/666542197 >> v1:172.21.15.171:6789/0 conn(0x7f012c15cb70 legacy=0x7f012c0d1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f012bfff700 1 -- 172.21.15.7:0/666542197 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c0cda40 con 0x7f012c0d1fa0 2024-09-13T18:52:30.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f012bfff700 1 --2- 172.21.15.7:0/666542197 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f012c0d1fa0 0x7f012c15e420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0118009910 tx=0x7f0118009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=661d3017740d9e48 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:30.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f012b7fe700 1 -- 172.21.15.7:0/666542197 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0118011830 con 0x7f012c0d1fa0 2024-09-13T18:52:30.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f012b7fe700 1 -- 172.21.15.7:0/666542197 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0118011990 con 0x7f012c0d1fa0 2024-09-13T18:52:30.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f012b7fe700 1 -- 172.21.15.7:0/666542197 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0118011d40 con 0x7f012c0d1fa0 2024-09-13T18:52:30.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f0131b12700 1 -- 172.21.15.7:0/666542197 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f012c0d1fa0 msgr2=0x7f012c15e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f0131b12700 1 --2- 172.21.15.7:0/666542197 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f012c0d1fa0 0x7f012c15e420 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0118009910 tx=0x7f0118009c20 comp rx=0 tx=0).stop 2024-09-13T18:52:30.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f0131b12700 1 -- 172.21.15.7:0/666542197 shutdown_connections 2024-09-13T18:52:30.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f0131b12700 1 --2- 172.21.15.7:0/666542197 >> v2:172.21.15.171:3300/0 conn(0x7f012c06b480 0x7f012c06b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f0131b12700 1 --2- 172.21.15.7:0/666542197 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f012c0d1fa0 0x7f012c15e420 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f0131b12700 1 -- 172.21.15.7:0/666542197 >> 172.21.15.7:0/666542197 conn(0x7f012c0cf730 msgr2=0x7f012c0cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:30.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f0131b12700 1 -- 172.21.15.7:0/666542197 shutdown_connections 2024-09-13T18:52:30.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f0131b12700 1 -- 172.21.15.7:0/666542197 wait complete. 2024-09-13T18:52:30.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f0131b12700 1 Processor -- start 2024-09-13T18:52:30.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.537+0000 7f0131b12700 1 -- start start 2024-09-13T18:52:30.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0131b12700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f012c06b480 0x7f012c0cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0131b12700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f012c0d1fa0 0x7f012c0c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0131b12700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f012c15cb70 0x7f012c1f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0131b12700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c157c70 con 0x7f012c15cb70 2024-09-13T18:52:30.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0131b12700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c0c7cc0 con 0x7f012c0d1fa0 2024-09-13T18:52:30.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0131b12700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012c1f2bb0 con 0x7f012c06b480 2024-09-13T18:52:30.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0131311700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f012c15cb70 0x7f012c1f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0131311700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f012c15cb70 0x7f012c1f2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49798/0 (socket says 172.21.15.7:49798) 2024-09-13T18:52:30.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0131311700 1 -- 172.21.15.7:0/2119863532 learned_addr learned my addr 172.21.15.7:0/2119863532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:30.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0130b10700 1 --2- 172.21.15.7:0/2119863532 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f012c06b480 0x7f012c0cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f012bfff700 1 --2- 172.21.15.7:0/2119863532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f012c0d1fa0 0x7f012c0c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0130b10700 1 -- 172.21.15.7:0/2119863532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f012c0d1fa0 msgr2=0x7f012c0c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0130b10700 1 --2- 172.21.15.7:0/2119863532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f012c0d1fa0 0x7f012c0c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0130b10700 1 -- 172.21.15.7:0/2119863532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f012c15cb70 msgr2=0x7f012c1f2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0130b10700 1 --2- 172.21.15.7:0/2119863532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f012c15cb70 0x7f012c1f2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0130b10700 1 -- 172.21.15.7:0/2119863532 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0118009570 con 0x7f012c06b480 2024-09-13T18:52:30.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0131311700 1 --2- 172.21.15.7:0/2119863532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f012c15cb70 0x7f012c1f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:30.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0130b10700 1 --2- 172.21.15.7:0/2119863532 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f012c06b480 0x7f012c0cc750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f012000c430 tx=0x7f012000c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:30.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f01297fa700 1 -- 172.21.15.7:0/2119863532 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120004a90 con 0x7f012c06b480 2024-09-13T18:52:30.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f01297fa700 1 -- 172.21.15.7:0/2119863532 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0120004bf0 con 0x7f012c06b480 2024-09-13T18:52:30.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0131b12700 1 -- 172.21.15.7:0/2119863532 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012c1f2e90 con 0x7f012c06b480 2024-09-13T18:52:30.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f01297fa700 1 -- 172.21.15.7:0/2119863532 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01200070d0 con 0x7f012c06b480 2024-09-13T18:52:30.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.541+0000 7f0131b12700 1 -- 172.21.15.7:0/2119863532 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012c0ca2c0 con 0x7f012c06b480 2024-09-13T18:52:30.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.545+0000 7f0131b12700 1 -- 172.21.15.7:0/2119863532 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f012c06b8d0 con 0x7f012c06b480 2024-09-13T18:52:30.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.545+0000 7f01297fa700 1 -- 172.21.15.7:0/2119863532 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0120007440 con 0x7f012c06b480 2024-09-13T18:52:30.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.545+0000 7f01297fa700 1 --2- 172.21.15.7:0/2119863532 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0110066730 0x7f0110068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:30.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.545+0000 7f01297fa700 1 -- 172.21.15.7:0/2119863532 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0120084340 con 0x7f012c06b480 2024-09-13T18:52:30.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.545+0000 7f012bfff700 1 --2- 172.21.15.7:0/2119863532 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0110066730 0x7f0110068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:30.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.545+0000 7f012bfff700 1 --2- 172.21.15.7:0/2119863532 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0110066730 0x7f0110068bf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0118000c00 tx=0x7f0118008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:30.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.553+0000 7f01297fa700 1 -- 172.21.15.7:0/2119863532 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0120051690 con 0x7f012c06b480 2024-09-13T18:52:30.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.757+0000 7f0131b12700 1 -- 172.21.15.7:0/2119863532 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f012c0b1ba0 con 0x7f012c06b480 2024-09-13T18:52:30.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.761+0000 7f01297fa700 1 -- 172.21.15.7:0/2119863532 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4725 (secure 0 0 0) 0x7f0120054cb0 con 0x7f012c06b480 2024-09-13T18:52:30.766 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:30.766 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1309419125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1309419125},{"type":"v1","addr":"172.21.15.7:6835","nonce":1309419125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/951828843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":951828843},{"type":"v1","addr":"172.21.15.7:6833","nonce":951828843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4226627580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4226627580},{"type":"v1","addr":"172.21.15.171:6835","nonce":4226627580}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:52:17.908937+0000","modified":"2024-09-13T18:52:20.930515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6837/3658296803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3658296803},{"type":"v1","addr":"172.21.15.171:6837","nonce":3658296803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:52:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.761+0000 7f0131b12700 1 -- 172.21.15.7:0/2119863532 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0110066730 msgr2=0x7f0110068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.761+0000 7f0131b12700 1 --2- 172.21.15.7:0/2119863532 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0110066730 0x7f0110068bf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0118000c00 tx=0x7f0118008030 comp rx=0 tx=0).stop 2024-09-13T18:52:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.761+0000 7f0131b12700 1 -- 172.21.15.7:0/2119863532 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f012c06b480 msgr2=0x7f012c0cc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.761+0000 7f0131b12700 1 --2- 172.21.15.7:0/2119863532 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f012c06b480 0x7f012c0cc750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f012000c430 tx=0x7f012000c740 comp rx=0 tx=0).stop 2024-09-13T18:52:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.761+0000 7f0131b12700 1 -- 172.21.15.7:0/2119863532 shutdown_connections 2024-09-13T18:52:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.761+0000 7f0131b12700 1 --2- 172.21.15.7:0/2119863532 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0110066730 0x7f0110068bf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.761+0000 7f0131b12700 1 --2- 172.21.15.7:0/2119863532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f012c15cb70 0x7f012c1f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.761+0000 7f0131b12700 1 --2- 172.21.15.7:0/2119863532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f012c0d1fa0 0x7f012c0c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.761+0000 7f0131b12700 1 --2- 172.21.15.7:0/2119863532 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f012c06b480 0x7f012c0cc750 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.761+0000 7f0131b12700 1 -- 172.21.15.7:0/2119863532 >> 172.21.15.7:0/2119863532 conn(0x7f012c0cf730 msgr2=0x7f012c0cebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.761+0000 7f0131b12700 1 -- 172.21.15.7:0/2119863532 shutdown_connections 2024-09-13T18:52:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:30.761+0000 7f0131b12700 1 -- 172.21.15.7:0/2119863532 wait complete. 2024-09-13T18:52:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-13T18:52:31.183 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T18:52:17.908937+0000', 'modified': '2024-09-13T18:52:20.930515+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4270}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4270': {'gid': 4270, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.171:6837/3658296803', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 3658296803}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 3658296803}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T18:52:31.183 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T18:52:31.184 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-13T18:52:31.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.305+0000 7fec48a1d700 1 Processor -- start 2024-09-13T18:52:31.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.305+0000 7fec48a1d700 1 -- start start 2024-09-13T18:52:31.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.305+0000 7fec48a1d700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fec440712b0 0x7fec44159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.305+0000 7fec48a1d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fec4415a460 0x7fec4406c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.305+0000 7fec48a1d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec4415ac50 con 0x7fec441573e0 2024-09-13T18:52:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.305+0000 7fec48a1d700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec440cc1b0 con 0x7fec440712b0 2024-09-13T18:52:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.305+0000 7fec48a1d700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec440cb3f0 con 0x7fec4415a460 2024-09-13T18:52:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.305+0000 7fec42d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fec441573e0 0x7fec44070ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57146/0 (socket says 172.21.15.7:57146) 2024-09-13T18:52:31.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.305+0000 7fec42d9d700 1 -- 172.21.15.7:0/1143478571 learned_addr learned my addr 172.21.15.7:0/1143478571 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.305+0000 7fec4259c700 1 --2- 172.21.15.7:0/1143478571 >> v2:172.21.15.171:3300/0 conn(0x7fec440712b0 0x7fec44159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec4359e700 1 --2- 172.21.15.7:0/1143478571 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fec4415a460 0x7fec4406c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec41d9b700 1 -- 172.21.15.7:0/1143478571 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700387970 0 0) 0x7fec4415ac50 con 0x7fec441573e0 2024-09-13T18:52:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec41d9b700 1 -- 172.21.15.7:0/1143478571 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec2c0035d0 con 0x7fec441573e0 2024-09-13T18:52:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec4359e700 1 -- 172.21.15.7:0/1143478571 >> v2:172.21.15.171:3300/0 conn(0x7fec440712b0 msgr2=0x7fec44159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec4359e700 1 --2- 172.21.15.7:0/1143478571 >> v2:172.21.15.171:3300/0 conn(0x7fec440712b0 0x7fec44159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec4359e700 1 -- 172.21.15.7:0/1143478571 >> v1:172.21.15.7:6789/0 conn(0x7fec441573e0 legacy=0x7fec44070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec4359e700 1 -- 172.21.15.7:0/1143478571 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec440cb6d0 con 0x7fec4415a460 2024-09-13T18:52:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec4259c700 1 --2- 172.21.15.7:0/1143478571 >> v2:172.21.15.171:3300/0 conn(0x7fec440712b0 0x7fec44159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec4359e700 1 --2- 172.21.15.7:0/1143478571 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fec4415a460 0x7fec4406c3e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009d90 tx=0x7fec3c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c02d02a915629577 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec41d9b700 1 -- 172.21.15.7:0/1143478571 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c00e070 con 0x7fec4415a460 2024-09-13T18:52:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec41d9b700 1 -- 172.21.15.7:0/1143478571 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c00bb20 con 0x7fec4415a460 2024-09-13T18:52:31.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec41d9b700 1 -- 172.21.15.7:0/1143478571 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c00be90 con 0x7fec4415a460 2024-09-13T18:52:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec48a1d700 1 -- 172.21.15.7:0/1143478571 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fec4415a460 msgr2=0x7fec4406c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec48a1d700 1 --2- 172.21.15.7:0/1143478571 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fec4415a460 0x7fec4406c3e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009d90 tx=0x7fec3c00b330 comp rx=0 tx=0).stop 2024-09-13T18:52:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec48a1d700 1 -- 172.21.15.7:0/1143478571 shutdown_connections 2024-09-13T18:52:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec48a1d700 1 --2- 172.21.15.7:0/1143478571 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fec4415a460 0x7fec4406c3e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec48a1d700 1 --2- 172.21.15.7:0/1143478571 >> v2:172.21.15.171:3300/0 conn(0x7fec440712b0 0x7fec44159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec48a1d700 1 -- 172.21.15.7:0/1143478571 >> 172.21.15.7:0/1143478571 conn(0x7fec440b0e10 msgr2=0x7fec440b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec48a1d700 1 -- 172.21.15.7:0/1143478571 shutdown_connections 2024-09-13T18:52:31.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.309+0000 7fec48a1d700 1 -- 172.21.15.7:0/1143478571 wait complete. 2024-09-13T18:52:31.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec48a1d700 1 Processor -- start 2024-09-13T18:52:31.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec48a1d700 1 -- start start 2024-09-13T18:52:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec48a1d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec440712b0 0x7fec4415c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec48a1d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fec441573e0 0x7fec44151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec48a1d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec4415a460 0x7fec44151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec48a1d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec44153e10 con 0x7fec4415a460 2024-09-13T18:52:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec48a1d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec44153540 con 0x7fec440712b0 2024-09-13T18:52:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec48a1d700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec441f3080 con 0x7fec441573e0 2024-09-13T18:52:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec42d9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec440712b0 0x7fec4415c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec4359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec4415a460 0x7fec44151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec42d9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec440712b0 0x7fec4415c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:39374/0 (socket says 172.21.15.7:39374) 2024-09-13T18:52:31.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec42d9d700 1 -- 172.21.15.7:0/2394619456 learned_addr learned my addr 172.21.15.7:0/2394619456 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:31.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec4259c700 1 --2- 172.21.15.7:0/2394619456 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fec441573e0 0x7fec44151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec4359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec4415a460 0x7fec44151cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59752/0 (socket says 172.21.15.7:59752) 2024-09-13T18:52:31.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec42d9d700 1 -- 172.21.15.7:0/2394619456 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fec441573e0 msgr2=0x7fec44151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:31.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec42d9d700 1 --2- 172.21.15.7:0/2394619456 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fec441573e0 0x7fec44151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec42d9d700 1 -- 172.21.15.7:0/2394619456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec4415a460 msgr2=0x7fec44151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:31.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec42d9d700 1 --2- 172.21.15.7:0/2394619456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec4415a460 0x7fec44151cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec42d9d700 1 -- 172.21.15.7:0/2394619456 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec340094b0 con 0x7fec440712b0 2024-09-13T18:52:31.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec4359e700 1 --2- 172.21.15.7:0/2394619456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec4415a460 0x7fec44151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:31.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec42d9d700 1 --2- 172.21.15.7:0/2394619456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec440712b0 0x7fec4415c0c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fec3800c4f0 tx=0x7fec3800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:31.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec2b7fe700 1 -- 172.21.15.7:0/2394619456 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38005d90 con 0x7fec440712b0 2024-09-13T18:52:31.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec2b7fe700 1 -- 172.21.15.7:0/2394619456 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec380070d0 con 0x7fec440712b0 2024-09-13T18:52:31.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec48a1d700 1 -- 172.21.15.7:0/2394619456 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3c009a70 con 0x7fec440712b0 2024-09-13T18:52:31.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec2b7fe700 1 -- 172.21.15.7:0/2394619456 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38007440 con 0x7fec440712b0 2024-09-13T18:52:31.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.313+0000 7fec48a1d700 1 -- 172.21.15.7:0/2394619456 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec44153a30 con 0x7fec440712b0 2024-09-13T18:52:31.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.317+0000 7fec2b7fe700 1 -- 172.21.15.7:0/2394619456 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fec380075a0 con 0x7fec440712b0 2024-09-13T18:52:31.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.317+0000 7fec2b7fe700 1 --2- 172.21.15.7:0/2394619456 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fec30066730 0x7fec30068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.317+0000 7fec2b7fe700 1 -- 172.21.15.7:0/2394619456 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec38083450 con 0x7fec440712b0 2024-09-13T18:52:31.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.317+0000 7fec4259c700 1 --2- 172.21.15.7:0/2394619456 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fec30066730 0x7fec30068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.321+0000 7fec4259c700 1 --2- 172.21.15.7:0/2394619456 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fec30066730 0x7fec30068bf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fec441efe80 tx=0x7fec34009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:31.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.321+0000 7fec4c4cf740 1 Processor -- start 2024-09-13T18:52:31.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.321+0000 7fec4c4cf740 1 -- start start 2024-09-13T18:52:31.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.321+0000 7fec4c4cf740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1aa8230 0x1aa8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4c4cf740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1a86e90 0x1a70890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4c4cf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 0x1a70dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4c4cf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x19cfd80 con 0x1aa8b50 2024-09-13T18:52:31.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4c4cf740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x19cd510 con 0x1aa8230 2024-09-13T18:52:31.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4c4cf740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x19c8e00 con 0x1a86e90 2024-09-13T18:52:31.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 0x1a70dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 0x1a70dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59768/0 (socket says 172.21.15.7:59768) 2024-09-13T18:52:31.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4359e700 1 -- 172.21.15.7:0/939577347 learned_addr learned my addr 172.21.15.7:0/939577347 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:31.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec42d9d700 1 --2- 172.21.15.7:0/939577347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1aa8230 0x1aa8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4259c700 1 --2- 172.21.15.7:0/939577347 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1a86e90 0x1a70890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4359e700 1 -- 172.21.15.7:0/939577347 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1a86e90 msgr2=0x1a70890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:31.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4359e700 1 --2- 172.21.15.7:0/939577347 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1a86e90 0x1a70890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4359e700 1 -- 172.21.15.7:0/939577347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1aa8230 msgr2=0x1aa8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:31.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4359e700 1 --2- 172.21.15.7:0/939577347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1aa8230 0x1aa8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4359e700 1 -- 172.21.15.7:0/939577347 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x183b960 con 0x1aa8b50 2024-09-13T18:52:31.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4359e700 1 --2- 172.21.15.7:0/939577347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 0x1a70dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fec3c013000 tx=0x7fec3c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90faffbb596d3ef5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:31.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec28ff9700 1 -- 172.21.15.7:0/939577347 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c003e70 con 0x1aa8b50 2024-09-13T18:52:31.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec28ff9700 1 -- 172.21.15.7:0/939577347 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c004040 con 0x1aa8b50 2024-09-13T18:52:31.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec28ff9700 1 -- 172.21.15.7:0/939577347 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0043d0 con 0x1aa8b50 2024-09-13T18:52:31.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4c4cf740 1 -- 172.21.15.7:0/939577347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 msgr2=0x1a70dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:31.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/939577347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 0x1a70dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fec3c013000 tx=0x7fec3c010000 comp rx=0 tx=0).stop 2024-09-13T18:52:31.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4c4cf740 1 -- 172.21.15.7:0/939577347 shutdown_connections 2024-09-13T18:52:31.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/939577347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 0x1a70dd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/939577347 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1a86e90 0x1a70890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/939577347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1aa8230 0x1aa8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4c4cf740 1 -- 172.21.15.7:0/939577347 >> 172.21.15.7:0/939577347 conn(0x17f04d0 msgr2=0x1a89e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4c4cf740 1 -- 172.21.15.7:0/939577347 shutdown_connections 2024-09-13T18:52:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4c4cf740 1 -- 172.21.15.7:0/939577347 wait complete. 2024-09-13T18:52:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.325+0000 7fec4c4cf740 1 -- start start 2024-09-13T18:52:31.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4c4cf740 1 Processor -- start 2024-09-13T18:52:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4c4cf740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1a86e90 0x1a875e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4c4cf740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1aa8230 0x1a860b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4c4cf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 0x1a865f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4c4cf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x1986730 con 0x1aa8b50 2024-09-13T18:52:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4c4cf740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x198c5b0 con 0x1a86e90 2024-09-13T18:52:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4c4cf740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x1874aa0 con 0x1aa8230 2024-09-13T18:52:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4c4cf740 10 client.0 _ll_register_callbacks cb 0x19c2610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:52:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec42d9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1a86e90 0x1a875e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 0x1a865f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec42d9d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1a86e90 0x1a875e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:39392/0 (socket says 172.21.15.7:39392) 2024-09-13T18:52:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec42d9d700 1 -- 172.21.15.7:0/1541576048 learned_addr learned my addr 172.21.15.7:0/1541576048 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4259c700 1 --2- 172.21.15.7:0/1541576048 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1aa8230 0x1a860b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4359e700 1 -- 172.21.15.7:0/1541576048 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1aa8230 msgr2=0x1a860b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4359e700 1 --2- 172.21.15.7:0/1541576048 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1aa8230 0x1a860b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4359e700 1 -- 172.21.15.7:0/1541576048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1a86e90 msgr2=0x1a875e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4359e700 1 --2- 172.21.15.7:0/1541576048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1a86e90 0x1a875e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4359e700 1 -- 172.21.15.7:0/1541576048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec440cb6d0 con 0x1aa8b50 2024-09-13T18:52:31.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4359e700 1 --2- 172.21.15.7:0/1541576048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 0x1a865f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x198afb0 tx=0x7fec3c004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4191ebb89fd1a8e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:31.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-13T18:52:31.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec21ffb700 1 -- 172.21.15.7:0/1541576048 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c004980 con 0x1aa8b50 2024-09-13T18:52:31.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec21ffb700 1 -- 172.21.15.7:0/1541576048 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec3c030680 con 0x1aa8b50 2024-09-13T18:52:31.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4c4cf740 1 -- 172.21.15.7:0/1541576048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x17fd1e0 con 0x1aa8b50 2024-09-13T18:52:31.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec21ffb700 1 -- 172.21.15.7:0/1541576048 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec3c003e70 con 0x1aa8b50 2024-09-13T18:52:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4c4cf740 10 client.4671 fetch_fsmap learned FSMap version 8 2024-09-13T18:52:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4c4cf740 1 -- 172.21.15.7:0/1541576048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x198f220 con 0x1aa8b50 2024-09-13T18:52:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec21ffb700 1 -- 172.21.15.7:0/1541576048 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x17fd1e0 con 0x1aa8b50 2024-09-13T18:52:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec21ffb700 1 -- 172.21.15.7:0/1541576048 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fec3c030840 con 0x1aa8b50 2024-09-13T18:52:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4c4cf740 10 client.4671 fetch_fsmap finished waiting for FSMap version 8 2024-09-13T18:52:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.329+0000 7fec4c4cf740 10 client.4671 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T18:52:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec4c4cf740 20 client.4671 populate_metadata read hostname 'smithi007' 2024-09-13T18:52:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/1541576048 >> [v2:172.21.15.171:6836/3658296803,v1:172.21.15.171:6837/3658296803] conn(0x1a61190 0x1a61570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec4c4cf740 4 client.4671 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:52:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec4c4cf740 1 -- 172.21.15.7:0/1541576048 --> [v2:172.21.15.171:6836/3658296803,v1:172.21.15.171:6837/3658296803] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1847460 con 0x1a61190 2024-09-13T18:52:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec42d9d700 1 --2- 172.21.15.7:0/1541576048 >> [v2:172.21.15.171:6836/3658296803,v1:172.21.15.171:6837/3658296803] conn(0x1a61190 0x1a61570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec42d9d700 1 --2- 172.21.15.7:0/1541576048 >> [v2:172.21.15.171:6836/3658296803,v1:172.21.15.171:6837/3658296803] conn(0x1a61190 0x1a61570 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90d5ca7044667621 server_cookie=6def7e5cd4e5d6e5 in_seq=0 out_seq=0 2024-09-13T18:52:31.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec21ffb700 10 client.4671 ms_handle_connect on v2:172.21.15.171:6836/3658296803 2024-09-13T18:52:31.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec21ffb700 1 -- 172.21.15.7:0/1541576048 <== mds.0 v2:172.21.15.171:6836/3658296803 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x1847460 con 0x1a61190 2024-09-13T18:52:31.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec21ffb700 10 client.4671 handle_command_reply: tid=0 2024-09-13T18:52:31.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec4c4cf740 1 client.4671 shutdown 2024-09-13T18:52:31.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec4c4cf740 1 -- 172.21.15.7:0/1541576048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 msgr2=0x1a865f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:31.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/1541576048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 0x1a865f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x198afb0 tx=0x7fec3c004b60 comp rx=0 tx=0).stop 2024-09-13T18:52:31.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec4c4cf740 1 -- 172.21.15.7:0/1541576048 shutdown_connections 2024-09-13T18:52:31.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/1541576048 >> [v2:172.21.15.171:6836/3658296803,v1:172.21.15.171:6837/3658296803] conn(0x1a61190 0x1a61570 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/1541576048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 0x1a865f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/1541576048 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1aa8230 0x1a860b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec21ffb700 0 client.4671 ms_handle_reset on v2:172.21.15.171:6836/3658296803 2024-09-13T18:52:31.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/1541576048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1a86e90 0x1a875e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec4c4cf740 1 -- 172.21.15.7:0/1541576048 >> 172.21.15.7:0/1541576048 conn(0x17f04d0 msgr2=0x1a5c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:31.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec4c4cf740 1 -- 172.21.15.7:0/1541576048 shutdown_connections 2024-09-13T18:52:31.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec4c4cf740 1 -- 172.21.15.7:0/1541576048 wait complete. 2024-09-13T18:52:31.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.333+0000 7fec4c4cf740 20 client.4671 trim_cache size 0 max 16384 2024-09-13T18:52:31.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4c4cf740 1 Processor -- start 2024-09-13T18:52:31.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4c4cf740 1 -- start start 2024-09-13T18:52:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4c4cf740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1a86e90 0x1ab7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4c4cf740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1aa8230 0x1ab82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4c4cf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 0x1aa2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4c4cf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x19f7ec0 con 0x1aa8b50 2024-09-13T18:52:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4c4cf740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x19934f0 con 0x1aa8230 2024-09-13T18:52:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4c4cf740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x1986730 con 0x1a86e90 2024-09-13T18:52:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 0x1aa2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 0x1aa2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59786/0 (socket says 172.21.15.7:59786) 2024-09-13T18:52:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4359e700 1 -- 172.21.15.7:0/1844068903 learned_addr learned my addr 172.21.15.7:0/1844068903 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec42d9d700 1 --2- 172.21.15.7:0/1844068903 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1a86e90 0x1ab7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4259c700 1 --2- 172.21.15.7:0/1844068903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1aa8230 0x1ab82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4359e700 1 -- 172.21.15.7:0/1844068903 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1a86e90 msgr2=0x1ab7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4359e700 1 --2- 172.21.15.7:0/1844068903 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1a86e90 0x1ab7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4359e700 1 -- 172.21.15.7:0/1844068903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1aa8230 msgr2=0x1ab82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:31.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4359e700 1 --2- 172.21.15.7:0/1844068903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1aa8230 0x1ab82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4359e700 1 -- 172.21.15.7:0/1844068903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1975450 con 0x1aa8b50 2024-09-13T18:52:31.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4259c700 1 --2- 172.21.15.7:0/1844068903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1aa8230 0x1ab82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:31.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec4359e700 1 --2- 172.21.15.7:0/1844068903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 0x1aa2520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00b370 tx=0x7fec3c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf5506c1d133062 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:31.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec297fa700 1 -- 172.21.15.7:0/1844068903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c003df0 con 0x1aa8b50 2024-09-13T18:52:31.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec297fa700 1 -- 172.21.15.7:0/1844068903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c004800 con 0x1aa8b50 2024-09-13T18:52:31.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.357+0000 7fec297fa700 1 -- 172.21.15.7:0/1844068903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c030680 con 0x1aa8b50 2024-09-13T18:52:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4c4cf740 1 -- 172.21.15.7:0/1844068903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 msgr2=0x1aa2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/1844068903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 0x1aa2520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00b370 tx=0x7fec3c02c000 comp rx=0 tx=0).stop 2024-09-13T18:52:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4c4cf740 1 -- 172.21.15.7:0/1844068903 shutdown_connections 2024-09-13T18:52:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/1844068903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1aa8b50 0x1aa2520 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/1844068903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1aa8230 0x1ab82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/1844068903 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1a86e90 0x1ab7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4c4cf740 1 -- 172.21.15.7:0/1844068903 >> 172.21.15.7:0/1844068903 conn(0x17f04d0 msgr2=0x1ab7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4c4cf740 1 -- 172.21.15.7:0/1844068903 shutdown_connections 2024-09-13T18:52:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4c4cf740 1 -- 172.21.15.7:0/1844068903 wait complete. 2024-09-13T18:52:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4c4cf740 1 -- start start 2024-09-13T18:52:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4c4cf740 1 Processor -- start 2024-09-13T18:52:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4c4cf740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1a86e90 0x1ac5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4c4cf740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1aa8230 0x1b08740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4c4cf740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1aa8b50 0x1b08c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4c4cf740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x19c0b70 con 0x1a86e90 2024-09-13T18:52:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4c4cf740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x1967470 con 0x1aa8230 2024-09-13T18:52:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4c4cf740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x1a62f80 con 0x1aa8b50 2024-09-13T18:52:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4c4cf740 10 client.0 _ll_register_callbacks cb 0x197ace0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:52:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec42d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1a86e90 0x1ac5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec42d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1a86e90 0x1ac5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59796/0 (socket says 172.21.15.7:59796) 2024-09-13T18:52:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec42d9d700 1 -- 172.21.15.7:0/145007855 learned_addr learned my addr 172.21.15.7:0/145007855 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4359e700 1 --2- 172.21.15.7:0/145007855 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1aa8b50 0x1b08c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.361+0000 7fec4259c700 1 --2- 172.21.15.7:0/145007855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1aa8230 0x1b08740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec42d9d700 1 -- 172.21.15.7:0/145007855 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1aa8b50 msgr2=0x1b08c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec42d9d700 1 --2- 172.21.15.7:0/145007855 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1aa8b50 0x1b08c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec42d9d700 1 -- 172.21.15.7:0/145007855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1aa8230 msgr2=0x1b08740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec42d9d700 1 --2- 172.21.15.7:0/145007855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1aa8230 0x1b08740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec42d9d700 1 -- 172.21.15.7:0/145007855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x198f220 con 0x1a86e90 2024-09-13T18:52:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4359e700 1 --2- 172.21.15.7:0/145007855 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1aa8b50 0x1b08c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec42d9d700 1 --2- 172.21.15.7:0/145007855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1a86e90 0x1ac5330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fec38000c00 tx=0x7fec38000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9dc0f07557a4a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-13T18:52:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec21ffb700 1 -- 172.21.15.7:0/145007855 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec380582c0 con 0x1a86e90 2024-09-13T18:52:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec21ffb700 1 -- 172.21.15.7:0/145007855 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec3805c880 con 0x1a86e90 2024-09-13T18:52:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 1 -- 172.21.15.7:0/145007855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec3c004040 con 0x1a86e90 2024-09-13T18:52:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec21ffb700 1 -- 172.21.15.7:0/145007855 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec380528e0 con 0x1a86e90 2024-09-13T18:52:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 10 client.4677 fetch_fsmap learned FSMap version 8 2024-09-13T18:52:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 1 -- 172.21.15.7:0/145007855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x1969720 con 0x1a86e90 2024-09-13T18:52:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec21ffb700 1 -- 172.21.15.7:0/145007855 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c004040 con 0x1a86e90 2024-09-13T18:52:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec21ffb700 1 -- 172.21.15.7:0/145007855 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fec380547a0 con 0x1a86e90 2024-09-13T18:52:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 10 client.4677 fetch_fsmap finished waiting for FSMap version 8 2024-09-13T18:52:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 10 client.4677 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T18:52:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 20 client.4677 populate_metadata read hostname 'smithi007' 2024-09-13T18:52:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/145007855 >> [v2:172.21.15.171:6836/3658296803,v1:172.21.15.171:6837/3658296803] conn(0x1aabf90 0x1aac370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 4 client.4677 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:52:31.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 1 -- 172.21.15.7:0/145007855 --> [v2:172.21.15.171:6836/3658296803,v1:172.21.15.171:6837/3658296803] -- command(tid 0: {"prefix": "status"}) v1 -- 0x186c1e0 con 0x1aabf90 2024-09-13T18:52:31.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4259c700 1 --2- 172.21.15.7:0/145007855 >> [v2:172.21.15.171:6836/3658296803,v1:172.21.15.171:6837/3658296803] conn(0x1aabf90 0x1aac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec21ffb700 10 client.4677 ms_handle_connect on v2:172.21.15.171:6836/3658296803 2024-09-13T18:52:31.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4259c700 1 --2- 172.21.15.7:0/145007855 >> [v2:172.21.15.171:6836/3658296803,v1:172.21.15.171:6837/3658296803] conn(0x1aabf90 0x1aac370 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f552950a2c2c635 server_cookie=d24028901b50d1c7 in_seq=0 out_seq=0 2024-09-13T18:52:31.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec21ffb700 1 -- 172.21.15.7:0/145007855 <== mds.0 v2:172.21.15.171:6836/3658296803 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x186c1e0 con 0x1aabf90 2024-09-13T18:52:31.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec21ffb700 10 client.4677 handle_command_reply: tid=0 2024-09-13T18:52:31.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 1 client.4677 shutdown 2024-09-13T18:52:31.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 1 -- 172.21.15.7:0/145007855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1a86e90 msgr2=0x1ac5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:31.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/145007855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1a86e90 0x1ac5330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fec38000c00 tx=0x7fec38000ed0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 1 -- 172.21.15.7:0/145007855 shutdown_connections 2024-09-13T18:52:31.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/145007855 >> [v2:172.21.15.171:6836/3658296803,v1:172.21.15.171:6837/3658296803] conn(0x1aabf90 0x1aac370 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/145007855 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1aa8b50 0x1b08c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec21ffb700 0 client.4677 ms_handle_reset on v2:172.21.15.171:6836/3658296803 2024-09-13T18:52:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/145007855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1aa8230 0x1b08740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 1 --2- 172.21.15.7:0/145007855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x1a86e90 0x1ac5330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 1 -- 172.21.15.7:0/145007855 >> 172.21.15.7:0/145007855 conn(0x17f04d0 msgr2=0x1aa77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.365+0000 7fec4c4cf740 1 -- 172.21.15.7:0/145007855 shutdown_connections 2024-09-13T18:52:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.369+0000 7fec4c4cf740 1 -- 172.21.15.7:0/145007855 wait complete. 2024-09-13T18:52:31.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.369+0000 7fec4c4cf740 20 client.4677 trim_cache size 0 max 16384 2024-09-13T18:52:31.376 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:52:31.376 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "33406d92-f401-438a-876e-a56de3962d8e", 2024-09-13T18:52:31.376 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-13T18:52:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4270, 2024-09-13T18:52:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-13T18:52:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-13T18:52:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-13T18:52:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 13.456966426999999, 2024-09-13T18:52:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 8, 2024-09-13T18:52:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 22, 2024-09-13T18:52:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 22, 2024-09-13T18:52:31.377 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 20.161615461 2024-09-13T18:52:31.377 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:52:31.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.369+0000 7fec48a1d700 1 -- 172.21.15.7:0/2394619456 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fec30066730 msgr2=0x7fec30068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:31.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.369+0000 7fec48a1d700 1 --2- 172.21.15.7:0/2394619456 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fec30066730 0x7fec30068bf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fec441efe80 tx=0x7fec34009240 comp rx=0 tx=0).stop 2024-09-13T18:52:31.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.369+0000 7fec48a1d700 1 -- 172.21.15.7:0/2394619456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec440712b0 msgr2=0x7fec4415c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:31.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.369+0000 7fec48a1d700 1 --2- 172.21.15.7:0/2394619456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec440712b0 0x7fec4415c0c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fec3800c4f0 tx=0x7fec3800c800 comp rx=0 tx=0).stop 2024-09-13T18:52:31.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.369+0000 7fec48a1d700 1 -- 172.21.15.7:0/2394619456 shutdown_connections 2024-09-13T18:52:31.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.369+0000 7fec48a1d700 1 --2- 172.21.15.7:0/2394619456 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fec30066730 0x7fec30068bf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.369+0000 7fec48a1d700 1 --2- 172.21.15.7:0/2394619456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fec4415a460 0x7fec44151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.369+0000 7fec48a1d700 1 --2- 172.21.15.7:0/2394619456 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fec441573e0 0x7fec44151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.369+0000 7fec48a1d700 1 --2- 172.21.15.7:0/2394619456 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fec440712b0 0x7fec4415c0c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.369+0000 7fec48a1d700 1 -- 172.21.15.7:0/2394619456 >> 172.21.15.7:0/2394619456 conn(0x7fec440b0e10 msgr2=0x7fec440aeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.369+0000 7fec48a1d700 1 -- 172.21.15.7:0/2394619456 shutdown_connections 2024-09-13T18:52:31.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.369+0000 7fec48a1d700 1 -- 172.21.15.7:0/2394619456 wait complete. 2024-09-13T18:52:31.393 INFO:teuthology.run_tasks:Running task sequential... 2024-09-13T18:52:31.404 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-13T18:52:31.404 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-13T18:52:31.405 INFO:teuthology.task.print:Enabling mgr modules 2024-09-13T18:52:31.405 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-13T18:52:31.465 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-13T18:52:31.465 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-13T18:52:31.466 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-13T18:52:31.466 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T18:52:31.466 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T18:52:31.466 INFO:tasks.ceph_fuse:client.2 config is {} 2024-09-13T18:52:31.466 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T18:52:31.466 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-13T18:52:31.466 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-13T18:52:31.466 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T18:52:31.466 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-13T18:52:31.466 INFO:tasks.ceph_fuse:client.3 config is {} 2024-09-13T18:52:31.467 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T18:52:31.467 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-13T18:52:31.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:31.467 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list 2024-09-13T18:52:31.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:31.475 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link delete ceph-brx 2024-09-13T18:52:31.494 INFO:teuthology.orchestra.run.smithi171.stderr:Cannot find device "ceph-brx" 2024-09-13T18:52:31.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:52:31.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:31.496 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-13T18:52:31.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:31.504 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-09-13T18:52:31.567 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-09-13T18:52:31.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:52:31.568 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-13T18:52:31.569 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T18:52:31.569 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T18:52:31.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.681+0000 7f812297d700 1 Processor -- start 2024-09-13T18:52:31.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.681+0000 7f812297d700 1 -- start start 2024-09-13T18:52:31.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.681+0000 7f812297d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f811c0cdad0 0x7f811c0cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.681+0000 7f812297d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811c15e080 con 0x7f811c071c90 2024-09-13T18:52:31.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.681+0000 7f812297d700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811c160a60 con 0x7f811c153ba0 2024-09-13T18:52:31.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.681+0000 7f812297d700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811c0ce430 con 0x7f811c0cdad0 2024-09-13T18:52:31.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.681+0000 7f812217c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f811c071c90 0x7f811c15d150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57158/0 (socket says 172.21.15.7:57158) 2024-09-13T18:52:31.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f812217c700 1 -- 172.21.15.7:0/3120578520 learned_addr learned my addr 172.21.15.7:0/3120578520 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:31.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f812117a700 1 --2- 172.21.15.7:0/3120578520 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f811c0cdad0 0x7f811c0cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f8120979700 1 -- 172.21.15.7:0/3120578520 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165814200 0 0) 0x7f811c15e080 con 0x7f811c071c90 2024-09-13T18:52:31.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f8120979700 1 -- 172.21.15.7:0/3120578520 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81080035d0 con 0x7f811c071c90 2024-09-13T18:52:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f812117a700 1 -- 172.21.15.7:0/3120578520 >> v1:172.21.15.171:6789/0 conn(0x7f811c153ba0 legacy=0x7f811c0c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f8120979700 1 -- 172.21.15.7:0/3120578520 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935658316 0 0) 0x7f811c160a60 con 0x7f811c153ba0 2024-09-13T18:52:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f812117a700 1 -- 172.21.15.7:0/3120578520 >> v1:172.21.15.7:6789/0 conn(0x7f811c071c90 legacy=0x7f811c15d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f812117a700 1 -- 172.21.15.7:0/3120578520 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c15dda0 con 0x7f811c0cdad0 2024-09-13T18:52:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f812117a700 1 --2- 172.21.15.7:0/3120578520 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f811c0cdad0 0x7f811c0cdef0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f810c009880 tx=0x7f810c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f906a62762f60f7e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f8120979700 1 -- 172.21.15.7:0/3120578520 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c011830 con 0x7f811c0cdad0 2024-09-13T18:52:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f8120979700 1 -- 172.21.15.7:0/3120578520 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810c011990 con 0x7f811c0cdad0 2024-09-13T18:52:31.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f8120979700 1 -- 172.21.15.7:0/3120578520 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c011d20 con 0x7f811c0cdad0 2024-09-13T18:52:31.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f812297d700 1 -- 172.21.15.7:0/3120578520 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f811c0cdad0 msgr2=0x7f811c0cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:31.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f812297d700 1 --2- 172.21.15.7:0/3120578520 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f811c0cdad0 0x7f811c0cdef0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f810c009880 tx=0x7f810c009b90 comp rx=0 tx=0).stop 2024-09-13T18:52:31.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f812297d700 1 -- 172.21.15.7:0/3120578520 shutdown_connections 2024-09-13T18:52:31.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f812297d700 1 --2- 172.21.15.7:0/3120578520 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f811c0cdad0 0x7f811c0cdef0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f812297d700 1 -- 172.21.15.7:0/3120578520 >> 172.21.15.7:0/3120578520 conn(0x7f811c06a310 msgr2=0x7f811c06a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:31.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f812297d700 1 -- 172.21.15.7:0/3120578520 shutdown_connections 2024-09-13T18:52:31.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.685+0000 7f812297d700 1 -- 172.21.15.7:0/3120578520 wait complete. 2024-09-13T18:52:31.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.689+0000 7f812297d700 1 Processor -- start 2024-09-13T18:52:31.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.689+0000 7f812297d700 1 -- start start 2024-09-13T18:52:31.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812297d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f811c071c90 0x7f811c13f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812297d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f811c0cdad0 0x7f811c15f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812297d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f811c153ba0 0x7f811c157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812297d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c155790 con 0x7f811c071c90 2024-09-13T18:52:31.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812297d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c154b80 con 0x7f811c0cdad0 2024-09-13T18:52:31.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812297d700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811c154e60 con 0x7f811c153ba0 2024-09-13T18:52:31.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812197b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f811c071c90 0x7f811c13f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812197b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f811c071c90 0x7f811c13f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59806/0 (socket says 172.21.15.7:59806) 2024-09-13T18:52:31.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812197b700 1 -- 172.21.15.7:0/1659326210 learned_addr learned my addr 172.21.15.7:0/1659326210 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:31.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812217c700 1 --2- 172.21.15.7:0/1659326210 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f811c153ba0 0x7f811c157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812117a700 1 --2- 172.21.15.7:0/1659326210 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f811c0cdad0 0x7f811c15f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812197b700 1 -- 172.21.15.7:0/1659326210 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f811c153ba0 msgr2=0x7f811c157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812197b700 1 --2- 172.21.15.7:0/1659326210 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f811c153ba0 0x7f811c157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812197b700 1 -- 172.21.15.7:0/1659326210 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f811c0cdad0 msgr2=0x7f811c15f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812197b700 1 --2- 172.21.15.7:0/1659326210 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f811c0cdad0 0x7f811c15f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812197b700 1 -- 172.21.15.7:0/1659326210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810c0094b0 con 0x7f811c071c90 2024-09-13T18:52:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812197b700 1 --2- 172.21.15.7:0/1659326210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f811c071c90 0x7f811c13f7b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f811800c520 tx=0x7f811800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f81127fc700 1 -- 172.21.15.7:0/1659326210 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81180054e0 con 0x7f811c071c90 2024-09-13T18:52:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f81127fc700 1 -- 172.21.15.7:0/1659326210 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81180036a0 con 0x7f811c071c90 2024-09-13T18:52:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812297d700 1 -- 172.21.15.7:0/1659326210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811c155140 con 0x7f811c071c90 2024-09-13T18:52:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f81127fc700 1 -- 172.21.15.7:0/1659326210 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118003a50 con 0x7f811c071c90 2024-09-13T18:52:31.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812297d700 1 -- 172.21.15.7:0/1659326210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811c15fad0 con 0x7f811c071c90 2024-09-13T18:52:31.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.693+0000 7f812297d700 1 -- 172.21.15.7:0/1659326210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811c1615c0 con 0x7f811c071c90 2024-09-13T18:52:31.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.697+0000 7f81127fc700 1 -- 172.21.15.7:0/1659326210 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8118003bb0 con 0x7f811c071c90 2024-09-13T18:52:31.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.697+0000 7f81127fc700 1 --2- 172.21.15.7:0/1659326210 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8100066730 0x7f8100068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:31.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.697+0000 7f81127fc700 1 -- 172.21.15.7:0/1659326210 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8118083510 con 0x7f811c071c90 2024-09-13T18:52:31.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.697+0000 7f812117a700 1 --2- 172.21.15.7:0/1659326210 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8100066730 0x7f8100068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:31.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.697+0000 7f812117a700 1 --2- 172.21.15.7:0/1659326210 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8100066730 0x7f8100068bf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f810c0090b0 tx=0x7f810c01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:31.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.705+0000 7f81127fc700 1 -- 172.21.15.7:0/1659326210 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8118050880 con 0x7f811c071c90 2024-09-13T18:52:31.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.913+0000 7f812297d700 1 -- 172.21.15.7:0/1659326210 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f811c0cb1e0 con 0x7f811c071c90 2024-09-13T18:52:31.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.913+0000 7f81127fc700 1 -- 172.21.15.7:0/1659326210 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8118017060 con 0x7f811c071c90 2024-09-13T18:52:31.918 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T18:52:31.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.913+0000 7f812297d700 1 -- 172.21.15.7:0/1659326210 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8100066730 msgr2=0x7f8100068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:31.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.913+0000 7f812297d700 1 --2- 172.21.15.7:0/1659326210 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8100066730 0x7f8100068bf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f810c0090b0 tx=0x7f810c01c040 comp rx=0 tx=0).stop 2024-09-13T18:52:31.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.913+0000 7f812297d700 1 -- 172.21.15.7:0/1659326210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f811c071c90 msgr2=0x7f811c13f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:31.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.913+0000 7f812297d700 1 --2- 172.21.15.7:0/1659326210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f811c071c90 0x7f811c13f7b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f811800c520 tx=0x7f811800c830 comp rx=0 tx=0).stop 2024-09-13T18:52:31.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.913+0000 7f812297d700 1 -- 172.21.15.7:0/1659326210 shutdown_connections 2024-09-13T18:52:31.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.913+0000 7f812297d700 1 --2- 172.21.15.7:0/1659326210 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8100066730 0x7f8100068bf0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.913+0000 7f812297d700 1 --2- 172.21.15.7:0/1659326210 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f811c153ba0 0x7f811c157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.913+0000 7f812297d700 1 --2- 172.21.15.7:0/1659326210 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f811c0cdad0 0x7f811c15f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.913+0000 7f812297d700 1 --2- 172.21.15.7:0/1659326210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f811c071c90 0x7f811c13f7b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:31.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.913+0000 7f812297d700 1 -- 172.21.15.7:0/1659326210 >> 172.21.15.7:0/1659326210 conn(0x7f811c06a310 msgr2=0x7f811c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:31.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.913+0000 7f812297d700 1 -- 172.21.15.7:0/1659326210 shutdown_connections 2024-09-13T18:52:31.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:31.913+0000 7f812297d700 1 -- 172.21.15.7:0/1659326210 wait complete. 2024-09-13T18:52:31.934 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T18:52:31.934 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T18:52:31.934 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-13T18:52:31.934 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T18:52:31.935 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T18:52:31.935 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T18:52:31.935 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T18:52:31.935 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T18:52:31.935 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T18:52:31.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:31.935 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-13T18:52:31.943 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T18:52:31.943 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T18:52:31.943 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T18:52:31.943 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:31.943 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-13T18:52:31.944 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:31.946 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:52:31.946 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:31.946 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:52:31.946 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:31.946 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T18:52:31.946 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:31.947 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T18:52:31.947 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42560sec preferred_lft 42560sec 2024-09-13T18:52:31.947 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-13T18:52:31.947 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:31.947 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:52:31.948 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:31.948 INFO:teuthology.orchestra.run.smithi007.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T18:52:31.948 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 02:42:8e:73:06:85 brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:31.948 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T18:52:31.948 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:31.949 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T18:52:31.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:31.950 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-09-13T18:52:31.950 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-13T18:52:31.950 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-09-13T18:52:31.950 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-09-13T18:52:31.950 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-09-13T18:52:31.950 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T18:52:31.951 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-13T18:52:31.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:31.998 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T18:52:32.007 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-13T18:52:32.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:32.008 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-13T18:52:32.039 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-13T18:52:32.039 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T18:52:32.039 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T18:52:32.039 INFO:teuthology.orchestra.run.smithi007.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T18:52:32.039 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T18:52:32.039 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T18:52:32.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:32.040 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-09-13T18:52:32.040 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-13T18:52:32.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T18:52:32.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T18:52:32.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T18:52:32.040 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-13T18:52:32.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:32.081 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-13T18:52:32.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:32.132 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-09-13T18:52:32.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:32.140 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-09-13T18:52:32.140 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-13T18:52:32.140 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T18:52:32.140 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-13T18:52:32.140 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-13T18:52:32.171 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-13T18:52:32.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:32.171 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-09-13T18:52:32.171 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-13T18:52:32.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-13T18:52:32.172 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T18:52:32.172 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-13T18:52:32.172 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-13T18:52:32.172 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-13T18:52:32.172 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-13T18:52:32.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:32.253 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-09-13T18:52:32.253 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-13T18:52:32.254 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-13T18:52:32.254 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T18:52:32.254 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-13T18:52:32.281 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-13T18:52:32.281 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:52:32.281 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T18:52:32.286 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T18:52:32.286 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:52:32.286 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T18:52:32.293 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-09-13T18:52:32.354 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:52:32.354 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:52:32.369 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:52:32.369 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:52:32.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:32.370 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-09-13T18:52:32.377 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T18:52:32.377 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-13T18:52:32.379 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-09-13T18:52:32.394 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:52:32.394 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:52:32.450 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:52:32.451 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:52:32.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:32.452 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-09-13T18:52:32.463 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:2024-09-13T18:52:32.457+0000 7f8eea885640 -1 init, newargv = 0x5566e3360940 newargc=15 2024-09-13T18:52:32.464 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[32465]: starting ceph client 2024-09-13T18:52:32.473 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[32465]: starting fuse 2024-09-13T18:52:33.559 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-09-13T18:52:33.578 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:52:33.578 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:52:33.637 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:52:33.637 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:52:33.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:33.638 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-09-13T18:52:33.645 INFO:teuthology.orchestra.run.smithi007.stdout:52 2024-09-13T18:52:33.646 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-09-13T18:52:33.646 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:52:33.646 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2024-09-13T18:52:33.646 DEBUG:teuthology.orchestra.run.smithi007:> import re 2024-09-13T18:52:33.646 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-13T18:52:33.646 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-09-13T18:52:33.646 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:52:33.646 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2024-09-13T18:52:33.647 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T18:52:33.647 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2024-09-13T18:52:33.647 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T18:52:33.647 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:52:33.647 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2024-09-13T18:52:33.647 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2024-09-13T18:52:33.647 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2024-09-13T18:52:33.647 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2024-09-13T18:52:33.647 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:52:33.647 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2024-09-13T18:52:33.647 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T18:52:33.647 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T18:52:33.647 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T18:52:33.647 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2024-09-13T18:52:33.647 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2024-09-13T18:52:33.648 DEBUG:teuthology.orchestra.run.smithi007:> return f 2024-09-13T18:52:33.648 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T18:52:33.648 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:52:33.648 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.0")) 2024-09-13T18:52:33.648 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-13T18:52:33.732 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.0.32465.asok 2024-09-13T18:52:33.916 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.32465.asok 2024-09-13T18:52:33.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:33.916 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.32465.asok status 2024-09-13T18:52:34.017 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:52:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2024-09-13T18:52:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T18:52:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T18:52:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "0", 2024-09-13T18:52:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2024-09-13T18:52:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T18:52:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "32465", 2024-09-13T18:52:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2024-09-13T18:52:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:52:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2024-09-13T18:52:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2024-09-13T18:52:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4692, 2024-09-13T18:52:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2024-09-13T18:52:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2024-09-13T18:52:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-09-13T18:52:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4692 2024-09-13T18:52:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:52:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-09-13T18:52:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-13T18:52:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2024-09-13T18:52:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1060397471 2024-09-13T18:52:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:52:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:52:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-09-13T18:52:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-13T18:52:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2024-09-13T18:52:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1060397471 2024-09-13T18:52:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:52:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.4692 192.168.0.1:0/1060397471", 2024-09-13T18:52:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.1:0/1060397471", 2024-09-13T18:52:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2024-09-13T18:52:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 8, 2024-09-13T18:52:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 22, 2024-09-13T18:52:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2024-09-13T18:52:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2024-09-13T18:52:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2024-09-13T18:52:34.021 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:52:34.027 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T18:52:34.028 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T18:52:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.133+0000 7f4d1709c700 1 Processor -- start 2024-09-13T18:52:34.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.133+0000 7f4d1709c700 1 -- start start 2024-09-13T18:52:34.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.133+0000 7f4d1709c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4d100cf300 0x7f4d100b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:34.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.133+0000 7f4d1709c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d100cf6e0 con 0x7f4d100b4d00 2024-09-13T18:52:34.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.133+0000 7f4d1709c700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d100b2b20 con 0x7f4d100b5190 2024-09-13T18:52:34.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.133+0000 7f4d1709c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d100b2de0 con 0x7f4d100cf300 2024-09-13T18:52:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15899700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4d100b4d00 0x7f4d10070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57172/0 (socket says 172.21.15.7:57172) 2024-09-13T18:52:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15899700 1 -- 172.21.15.7:0/4206063798 learned_addr learned my addr 172.21.15.7:0/4206063798 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d1609a700 1 --2- 172.21.15.7:0/4206063798 >> v2:172.21.15.7:3300/0 conn(0x7f4d100cf300 0x7f4d100b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15098700 1 -- 172.21.15.7:0/4206063798 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4113345960 0 0) 0x7f4d100cf6e0 con 0x7f4d100b4d00 2024-09-13T18:52:34.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15098700 1 -- 172.21.15.7:0/4206063798 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cfc0035d0 con 0x7f4d100b4d00 2024-09-13T18:52:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15098700 1 -- 172.21.15.7:0/4206063798 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1113004670 0 0) 0x7f4cfc0035d0 con 0x7f4d100b4d00 2024-09-13T18:52:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15098700 1 -- 172.21.15.7:0/4206063798 >> v2:172.21.15.7:3300/0 conn(0x7f4d100cf300 msgr2=0x7f4d100b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15098700 1 --2- 172.21.15.7:0/4206063798 >> v2:172.21.15.7:3300/0 conn(0x7f4d100cf300 0x7f4d100b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15098700 1 -- 172.21.15.7:0/4206063798 >> v1:172.21.15.171:6789/0 conn(0x7f4d100b5190 legacy=0x7f4d100ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15098700 1 -- 172.21.15.7:0/4206063798 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d100b30c0 con 0x7f4d100b4d00 2024-09-13T18:52:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15098700 1 -- 172.21.15.7:0/4206063798 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7f4d04001670 con 0x7f4d100b4d00 2024-09-13T18:52:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15098700 1 -- 172.21.15.7:0/4206063798 >> v1:172.21.15.7:6789/0 conn(0x7f4d100b4d00 legacy=0x7f4d10070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15098700 1 --2- 172.21.15.7:0/4206063798 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4cfc003a20 0x7f4cfc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15098700 1 --2- 172.21.15.7:0/4206063798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cfc004420 0x7f4cfc0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15098700 1 --2- 172.21.15.7:0/4206063798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc0050f0 0x7f4cfc00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15098700 1 -- 172.21.15.7:0/4206063798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100b2de0 con 0x7f4cfc0050f0 2024-09-13T18:52:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15098700 1 -- 172.21.15.7:0/4206063798 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc00c050 con 0x7f4cfc004420 2024-09-13T18:52:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15098700 1 -- 172.21.15.7:0/4206063798 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc00c210 con 0x7f4cfc003a20 2024-09-13T18:52:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d1689b700 1 --2- 172.21.15.7:0/4206063798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc0050f0 0x7f4cfc00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15899700 1 --2- 172.21.15.7:0/4206063798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cfc004420 0x7f4cfc0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d1609a700 1 --2- 172.21.15.7:0/4206063798 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4cfc003a20 0x7f4cfc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d1689b700 1 -- 172.21.15.7:0/4206063798 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4cfc003a20 msgr2=0x7f4cfc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d1689b700 1 --2- 172.21.15.7:0/4206063798 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4cfc003a20 0x7f4cfc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d1689b700 1 -- 172.21.15.7:0/4206063798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cfc004420 msgr2=0x7f4cfc0048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d1689b700 1 --2- 172.21.15.7:0/4206063798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4cfc004420 0x7f4cfc0048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d1689b700 1 -- 172.21.15.7:0/4206063798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc0035d0 con 0x7f4cfc0050f0 2024-09-13T18:52:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d1689b700 1 -- 172.21.15.7:0/4206063798 reap_dead start 2024-09-13T18:52:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d1689b700 1 --2- 172.21.15.7:0/4206063798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc0050f0 0x7f4cfc00ba30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800c190 tx=0x7f4cfc0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6433a86ee620ea77 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15098700 1 -- 172.21.15.7:0/4206063798 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08001db0 con 0x7f4cfc0050f0 2024-09-13T18:52:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15098700 1 -- 172.21.15.7:0/4206063798 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0800ccf0 con 0x7f4cfc0050f0 2024-09-13T18:52:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d15098700 1 -- 172.21.15.7:0/4206063798 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08003c60 con 0x7f4cfc0050f0 2024-09-13T18:52:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.137+0000 7f4d1709c700 1 -- 172.21.15.7:0/4206063798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc0050f0 msgr2=0x7f4cfc00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.141+0000 7f4d1709c700 1 --2- 172.21.15.7:0/4206063798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc0050f0 0x7f4cfc00ba30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800c190 tx=0x7f4cfc0058a0 comp rx=0 tx=0).stop 2024-09-13T18:52:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.141+0000 7f4d1709c700 1 -- 172.21.15.7:0/4206063798 shutdown_connections 2024-09-13T18:52:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.141+0000 7f4d1709c700 1 --2- 172.21.15.7:0/4206063798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc0050f0 0x7f4cfc00ba30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.141+0000 7f4d1709c700 1 -- 172.21.15.7:0/4206063798 >> 172.21.15.7:0/4206063798 conn(0x7f4d100b16a0 msgr2=0x7f4d100b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.141+0000 7f4d1709c700 1 -- 172.21.15.7:0/4206063798 shutdown_connections 2024-09-13T18:52:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.141+0000 7f4d1709c700 1 -- 172.21.15.7:0/4206063798 wait complete. 2024-09-13T18:52:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.141+0000 7f4d1709c700 1 Processor -- start 2024-09-13T18:52:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.141+0000 7f4d1709c700 1 -- start start 2024-09-13T18:52:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d1709c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc0050f0 0x7f4d1006b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d1709c700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4d10153310 0x7f4d1006cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d1709c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d10146d90 0x7f4d10160560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d1709c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1015e150 con 0x7f4cfc0050f0 2024-09-13T18:52:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d1709c700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d10153770 con 0x7f4d10146d90 2024-09-13T18:52:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d1709c700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1006bc00 con 0x7f4d10153310 2024-09-13T18:52:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d1609a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc0050f0 0x7f4d1006b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d1609a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc0050f0 0x7f4d1006b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59840/0 (socket says 172.21.15.7:59840) 2024-09-13T18:52:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d1609a700 1 -- 172.21.15.7:0/463453988 learned_addr learned my addr 172.21.15.7:0/463453988 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d1689b700 1 --2- 172.21.15.7:0/463453988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d10146d90 0x7f4d10160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d15899700 1 --2- 172.21.15.7:0/463453988 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4d10153310 0x7f4d1006cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d1609a700 1 -- 172.21.15.7:0/463453988 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4d10153310 msgr2=0x7f4d1006cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d1609a700 1 --2- 172.21.15.7:0/463453988 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4d10153310 0x7f4d1006cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d1609a700 1 -- 172.21.15.7:0/463453988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d10146d90 msgr2=0x7f4d10160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d1609a700 1 --2- 172.21.15.7:0/463453988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d10146d90 0x7f4d10160560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d1609a700 1 -- 172.21.15.7:0/463453988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1015e410 con 0x7f4cfc0050f0 2024-09-13T18:52:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d1609a700 1 --2- 172.21.15.7:0/463453988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc0050f0 0x7f4d1006b5b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0119b0 tx=0x7f4d0c011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d02ffd700 1 -- 172.21.15.7:0/463453988 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c011e40 con 0x7f4cfc0050f0 2024-09-13T18:52:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d02ffd700 1 -- 172.21.15.7:0/463453988 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c00f940 con 0x7f4cfc0050f0 2024-09-13T18:52:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d1709c700 1 -- 172.21.15.7:0/463453988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1015e6f0 con 0x7f4cfc0050f0 2024-09-13T18:52:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d02ffd700 1 -- 172.21.15.7:0/463453988 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c00fcd0 con 0x7f4cfc0050f0 2024-09-13T18:52:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.145+0000 7f4d1709c700 1 -- 172.21.15.7:0/463453988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1006d4d0 con 0x7f4cfc0050f0 2024-09-13T18:52:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.149+0000 7f4d02ffd700 1 -- 172.21.15.7:0/463453988 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4d0c014440 con 0x7f4cfc0050f0 2024-09-13T18:52:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.149+0000 7f4d1709c700 1 -- 172.21.15.7:0/463453988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d100b3ba0 con 0x7f4cfc0050f0 2024-09-13T18:52:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.149+0000 7f4d02ffd700 1 --2- 172.21.15.7:0/463453988 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4cf4066760 0x7f4cf4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:34.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.149+0000 7f4d02ffd700 1 -- 172.21.15.7:0/463453988 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d0c084820 con 0x7f4cfc0050f0 2024-09-13T18:52:34.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.149+0000 7f4d15899700 1 --2- 172.21.15.7:0/463453988 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4cf4066760 0x7f4cf4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:34.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.153+0000 7f4d15899700 1 --2- 172.21.15.7:0/463453988 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4cf4066760 0x7f4cf4068c20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4d04000c00 tx=0x7f4d04008ce0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:34.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.157+0000 7f4d02ffd700 1 -- 172.21.15.7:0/463453988 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4d0c051a70 con 0x7f4cfc0050f0 2024-09-13T18:52:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.369+0000 7f4d1709c700 1 -- 172.21.15.7:0/463453988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4d1006e5e0 con 0x7f4cfc0050f0 2024-09-13T18:52:34.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.369+0000 7f4d02ffd700 1 -- 172.21.15.7:0/463453988 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4d0c055090 con 0x7f4cfc0050f0 2024-09-13T18:52:34.374 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T18:52:34.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.369+0000 7f4d1709c700 1 -- 172.21.15.7:0/463453988 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4cf4066760 msgr2=0x7f4cf4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:34.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.369+0000 7f4d1709c700 1 --2- 172.21.15.7:0/463453988 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4cf4066760 0x7f4cf4068c20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4d04000c00 tx=0x7f4d04008ce0 comp rx=0 tx=0).stop 2024-09-13T18:52:34.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.369+0000 7f4d1709c700 1 -- 172.21.15.7:0/463453988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc0050f0 msgr2=0x7f4d1006b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:34.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.369+0000 7f4d1709c700 1 --2- 172.21.15.7:0/463453988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc0050f0 0x7f4d1006b5b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0119b0 tx=0x7f4d0c011cc0 comp rx=0 tx=0).stop 2024-09-13T18:52:34.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.369+0000 7f4d1709c700 1 -- 172.21.15.7:0/463453988 shutdown_connections 2024-09-13T18:52:34.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.369+0000 7f4d1709c700 1 --2- 172.21.15.7:0/463453988 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4cf4066760 0x7f4cf4068c20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:34.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.369+0000 7f4d1709c700 1 --2- 172.21.15.7:0/463453988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4d10146d90 0x7f4d10160560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:34.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.369+0000 7f4d1709c700 1 --2- 172.21.15.7:0/463453988 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4d10153310 0x7f4d1006cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:34.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.369+0000 7f4d1709c700 1 --2- 172.21.15.7:0/463453988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cfc0050f0 0x7f4d1006b5b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:34.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.369+0000 7f4d1709c700 1 -- 172.21.15.7:0/463453988 >> 172.21.15.7:0/463453988 conn(0x7f4d100b16a0 msgr2=0x7f4d100af610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:34.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.369+0000 7f4d1709c700 1 -- 172.21.15.7:0/463453988 shutdown_connections 2024-09-13T18:52:34.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:34.369+0000 7f4d1709c700 1 -- 172.21.15.7:0/463453988 wait complete. 2024-09-13T18:52:34.390 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T18:52:34.390 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T18:52:34.390 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-13T18:52:34.390 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-13T18:52:34.390 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-13T18:52:34.390 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T18:52:34.390 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T18:52:34.390 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T18:52:34.390 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-13T18:52:34.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:34.391 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-13T18:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T18:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T18:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T18:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-13T18:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T18:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T18:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42558sec preferred_lft 42558sec 2024-09-13T18:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-13T18:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T18:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 02:42:8e:73:06:85 brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T18:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T18:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 72:5b:90:5f:a5:2f brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T18:52:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::9896:a1ff:fe28:24ca/64 scope link 2024-09-13T18:52:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:34.401 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T18:52:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 72:5b:90:5f:a5:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T18:52:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::705b:90ff:fe5f:a52f/64 scope link 2024-09-13T18:52:34.401 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:34.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:34.402 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T18:52:34.415 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-13T18:52:34.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:34.417 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-13T18:52:34.474 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-13T18:52:34.474 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T18:52:34.474 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T18:52:34.474 INFO:teuthology.orchestra.run.smithi007.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T18:52:34.474 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T18:52:34.474 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T18:52:34.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:34.475 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-09-13T18:52:34.475 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-13T18:52:34.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T18:52:34.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T18:52:34.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T18:52:34.475 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-13T18:52:34.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:34.516 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-13T18:52:34.523 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T18:52:34.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:34.524 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-09-13T18:52:34.532 INFO:teuthology.orchestra.run.smithi007.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-13T18:52:34.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:34.533 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-09-13T18:52:34.533 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-13T18:52:34.533 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T18:52:34.533 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-13T18:52:34.533 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-13T18:52:34.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:34.609 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-13T18:52:34.622 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T18:52:34.622 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T18:52:34.622 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T18:52:34.622 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:34.622 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-13T18:52:34.622 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:34.622 INFO:teuthology.orchestra.run.smithi007.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T18:52:34.622 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ea:c0:78:10:96:8c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-13T18:52:34.623 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-13T18:52:34.623 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:34.623 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::e8c0:78ff:fe10:968c/64 scope link 2024-09-13T18:52:34.623 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:34.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:34.624 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-13T18:52:34.636 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T18:52:34.636 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T18:52:34.636 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T18:52:34.636 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:34.637 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-13T18:52:34.637 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:34.637 INFO:teuthology.orchestra.run.smithi007.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T18:52:34.637 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ea:c0:78:10:96:8c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-13T18:52:34.637 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-13T18:52:34.637 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:34.637 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::e8c0:78ff:fe10:968c/64 scope link 2024-09-13T18:52:34.637 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:34.638 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-13T18:52:34.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:34.638 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-09-13T18:52:34.638 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-13T18:52:34.638 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-13T18:52:34.639 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-13T18:52:34.639 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-13T18:52:34.639 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-13T18:52:34.639 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-13T18:52:34.639 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-13T18:52:34.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:34.743 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-09-13T18:52:34.743 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-13T18:52:34.743 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.1 up 2024-09-13T18:52:34.744 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.1 master ceph-brx 2024-09-13T18:52:34.744 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-13T18:52:34.771 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-09-13T18:52:34.771 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:52:34.771 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-13T18:52:34.776 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-13T18:52:34.776 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:52:34.777 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-13T18:52:34.781 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-09-13T18:52:34.846 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:52:34.846 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:52:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:52:34.862 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:52:34.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:34.863 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-09-13T18:52:34.867 INFO:teuthology.orchestra.run.smithi007.stdout:52 2024-09-13T18:52:34.868 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2024-09-13T18:52:34.868 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-09-13T18:52:34.870 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-09-13T18:52:34.888 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:52:34.889 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:52:34.898 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:52:34.898 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:52:34.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:34.899 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-09-13T18:52:34.947 INFO:teuthology.orchestra.run.smithi007.stdout:52 2024-09-13T18:52:34.958 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi007.stderr:2024-09-13T18:52:34.953+0000 7f1322984640 -1 init, newargv = 0x562cdb358460 newargc=15 2024-09-13T18:52:34.959 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi007.stderr:ceph-fuse[32608]: starting ceph client 2024-09-13T18:52:34.969 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi007.stderr:ceph-fuse[32608]: starting fuse 2024-09-13T18:52:36.049 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-09-13T18:52:36.069 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:52:36.069 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:52:36.084 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:52:36.085 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:52:36.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:36.085 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-09-13T18:52:36.135 INFO:teuthology.orchestra.run.smithi007.stdout:52 2024-09-13T18:52:36.136 INFO:teuthology.orchestra.run.smithi007.stdout:53 2024-09-13T18:52:36.136 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2024-09-13T18:52:36.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:52:36.136 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2024-09-13T18:52:36.136 DEBUG:teuthology.orchestra.run.smithi007:> import re 2024-09-13T18:52:36.137 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-13T18:52:36.137 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-09-13T18:52:36.137 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:52:36.137 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2024-09-13T18:52:36.137 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-09-13T18:52:36.137 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2024-09-13T18:52:36.137 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-09-13T18:52:36.137 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:52:36.137 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2024-09-13T18:52:36.137 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2024-09-13T18:52:36.137 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2024-09-13T18:52:36.137 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2024-09-13T18:52:36.137 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:52:36.137 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2024-09-13T18:52:36.138 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T18:52:36.138 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T18:52:36.138 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T18:52:36.138 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2024-09-13T18:52:36.138 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2024-09-13T18:52:36.138 DEBUG:teuthology.orchestra.run.smithi007:> return f 2024-09-13T18:52:36.138 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T18:52:36.138 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:52:36.138 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.2")) 2024-09-13T18:52:36.138 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-13T18:52:36.224 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.2.32608.asok 2024-09-13T18:52:36.406 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.32608.asok 2024-09-13T18:52:36.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:36.406 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.32608.asok status 2024-09-13T18:52:36.514 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:52:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2024-09-13T18:52:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T18:52:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T18:52:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "2", 2024-09-13T18:52:36.514 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2024-09-13T18:52:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-09-13T18:52:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "32608", 2024-09-13T18:52:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2024-09-13T18:52:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:52:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2024-09-13T18:52:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2024-09-13T18:52:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4710, 2024-09-13T18:52:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2024-09-13T18:52:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2024-09-13T18:52:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-09-13T18:52:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4710 2024-09-13T18:52:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:52:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-09-13T18:52:36.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-13T18:52:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.2:0", 2024-09-13T18:52:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2450087846 2024-09-13T18:52:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:52:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:52:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-09-13T18:52:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-13T18:52:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.2:0", 2024-09-13T18:52:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2450087846 2024-09-13T18:52:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:52:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.4710 192.168.0.2:0/2450087846", 2024-09-13T18:52:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.2:0/2450087846", 2024-09-13T18:52:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2024-09-13T18:52:36.516 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 8, 2024-09-13T18:52:36.517 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 22, 2024-09-13T18:52:36.517 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2024-09-13T18:52:36.517 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2024-09-13T18:52:36.517 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2024-09-13T18:52:36.517 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:52:36.524 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T18:52:36.524 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T18:52:36.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.637+0000 7fee6efe9700 1 Processor -- start 2024-09-13T18:52:36.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.637+0000 7fee6efe9700 1 -- start start 2024-09-13T18:52:36.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.637+0000 7fee6efe9700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fee680b3420 0x7fee680b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:36.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.637+0000 7fee6efe9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee6806ade0 con 0x7fee6815ae40 2024-09-13T18:52:36.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.637+0000 7fee6efe9700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee680b3d40 con 0x7fee6806b3c0 2024-09-13T18:52:36.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.637+0000 7fee6efe9700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee6806c040 con 0x7fee680b3420 2024-09-13T18:52:36.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.637+0000 7fee6e7e8700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fee6815ae40 0x7fee6815b250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57186/0 (socket says 172.21.15.7:57186) 2024-09-13T18:52:36.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.637+0000 7fee6e7e8700 1 -- 172.21.15.7:0/667844694 learned_addr learned my addr 172.21.15.7:0/667844694 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:36.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.637+0000 7fee6dfe7700 1 --2- 172.21.15.7:0/667844694 >> v2:172.21.15.171:3300/0 conn(0x7fee680b3420 0x7fee680b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:36.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.637+0000 7fee6cfe5700 1 -- 172.21.15.7:0/667844694 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570076118 0 0) 0x7fee6806ade0 con 0x7fee6815ae40 2024-09-13T18:52:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.637+0000 7fee6cfe5700 1 -- 172.21.15.7:0/667844694 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee540035d0 con 0x7fee6815ae40 2024-09-13T18:52:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.637+0000 7fee6cfe5700 1 -- 172.21.15.7:0/667844694 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822616375 0 0) 0x7fee680b3d40 con 0x7fee6806b3c0 2024-09-13T18:52:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.637+0000 7fee6cfe5700 1 -- 172.21.15.7:0/667844694 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee6806ade0 con 0x7fee6806b3c0 2024-09-13T18:52:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.637+0000 7fee6dfe7700 1 -- 172.21.15.7:0/667844694 >> v1:172.21.15.171:6789/0 conn(0x7fee6806b3c0 legacy=0x7fee6806b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.637+0000 7fee6dfe7700 1 -- 172.21.15.7:0/667844694 >> v1:172.21.15.7:6789/0 conn(0x7fee6815ae40 legacy=0x7fee6815b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.637+0000 7fee6dfe7700 1 -- 172.21.15.7:0/667844694 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6806a4b0 con 0x7fee680b3420 2024-09-13T18:52:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.641+0000 7fee6dfe7700 1 --2- 172.21.15.7:0/667844694 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee680b3420 0x7fee680b3800 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009850 tx=0x7fee5c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85814f9ec32173cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.641+0000 7fee6cfe5700 1 -- 172.21.15.7:0/667844694 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c011830 con 0x7fee680b3420 2024-09-13T18:52:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.641+0000 7fee6cfe5700 1 -- 172.21.15.7:0/667844694 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c011990 con 0x7fee680b3420 2024-09-13T18:52:36.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.641+0000 7fee6cfe5700 1 -- 172.21.15.7:0/667844694 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c011d40 con 0x7fee680b3420 2024-09-13T18:52:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.641+0000 7fee6efe9700 1 -- 172.21.15.7:0/667844694 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee680b3420 msgr2=0x7fee680b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.641+0000 7fee6efe9700 1 --2- 172.21.15.7:0/667844694 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee680b3420 0x7fee680b3800 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009850 tx=0x7fee5c009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.641+0000 7fee6efe9700 1 -- 172.21.15.7:0/667844694 shutdown_connections 2024-09-13T18:52:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.641+0000 7fee6efe9700 1 --2- 172.21.15.7:0/667844694 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee680b3420 0x7fee680b3800 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.641+0000 7fee6efe9700 1 -- 172.21.15.7:0/667844694 >> 172.21.15.7:0/667844694 conn(0x7fee680b0f00 msgr2=0x7fee680b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.641+0000 7fee6efe9700 1 -- 172.21.15.7:0/667844694 shutdown_connections 2024-09-13T18:52:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.641+0000 7fee6efe9700 1 -- 172.21.15.7:0/667844694 wait complete. 2024-09-13T18:52:36.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.641+0000 7fee6efe9700 1 Processor -- start 2024-09-13T18:52:36.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.641+0000 7fee6efe9700 1 -- start start 2024-09-13T18:52:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6efe9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee6806b3c0 0x7fee681f2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6efe9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee680b3420 0x7fee68157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6efe9700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fee6815ae40 0x7fee68154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6efe9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee68158480 con 0x7fee680b3420 2024-09-13T18:52:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6efe9700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee68158230 con 0x7fee6806b3c0 2024-09-13T18:52:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6efe9700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee680cb6e0 con 0x7fee6815ae40 2024-09-13T18:52:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6d7e6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee680b3420 0x7fee68157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6d7e6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee680b3420 0x7fee68157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59854/0 (socket says 172.21.15.7:59854) 2024-09-13T18:52:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6d7e6700 1 -- 172.21.15.7:0/3023843312 learned_addr learned my addr 172.21.15.7:0/3023843312 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6e7e8700 1 --2- 172.21.15.7:0/3023843312 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fee6815ae40 0x7fee68154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6dfe7700 1 --2- 172.21.15.7:0/3023843312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee6806b3c0 0x7fee681f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6d7e6700 1 -- 172.21.15.7:0/3023843312 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fee6815ae40 msgr2=0x7fee68154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6d7e6700 1 --2- 172.21.15.7:0/3023843312 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fee6815ae40 0x7fee68154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6d7e6700 1 -- 172.21.15.7:0/3023843312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee6806b3c0 msgr2=0x7fee681f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6d7e6700 1 --2- 172.21.15.7:0/3023843312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee6806b3c0 0x7fee681f2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6d7e6700 1 -- 172.21.15.7:0/3023843312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c0094b0 con 0x7fee680b3420 2024-09-13T18:52:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6d7e6700 1 --2- 172.21.15.7:0/3023843312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee680b3420 0x7fee68157cf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee64001f00 tx=0x7fee6400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:36.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee5a7fc700 1 -- 172.21.15.7:0/3023843312 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64002100 con 0x7fee680b3420 2024-09-13T18:52:36.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee5a7fc700 1 -- 172.21.15.7:0/3023843312 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6400ade0 con 0x7fee680b3420 2024-09-13T18:52:36.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6efe9700 1 -- 172.21.15.7:0/3023843312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee680cb980 con 0x7fee680b3420 2024-09-13T18:52:36.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee5a7fc700 1 -- 172.21.15.7:0/3023843312 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64003c60 con 0x7fee680b3420 2024-09-13T18:52:36.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6efe9700 1 -- 172.21.15.7:0/3023843312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee680cbf40 con 0x7fee680b3420 2024-09-13T18:52:36.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.645+0000 7fee6efe9700 1 -- 172.21.15.7:0/3023843312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee680b3800 con 0x7fee680b3420 2024-09-13T18:52:36.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.649+0000 7fee5a7fc700 1 -- 172.21.15.7:0/3023843312 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fee640050d0 con 0x7fee680b3420 2024-09-13T18:52:36.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.649+0000 7fee5a7fc700 1 --2- 172.21.15.7:0/3023843312 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fee4c066730 0x7fee4c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:36.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.649+0000 7fee5a7fc700 1 -- 172.21.15.7:0/3023843312 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee64084420 con 0x7fee680b3420 2024-09-13T18:52:36.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.649+0000 7fee6dfe7700 1 --2- 172.21.15.7:0/3023843312 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fee4c066730 0x7fee4c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:36.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.649+0000 7fee6dfe7700 1 --2- 172.21.15.7:0/3023843312 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fee4c066730 0x7fee4c068bf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0090b0 tx=0x7fee5c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.653+0000 7fee5a7fc700 1 -- 172.21.15.7:0/3023843312 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fee640516f0 con 0x7fee680b3420 2024-09-13T18:52:36.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.861+0000 7fee6efe9700 1 -- 172.21.15.7:0/3023843312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fee68155240 con 0x7fee680b3420 2024-09-13T18:52:36.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.865+0000 7fee5a7fc700 1 -- 172.21.15.7:0/3023843312 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fee64054d10 con 0x7fee680b3420 2024-09-13T18:52:36.869 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T18:52:36.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.865+0000 7fee6efe9700 1 -- 172.21.15.7:0/3023843312 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fee4c066730 msgr2=0x7fee4c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:36.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.865+0000 7fee6efe9700 1 --2- 172.21.15.7:0/3023843312 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fee4c066730 0x7fee4c068bf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0090b0 tx=0x7fee5c008030 comp rx=0 tx=0).stop 2024-09-13T18:52:36.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.865+0000 7fee6efe9700 1 -- 172.21.15.7:0/3023843312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee680b3420 msgr2=0x7fee68157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:36.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.865+0000 7fee6efe9700 1 --2- 172.21.15.7:0/3023843312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee680b3420 0x7fee68157cf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee64001f00 tx=0x7fee6400a340 comp rx=0 tx=0).stop 2024-09-13T18:52:36.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.865+0000 7fee6efe9700 1 -- 172.21.15.7:0/3023843312 shutdown_connections 2024-09-13T18:52:36.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.865+0000 7fee6efe9700 1 --2- 172.21.15.7:0/3023843312 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fee4c066730 0x7fee4c068bf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:36.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.865+0000 7fee6efe9700 1 --2- 172.21.15.7:0/3023843312 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fee6815ae40 0x7fee68154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:36.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.865+0000 7fee6efe9700 1 --2- 172.21.15.7:0/3023843312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee680b3420 0x7fee68157cf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:36.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.865+0000 7fee6efe9700 1 --2- 172.21.15.7:0/3023843312 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee6806b3c0 0x7fee681f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:36.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.865+0000 7fee6efe9700 1 -- 172.21.15.7:0/3023843312 >> 172.21.15.7:0/3023843312 conn(0x7fee680b0f00 msgr2=0x7fee6815ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:36.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.865+0000 7fee6efe9700 1 -- 172.21.15.7:0/3023843312 shutdown_connections 2024-09-13T18:52:36.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:36.865+0000 7fee6efe9700 1 -- 172.21.15.7:0/3023843312 wait complete. 2024-09-13T18:52:36.885 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T18:52:36.885 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T18:52:36.885 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi171.front.sepia.ceph.com 2024-09-13T18:52:36.885 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-13T18:52:36.885 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-13T18:52:36.885 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T18:52:36.885 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T18:52:36.885 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T18:52:36.885 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-13T18:52:36.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:36.886 DEBUG:teuthology.orchestra.run.smithi171:> ip addr 2024-09-13T18:52:36.894 INFO:teuthology.orchestra.run.smithi171.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T18:52:36.894 INFO:teuthology.orchestra.run.smithi171.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T18:52:36.894 INFO:teuthology.orchestra.run.smithi171.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T18:52:36.894 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:36.894 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 ::1/128 scope host 2024-09-13T18:52:36.894 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:36.894 INFO:teuthology.orchestra.run.smithi171.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:52:36.894 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:d9:39:a2 brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:36.894 INFO:teuthology.orchestra.run.smithi171.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:52:36.894 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:d9:39:a3 brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:36.894 INFO:teuthology.orchestra.run.smithi171.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:52:36.895 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:88:6f:6c brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:36.895 INFO:teuthology.orchestra.run.smithi171.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T18:52:36.895 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:88:6f:6d brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:36.895 INFO:teuthology.orchestra.run.smithi171.stdout: inet 172.21.15.171/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T18:52:36.895 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft 42563sec preferred_lft 42563sec 2024-09-13T18:52:36.895 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 fe80::ec4:7aff:fe88:6f6d/64 scope link 2024-09-13T18:52:36.895 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:36.895 INFO:teuthology.orchestra.run.smithi171.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T18:52:36.895 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 02:42:11:0b:d1:0b brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:36.895 INFO:teuthology.orchestra.run.smithi171.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T18:52:36.895 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:36.895 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T18:52:36.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:36.896 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T18:52:36.896 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T18:52:36.896 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link add name ceph-brx type bridge 2024-09-13T18:52:36.896 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip addr flush dev ceph-brx 2024-09-13T18:52:36.896 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link set ceph-brx up 2024-09-13T18:52:36.896 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T18:52:36.896 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T18:52:36.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:36.986 DEBUG:teuthology.orchestra.run.smithi171:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T18:52:36.998 INFO:teuthology.orchestra.run.smithi171.stdout:1 2024-09-13T18:52:36.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:36.999 DEBUG:teuthology.orchestra.run.smithi171:> route 2024-09-13T18:52:37.040 INFO:teuthology.orchestra.run.smithi171.stdout:Kernel IP routing table 2024-09-13T18:52:37.040 INFO:teuthology.orchestra.run.smithi171.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T18:52:37.041 INFO:teuthology.orchestra.run.smithi171.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T18:52:37.041 INFO:teuthology.orchestra.run.smithi171.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T18:52:37.041 INFO:teuthology.orchestra.run.smithi171.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T18:52:37.041 INFO:teuthology.orchestra.run.smithi171.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-09-13T18:52:37.041 INFO:teuthology.orchestra.run.smithi171.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T18:52:37.041 INFO:teuthology.orchestra.run.smithi171.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T18:52:37.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:37.041 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T18:52:37.042 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T18:52:37.042 DEBUG:teuthology.orchestra.run.smithi171:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T18:52:37.042 DEBUG:teuthology.orchestra.run.smithi171:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T18:52:37.042 DEBUG:teuthology.orchestra.run.smithi171:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T18:52:37.042 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T18:52:37.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:37.082 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list 2024-09-13T18:52:37.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:37.087 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list-id 2024-09-13T18:52:37.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:37.135 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T18:52:37.135 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T18:52:37.135 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T18:52:37.135 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-13T18:52:37.135 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T18:52:37.168 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-13T18:52:37.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:37.168 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T18:52:37.169 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T18:52:37.169 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-13T18:52:37.169 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T18:52:37.169 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-13T18:52:37.169 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-13T18:52:37.169 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-13T18:52:37.169 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T18:52:37.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:37.237 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T18:52:37.237 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T18:52:37.237 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link set brx.0 up 2024-09-13T18:52:37.238 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T18:52:37.238 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T18:52:37.260 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-13T18:52:37.261 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:52:37.261 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-13T18:52:37.266 INFO:teuthology.orchestra.run.smithi171.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-13T18:52:37.266 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:52:37.266 DEBUG:teuthology.orchestra.run.smithi171:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-13T18:52:37.272 DEBUG:teuthology.orchestra.run.smithi171:> sudo modprobe fuse 2024-09-13T18:52:37.287 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:52:37.287 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:52:37.297 INFO:teuthology.orchestra.run.smithi171.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:52:37.298 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:52:37.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:37.299 DEBUG:teuthology.orchestra.run.smithi171:> ls /sys/fs/fuse/connections 2024-09-13T18:52:37.303 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T18:52:37.304 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-13T18:52:37.306 DEBUG:teuthology.orchestra.run.smithi171:> sudo modprobe fuse 2024-09-13T18:52:37.322 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:52:37.322 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:52:37.379 INFO:teuthology.orchestra.run.smithi171.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:52:37.381 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:52:37.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:37.381 DEBUG:teuthology.orchestra.run.smithi171:> ls /sys/fs/fuse/connections 2024-09-13T18:52:37.390 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi171.stderr:2024-09-13T18:52:37.385+0000 7f81aa4b6640 -1 init, newargv = 0x5607c044c6f0 newargc=15 2024-09-13T18:52:37.390 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi171.stderr:ceph-fuse[32267]: starting ceph client 2024-09-13T18:52:37.398 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi171.stderr:ceph-fuse[32267]: starting fuse 2024-09-13T18:52:38.489 DEBUG:teuthology.orchestra.run.smithi171:> sudo modprobe fuse 2024-09-13T18:52:38.508 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:52:38.508 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:52:38.567 INFO:teuthology.orchestra.run.smithi171.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:52:38.568 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:52:38.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:38.569 DEBUG:teuthology.orchestra.run.smithi171:> ls /sys/fs/fuse/connections 2024-09-13T18:52:38.573 INFO:teuthology.orchestra.run.smithi171.stdout:52 2024-09-13T18:52:38.574 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-09-13T18:52:38.574 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:52:38.574 DEBUG:teuthology.orchestra.run.smithi171:> import glob 2024-09-13T18:52:38.574 DEBUG:teuthology.orchestra.run.smithi171:> import re 2024-09-13T18:52:38.574 DEBUG:teuthology.orchestra.run.smithi171:> import os 2024-09-13T18:52:38.574 DEBUG:teuthology.orchestra.run.smithi171:> import subprocess 2024-09-13T18:52:38.574 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T18:52:38.574 DEBUG:teuthology.orchestra.run.smithi171:> def _find_admin_socket(client_name): 2024-09-13T18:52:38.574 DEBUG:teuthology.orchestra.run.smithi171:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-13T18:52:38.575 DEBUG:teuthology.orchestra.run.smithi171:> files = glob.glob(asok_path) 2024-09-13T18:52:38.575 DEBUG:teuthology.orchestra.run.smithi171:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-13T18:52:38.575 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T18:52:38.575 DEBUG:teuthology.orchestra.run.smithi171:> # Given a non-glob path, it better be there 2024-09-13T18:52:38.575 DEBUG:teuthology.orchestra.run.smithi171:> if "*" not in asok_path: 2024-09-13T18:52:38.575 DEBUG:teuthology.orchestra.run.smithi171:> assert(len(files) == 1) 2024-09-13T18:52:38.575 DEBUG:teuthology.orchestra.run.smithi171:> return files[0] 2024-09-13T18:52:38.575 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T18:52:38.575 DEBUG:teuthology.orchestra.run.smithi171:> for f in files: 2024-09-13T18:52:38.575 DEBUG:teuthology.orchestra.run.smithi171:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T18:52:38.575 DEBUG:teuthology.orchestra.run.smithi171:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T18:52:38.575 DEBUG:teuthology.orchestra.run.smithi171:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T18:52:38.575 DEBUG:teuthology.orchestra.run.smithi171:> contents = proc_f.read() 2024-09-13T18:52:38.575 DEBUG:teuthology.orchestra.run.smithi171:> if mountpoint in contents: 2024-09-13T18:52:38.575 DEBUG:teuthology.orchestra.run.smithi171:> return f 2024-09-13T18:52:38.576 DEBUG:teuthology.orchestra.run.smithi171:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T18:52:38.576 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T18:52:38.576 DEBUG:teuthology.orchestra.run.smithi171:> print(_find_admin_socket("client.1")) 2024-09-13T18:52:38.576 DEBUG:teuthology.orchestra.run.smithi171:> ' 2024-09-13T18:52:38.656 INFO:teuthology.orchestra.run.smithi171.stdout:/var/run/ceph/ceph-client.1.32267.asok 2024-09-13T18:52:38.840 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.32267.asok 2024-09-13T18:52:38.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:38.841 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.32267.asok status 2024-09-13T18:52:38.989 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-13T18:52:38.989 INFO:teuthology.orchestra.run.smithi171.stdout: "metadata": { 2024-09-13T18:52:38.989 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T18:52:38.989 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T18:52:38.989 INFO:teuthology.orchestra.run.smithi171.stdout: "entity_id": "1", 2024-09-13T18:52:38.989 INFO:teuthology.orchestra.run.smithi171.stdout: "hostname": "smithi171", 2024-09-13T18:52:38.989 INFO:teuthology.orchestra.run.smithi171.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-13T18:52:38.989 INFO:teuthology.orchestra.run.smithi171.stdout: "pid": "32267", 2024-09-13T18:52:38.989 INFO:teuthology.orchestra.run.smithi171.stdout: "root": "/" 2024-09-13T18:52:38.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T18:52:38.990 INFO:teuthology.orchestra.run.smithi171.stdout: "dentry_count": 0, 2024-09-13T18:52:38.990 INFO:teuthology.orchestra.run.smithi171.stdout: "dentry_pinned_count": 0, 2024-09-13T18:52:38.990 INFO:teuthology.orchestra.run.smithi171.stdout: "id": 4654, 2024-09-13T18:52:38.990 INFO:teuthology.orchestra.run.smithi171.stdout: "inst": { 2024-09-13T18:52:38.990 INFO:teuthology.orchestra.run.smithi171.stdout: "name": { 2024-09-13T18:52:38.990 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "client", 2024-09-13T18:52:38.990 INFO:teuthology.orchestra.run.smithi171.stdout: "num": 4654 2024-09-13T18:52:38.990 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T18:52:38.990 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": { 2024-09-13T18:52:38.990 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-13T18:52:38.991 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "192.168.0.1:0", 2024-09-13T18:52:38.991 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1800046652 2024-09-13T18:52:38.991 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-13T18:52:38.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T18:52:38.991 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": { 2024-09-13T18:52:38.991 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-13T18:52:38.991 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "192.168.0.1:0", 2024-09-13T18:52:38.991 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 1800046652 2024-09-13T18:52:38.991 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T18:52:38.991 INFO:teuthology.orchestra.run.smithi171.stdout: "inst_str": "client.4654 192.168.0.1:0/1800046652", 2024-09-13T18:52:38.991 INFO:teuthology.orchestra.run.smithi171.stdout: "addr_str": "192.168.0.1:0/1800046652", 2024-09-13T18:52:38.992 INFO:teuthology.orchestra.run.smithi171.stdout: "inode_count": 1, 2024-09-13T18:52:38.992 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_epoch": 8, 2024-09-13T18:52:38.992 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_epoch": 22, 2024-09-13T18:52:38.992 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_epoch_barrier": 0, 2024-09-13T18:52:38.992 INFO:teuthology.orchestra.run.smithi171.stdout: "blocklisted": false, 2024-09-13T18:52:38.992 INFO:teuthology.orchestra.run.smithi171.stdout: "fs_name": "cephfs" 2024-09-13T18:52:38.992 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-13T18:52:38.998 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T18:52:38.999 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T18:52:39.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.117+0000 7f1cf6bcf700 1 Processor -- start 2024-09-13T18:52:39.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.117+0000 7f1cf6bcf700 1 -- start start 2024-09-13T18:52:39.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf6bcf700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f1ce804ec10 0x7f1ce801db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:39.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf6bcf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce8033e90 con 0x7f1ce8036d80 2024-09-13T18:52:39.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf6bcf700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce8033bb0 con 0x7f1ce8033510 2024-09-13T18:52:39.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf6bcf700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce801d080 con 0x7f1ce804ec10 2024-09-13T18:52:39.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf53cc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1ce8036d80 0x7f1ce804eff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57196/0 (socket says 172.21.15.7:57196) 2024-09-13T18:52:39.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf53cc700 1 -- 172.21.15.7:0/1858745812 learned_addr learned my addr 172.21.15.7:0/1858745812 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf5bcd700 1 --2- 172.21.15.7:0/1858745812 >> v2:172.21.15.171:3300/0 conn(0x7f1ce804ec10 0x7f1ce801db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf4bcb700 1 -- 172.21.15.7:0/1858745812 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602287760 0 0) 0x7f1ce8033e90 con 0x7f1ce8036d80 2024-09-13T18:52:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf4bcb700 1 -- 172.21.15.7:0/1858745812 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cdc0035d0 con 0x7f1ce8036d80 2024-09-13T18:52:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf5bcd700 1 -- 172.21.15.7:0/1858745812 >> v1:172.21.15.171:6789/0 conn(0x7f1ce8033510 legacy=0x7f1ce8031240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf5bcd700 1 -- 172.21.15.7:0/1858745812 >> v1:172.21.15.7:6789/0 conn(0x7f1ce8036d80 legacy=0x7f1ce804eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf5bcd700 1 -- 172.21.15.7:0/1858745812 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce801d360 con 0x7f1ce804ec10 2024-09-13T18:52:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf5bcd700 1 --2- 172.21.15.7:0/1858745812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ce804ec10 0x7f1ce801db60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009840 tx=0x7f1cec009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c05e29274a4a3015 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf4bcb700 1 -- 172.21.15.7:0/1858745812 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec005e50 con 0x7f1ce804ec10 2024-09-13T18:52:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf4bcb700 1 -- 172.21.15.7:0/1858745812 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec006030 con 0x7f1ce804ec10 2024-09-13T18:52:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf4bcb700 1 -- 172.21.15.7:0/1858745812 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0063c0 con 0x7f1ce804ec10 2024-09-13T18:52:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf6bcf700 1 -- 172.21.15.7:0/1858745812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ce804ec10 msgr2=0x7f1ce801db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf6bcf700 1 --2- 172.21.15.7:0/1858745812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ce804ec10 0x7f1ce801db60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009840 tx=0x7f1cec009b50 comp rx=0 tx=0).stop 2024-09-13T18:52:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf6bcf700 1 -- 172.21.15.7:0/1858745812 shutdown_connections 2024-09-13T18:52:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf6bcf700 1 --2- 172.21.15.7:0/1858745812 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ce804ec10 0x7f1ce801db60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf6bcf700 1 -- 172.21.15.7:0/1858745812 >> 172.21.15.7:0/1858745812 conn(0x7f1ce8036140 msgr2=0x7f1ce8036550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf6bcf700 1 -- 172.21.15.7:0/1858745812 shutdown_connections 2024-09-13T18:52:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.121+0000 7f1cf6bcf700 1 -- 172.21.15.7:0/1858745812 wait complete. 2024-09-13T18:52:39.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf6bcf700 1 Processor -- start 2024-09-13T18:52:39.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf6bcf700 1 -- start start 2024-09-13T18:52:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf6bcf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ce8033510 0x7f1ce80d9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf6bcf700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1ce8036d80 0x7f1ce80e8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf6bcf700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ce804ec10 0x7f1ce8049d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf6bcf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce80e9c10 con 0x7f1ce8033510 2024-09-13T18:52:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf6bcf700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce8048250 con 0x7f1ce804ec10 2024-09-13T18:52:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf6bcf700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce80da180 con 0x7f1ce8036d80 2024-09-13T18:52:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf5bcd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ce8033510 0x7f1ce80d9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf5bcd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ce8033510 0x7f1ce80d9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59860/0 (socket says 172.21.15.7:59860) 2024-09-13T18:52:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf5bcd700 1 -- 172.21.15.7:0/3481708212 learned_addr learned my addr 172.21.15.7:0/3481708212 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf53cc700 1 --2- 172.21.15.7:0/3481708212 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1ce8036d80 0x7f1ce80e8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf63ce700 1 --2- 172.21.15.7:0/3481708212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ce804ec10 0x7f1ce8049d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:39.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf5bcd700 1 -- 172.21.15.7:0/3481708212 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1ce8036d80 msgr2=0x7f1ce80e8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf5bcd700 1 --2- 172.21.15.7:0/3481708212 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1ce8036d80 0x7f1ce80e8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf5bcd700 1 -- 172.21.15.7:0/3481708212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ce804ec10 msgr2=0x7f1ce8049d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf5bcd700 1 --2- 172.21.15.7:0/3481708212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ce804ec10 0x7f1ce8049d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf5bcd700 1 -- 172.21.15.7:0/3481708212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec009450 con 0x7f1ce8033510 2024-09-13T18:52:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.125+0000 7f1cf5bcd700 1 --2- 172.21.15.7:0/3481708212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ce8033510 0x7f1ce80d9c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009b90 tx=0x7f1cec004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.129+0000 7f1ce67fc700 1 -- 172.21.15.7:0/3481708212 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0038c0 con 0x7f1ce8033510 2024-09-13T18:52:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.129+0000 7f1ce67fc700 1 -- 172.21.15.7:0/3481708212 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec003a20 con 0x7f1ce8033510 2024-09-13T18:52:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.129+0000 7f1ce67fc700 1 -- 172.21.15.7:0/3481708212 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec003dd0 con 0x7f1ce8033510 2024-09-13T18:52:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.129+0000 7f1cf6bcf700 1 -- 172.21.15.7:0/3481708212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce80e9930 con 0x7f1ce8033510 2024-09-13T18:52:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.129+0000 7f1cf6bcf700 1 -- 172.21.15.7:0/3481708212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce8047ee0 con 0x7f1ce8033510 2024-09-13T18:52:39.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.129+0000 7f1cf6bcf700 1 -- 172.21.15.7:0/3481708212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce8004800 con 0x7f1ce8033510 2024-09-13T18:52:39.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.129+0000 7f1ce67fc700 1 -- 172.21.15.7:0/3481708212 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1cec022070 con 0x7f1ce8033510 2024-09-13T18:52:39.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.129+0000 7f1ce67fc700 1 --2- 172.21.15.7:0/3481708212 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1cd4066760 0x7f1cd4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:39.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.133+0000 7f1ce67fc700 1 -- 172.21.15.7:0/3481708212 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cec0858e0 con 0x7f1ce8033510 2024-09-13T18:52:39.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.133+0000 7f1cf53cc700 1 --2- 172.21.15.7:0/3481708212 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1cd4066760 0x7f1cd4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:39.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.133+0000 7f1cf53cc700 1 --2- 172.21.15.7:0/3481708212 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1cd4066760 0x7f1cd4068c20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1ce804e8c0 tx=0x7f1ce0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:39.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.137+0000 7f1ce67fc700 1 -- 172.21.15.7:0/3481708212 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1cec052ce0 con 0x7f1ce8033510 2024-09-13T18:52:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.345+0000 7f1cf6bcf700 1 -- 172.21.15.7:0/3481708212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1ce801db60 con 0x7f1ce8033510 2024-09-13T18:52:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.345+0000 7f1ce67fc700 1 -- 172.21.15.7:0/3481708212 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1cec02a060 con 0x7f1ce8033510 2024-09-13T18:52:39.353 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T18:52:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.349+0000 7f1cf6bcf700 1 -- 172.21.15.7:0/3481708212 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1cd4066760 msgr2=0x7f1cd4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.349+0000 7f1cf6bcf700 1 --2- 172.21.15.7:0/3481708212 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1cd4066760 0x7f1cd4068c20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1ce804e8c0 tx=0x7f1ce0009040 comp rx=0 tx=0).stop 2024-09-13T18:52:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.349+0000 7f1cf6bcf700 1 -- 172.21.15.7:0/3481708212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ce8033510 msgr2=0x7f1ce80d9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.349+0000 7f1cf6bcf700 1 --2- 172.21.15.7:0/3481708212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ce8033510 0x7f1ce80d9c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009b90 tx=0x7f1cec004770 comp rx=0 tx=0).stop 2024-09-13T18:52:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.349+0000 7f1cf6bcf700 1 -- 172.21.15.7:0/3481708212 shutdown_connections 2024-09-13T18:52:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.349+0000 7f1cf6bcf700 1 --2- 172.21.15.7:0/3481708212 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1cd4066760 0x7f1cd4068c20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.349+0000 7f1cf6bcf700 1 --2- 172.21.15.7:0/3481708212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ce804ec10 0x7f1ce8049d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.349+0000 7f1cf6bcf700 1 --2- 172.21.15.7:0/3481708212 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1ce8036d80 0x7f1ce80e8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.349+0000 7f1cf6bcf700 1 --2- 172.21.15.7:0/3481708212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ce8033510 0x7f1ce80d9c40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.349+0000 7f1cf6bcf700 1 -- 172.21.15.7:0/3481708212 >> 172.21.15.7:0/3481708212 conn(0x7f1ce8036140 msgr2=0x7f1ce8031240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.349+0000 7f1cf6bcf700 1 -- 172.21.15.7:0/3481708212 shutdown_connections 2024-09-13T18:52:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:39.349+0000 7f1cf6bcf700 1 -- 172.21.15.7:0/3481708212 wait complete. 2024-09-13T18:52:39.367 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T18:52:39.367 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T18:52:39.367 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi171.front.sepia.ceph.com 2024-09-13T18:52:39.367 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-13T18:52:39.367 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-13T18:52:39.367 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T18:52:39.368 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T18:52:39.368 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T18:52:39.368 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-13T18:52:39.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:39.368 DEBUG:teuthology.orchestra.run.smithi171:> ip addr 2024-09-13T18:52:39.376 INFO:teuthology.orchestra.run.smithi171.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T18:52:39.377 INFO:teuthology.orchestra.run.smithi171.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T18:52:39.377 INFO:teuthology.orchestra.run.smithi171.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T18:52:39.377 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:39.377 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 ::1/128 scope host 2024-09-13T18:52:39.377 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:39.377 INFO:teuthology.orchestra.run.smithi171.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:52:39.377 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:d9:39:a2 brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:39.377 INFO:teuthology.orchestra.run.smithi171.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:52:39.377 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:d9:39:a3 brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:39.377 INFO:teuthology.orchestra.run.smithi171.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:52:39.377 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:88:6f:6c brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:39.377 INFO:teuthology.orchestra.run.smithi171.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T18:52:39.377 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 0c:c4:7a:88:6f:6d brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:39.378 INFO:teuthology.orchestra.run.smithi171.stdout: inet 172.21.15.171/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T18:52:39.378 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft 42561sec preferred_lft 42561sec 2024-09-13T18:52:39.378 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 fe80::ec4:7aff:fe88:6f6d/64 scope link 2024-09-13T18:52:39.378 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:39.378 INFO:teuthology.orchestra.run.smithi171.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T18:52:39.378 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether 02:42:11:0b:d1:0b brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:39.378 INFO:teuthology.orchestra.run.smithi171.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T18:52:39.378 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:39.378 INFO:teuthology.orchestra.run.smithi171.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T18:52:39.379 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether fa:05:f6:0d:45:b9 brd ff:ff:ff:ff:ff:ff 2024-09-13T18:52:39.379 INFO:teuthology.orchestra.run.smithi171.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T18:52:39.379 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:39.379 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 fe80::309e:bbff:fe31:f9c4/64 scope link 2024-09-13T18:52:39.379 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:39.379 INFO:teuthology.orchestra.run.smithi171.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T18:52:39.379 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether fa:05:f6:0d:45:b9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T18:52:39.379 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 fe80::f805:f6ff:fe0d:45b9/64 scope link 2024-09-13T18:52:39.379 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:39.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:39.379 DEBUG:teuthology.orchestra.run.smithi171:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T18:52:39.394 INFO:teuthology.orchestra.run.smithi171.stdout:1 2024-09-13T18:52:39.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:39.396 DEBUG:teuthology.orchestra.run.smithi171:> route 2024-09-13T18:52:39.448 INFO:teuthology.orchestra.run.smithi171.stdout:Kernel IP routing table 2024-09-13T18:52:39.449 INFO:teuthology.orchestra.run.smithi171.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T18:52:39.449 INFO:teuthology.orchestra.run.smithi171.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T18:52:39.449 INFO:teuthology.orchestra.run.smithi171.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T18:52:39.449 INFO:teuthology.orchestra.run.smithi171.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T18:52:39.449 INFO:teuthology.orchestra.run.smithi171.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2024-09-13T18:52:39.449 INFO:teuthology.orchestra.run.smithi171.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T18:52:39.449 INFO:teuthology.orchestra.run.smithi171.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T18:52:39.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:39.449 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T18:52:39.449 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T18:52:39.450 DEBUG:teuthology.orchestra.run.smithi171:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T18:52:39.450 DEBUG:teuthology.orchestra.run.smithi171:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T18:52:39.450 DEBUG:teuthology.orchestra.run.smithi171:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T18:52:39.450 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T18:52:39.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:39.491 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list 2024-09-13T18:52:39.497 INFO:teuthology.orchestra.run.smithi171.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-13T18:52:39.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:39.498 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list-id 2024-09-13T18:52:39.505 INFO:teuthology.orchestra.run.smithi171.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-13T18:52:39.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:39.505 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T18:52:39.505 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T18:52:39.505 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-13T18:52:39.505 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-13T18:52:39.505 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T18:52:39.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:39.535 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-13T18:52:39.594 INFO:teuthology.orchestra.run.smithi171.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T18:52:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T18:52:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T18:52:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 ::1/128 scope host 2024-09-13T18:52:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:39.594 INFO:teuthology.orchestra.run.smithi171.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T18:52:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether ee:32:b9:ed:64:65 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-13T18:52:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-13T18:52:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:39.594 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 fe80::ec32:b9ff:feed:6465/64 scope link 2024-09-13T18:52:39.595 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:39.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:39.596 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-13T18:52:39.608 INFO:teuthology.orchestra.run.smithi171.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T18:52:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T18:52:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T18:52:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 ::1/128 scope host 2024-09-13T18:52:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:39.609 INFO:teuthology.orchestra.run.smithi171.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T18:52:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: link/ether ee:32:b9:ed:64:65 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-13T18:52:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-13T18:52:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: inet6 fe80::ec32:b9ff:feed:6465/64 scope link 2024-09-13T18:52:39.609 INFO:teuthology.orchestra.run.smithi171.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:52:39.610 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-13T18:52:39.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:39.611 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T18:52:39.611 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T18:52:39.611 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-13T18:52:39.611 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-13T18:52:39.611 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-13T18:52:39.611 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-13T18:52:39.611 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-13T18:52:39.611 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T18:52:39.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:39.666 DEBUG:teuthology.orchestra.run.smithi171:> (cd / && exec bash -c ' 2024-09-13T18:52:39.666 DEBUG:teuthology.orchestra.run.smithi171:> set -e 2024-09-13T18:52:39.666 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link set brx.1 up 2024-09-13T18:52:39.666 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link set dev brx.1 master ceph-brx 2024-09-13T18:52:39.667 DEBUG:teuthology.orchestra.run.smithi171:> ') 2024-09-13T18:52:39.690 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-09-13T18:52:39.691 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:52:39.691 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-13T18:52:39.697 INFO:teuthology.orchestra.run.smithi171.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-13T18:52:39.698 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:52:39.698 DEBUG:teuthology.orchestra.run.smithi171:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-13T18:52:39.703 DEBUG:teuthology.orchestra.run.smithi171:> sudo modprobe fuse 2024-09-13T18:52:39.763 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:52:39.763 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:52:39.774 INFO:teuthology.orchestra.run.smithi171.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:52:39.775 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:52:39.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:39.776 DEBUG:teuthology.orchestra.run.smithi171:> ls /sys/fs/fuse/connections 2024-09-13T18:52:39.782 INFO:teuthology.orchestra.run.smithi171.stdout:52 2024-09-13T18:52:39.782 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [52] 2024-09-13T18:52:39.782 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-09-13T18:52:39.826 DEBUG:teuthology.orchestra.run.smithi171:> sudo modprobe fuse 2024-09-13T18:52:39.842 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:52:39.842 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:52:39.856 INFO:teuthology.orchestra.run.smithi171.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:52:39.856 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:52:39.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:39.857 DEBUG:teuthology.orchestra.run.smithi171:> ls /sys/fs/fuse/connections 2024-09-13T18:52:39.862 INFO:teuthology.orchestra.run.smithi171.stdout:52 2024-09-13T18:52:39.912 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi171.stderr:2024-09-13T18:52:39.905+0000 7fe552a30640 -1 init, newargv = 0x560508826880 newargc=15 2024-09-13T18:52:39.912 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi171.stderr:ceph-fuse[32384]: starting ceph client 2024-09-13T18:52:39.921 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi171.stderr:ceph-fuse[32384]: starting fuse 2024-09-13T18:52:40.964 DEBUG:teuthology.orchestra.run.smithi171:> sudo modprobe fuse 2024-09-13T18:52:40.983 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:52:40.983 DEBUG:teuthology.orchestra.run.smithi171:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:52:40.994 INFO:teuthology.orchestra.run.smithi171.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:52:40.995 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:52:40.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:40.996 DEBUG:teuthology.orchestra.run.smithi171:> ls /sys/fs/fuse/connections 2024-09-13T18:52:41.003 INFO:teuthology.orchestra.run.smithi171.stdout:52 2024-09-13T18:52:41.003 INFO:teuthology.orchestra.run.smithi171.stdout:53 2024-09-13T18:52:41.003 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52, 53] 2024-09-13T18:52:41.003 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:52:41.003 DEBUG:teuthology.orchestra.run.smithi171:> import glob 2024-09-13T18:52:41.003 DEBUG:teuthology.orchestra.run.smithi171:> import re 2024-09-13T18:52:41.003 DEBUG:teuthology.orchestra.run.smithi171:> import os 2024-09-13T18:52:41.004 DEBUG:teuthology.orchestra.run.smithi171:> import subprocess 2024-09-13T18:52:41.004 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T18:52:41.004 DEBUG:teuthology.orchestra.run.smithi171:> def _find_admin_socket(client_name): 2024-09-13T18:52:41.004 DEBUG:teuthology.orchestra.run.smithi171:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-09-13T18:52:41.004 DEBUG:teuthology.orchestra.run.smithi171:> files = glob.glob(asok_path) 2024-09-13T18:52:41.004 DEBUG:teuthology.orchestra.run.smithi171:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-09-13T18:52:41.004 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T18:52:41.004 DEBUG:teuthology.orchestra.run.smithi171:> # Given a non-glob path, it better be there 2024-09-13T18:52:41.004 DEBUG:teuthology.orchestra.run.smithi171:> if "*" not in asok_path: 2024-09-13T18:52:41.004 DEBUG:teuthology.orchestra.run.smithi171:> assert(len(files) == 1) 2024-09-13T18:52:41.004 DEBUG:teuthology.orchestra.run.smithi171:> return files[0] 2024-09-13T18:52:41.004 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T18:52:41.004 DEBUG:teuthology.orchestra.run.smithi171:> for f in files: 2024-09-13T18:52:41.004 DEBUG:teuthology.orchestra.run.smithi171:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T18:52:41.005 DEBUG:teuthology.orchestra.run.smithi171:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T18:52:41.005 DEBUG:teuthology.orchestra.run.smithi171:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T18:52:41.005 DEBUG:teuthology.orchestra.run.smithi171:> contents = proc_f.read() 2024-09-13T18:52:41.005 DEBUG:teuthology.orchestra.run.smithi171:> if mountpoint in contents: 2024-09-13T18:52:41.005 DEBUG:teuthology.orchestra.run.smithi171:> return f 2024-09-13T18:52:41.005 DEBUG:teuthology.orchestra.run.smithi171:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T18:52:41.005 DEBUG:teuthology.orchestra.run.smithi171:> 2024-09-13T18:52:41.005 DEBUG:teuthology.orchestra.run.smithi171:> print(_find_admin_socket("client.3")) 2024-09-13T18:52:41.005 DEBUG:teuthology.orchestra.run.smithi171:> ' 2024-09-13T18:52:41.126 INFO:teuthology.orchestra.run.smithi171.stdout:/var/run/ceph/ceph-client.3.32384.asok 2024-09-13T18:52:41.307 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.32384.asok 2024-09-13T18:52:41.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:41.308 DEBUG:teuthology.orchestra.run.smithi171:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.32384.asok status 2024-09-13T18:52:41.410 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-13T18:52:41.411 INFO:teuthology.orchestra.run.smithi171.stdout: "metadata": { 2024-09-13T18:52:41.411 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T18:52:41.411 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T18:52:41.411 INFO:teuthology.orchestra.run.smithi171.stdout: "entity_id": "3", 2024-09-13T18:52:41.411 INFO:teuthology.orchestra.run.smithi171.stdout: "hostname": "smithi171", 2024-09-13T18:52:41.411 INFO:teuthology.orchestra.run.smithi171.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-09-13T18:52:41.411 INFO:teuthology.orchestra.run.smithi171.stdout: "pid": "32384", 2024-09-13T18:52:41.411 INFO:teuthology.orchestra.run.smithi171.stdout: "root": "/" 2024-09-13T18:52:41.411 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T18:52:41.411 INFO:teuthology.orchestra.run.smithi171.stdout: "dentry_count": 0, 2024-09-13T18:52:41.411 INFO:teuthology.orchestra.run.smithi171.stdout: "dentry_pinned_count": 0, 2024-09-13T18:52:41.411 INFO:teuthology.orchestra.run.smithi171.stdout: "id": 4529, 2024-09-13T18:52:41.411 INFO:teuthology.orchestra.run.smithi171.stdout: "inst": { 2024-09-13T18:52:41.412 INFO:teuthology.orchestra.run.smithi171.stdout: "name": { 2024-09-13T18:52:41.412 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "client", 2024-09-13T18:52:41.412 INFO:teuthology.orchestra.run.smithi171.stdout: "num": 4529 2024-09-13T18:52:41.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T18:52:41.412 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": { 2024-09-13T18:52:41.412 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-13T18:52:41.412 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "192.168.0.2:0", 2024-09-13T18:52:41.412 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3543662595 2024-09-13T18:52:41.412 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-13T18:52:41.412 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T18:52:41.412 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": { 2024-09-13T18:52:41.412 INFO:teuthology.orchestra.run.smithi171.stdout: "type": "v1", 2024-09-13T18:52:41.412 INFO:teuthology.orchestra.run.smithi171.stdout: "addr": "192.168.0.2:0", 2024-09-13T18:52:41.412 INFO:teuthology.orchestra.run.smithi171.stdout: "nonce": 3543662595 2024-09-13T18:52:41.413 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-13T18:52:41.413 INFO:teuthology.orchestra.run.smithi171.stdout: "inst_str": "client.4529 192.168.0.2:0/3543662595", 2024-09-13T18:52:41.413 INFO:teuthology.orchestra.run.smithi171.stdout: "addr_str": "192.168.0.2:0/3543662595", 2024-09-13T18:52:41.413 INFO:teuthology.orchestra.run.smithi171.stdout: "inode_count": 1, 2024-09-13T18:52:41.413 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_epoch": 8, 2024-09-13T18:52:41.413 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_epoch": 22, 2024-09-13T18:52:41.413 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_epoch_barrier": 0, 2024-09-13T18:52:41.413 INFO:teuthology.orchestra.run.smithi171.stdout: "blocklisted": false, 2024-09-13T18:52:41.413 INFO:teuthology.orchestra.run.smithi171.stdout: "fs_name": "cephfs" 2024-09-13T18:52:41.413 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-13T18:52:41.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:41.420 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2024-09-13T18:52:41.420 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T18:52:41.429 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2024-09-13T18:52:41.429 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T18:52:41.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:41.429 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T18:52:41.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:41.445 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2024-09-13T18:52:41.445 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.2 2024-09-13T18:52:41.497 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2024-09-13T18:52:41.497 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-09-13T18:52:41.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:41.497 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-13T18:52:41.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:41.519 DEBUG:teuthology.orchestra.run.smithi171:> stat --file-system '--printf=%T 2024-09-13T18:52:41.519 DEBUG:teuthology.orchestra.run.smithi171:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-13T18:52:41.527 INFO:teuthology.orchestra.run.smithi171.stdout:fuseblk 2024-09-13T18:52:41.528 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-13T18:52:41.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:41.528 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-13T18:52:41.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:41.588 DEBUG:teuthology.orchestra.run.smithi171:> stat --file-system '--printf=%T 2024-09-13T18:52:41.588 DEBUG:teuthology.orchestra.run.smithi171:> ' -- /home/ubuntu/cephtest/mnt.3 2024-09-13T18:52:41.594 INFO:teuthology.orchestra.run.smithi171.stdout:fuseblk 2024-09-13T18:52:41.595 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-09-13T18:52:41.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:41.595 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-13T18:52:41.609 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-13T18:52:41.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:41.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.745+0000 7f6601da1700 1 Processor -- start 2024-09-13T18:52:41.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.745+0000 7f6601da1700 1 -- start start 2024-09-13T18:52:41.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.745+0000 7f6601da1700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65fc1551e0 0x7f65fc15b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:41.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.745+0000 7f6601da1700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f65fc0b2870 0x7f65fc0b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:41.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.745+0000 7f6601da1700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65fc141020 con 0x7f65fc154e00 2024-09-13T18:52:41.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.745+0000 7f6601da1700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65fc0b3240 con 0x7f65fc0b2870 2024-09-13T18:52:41.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.745+0000 7f6601da1700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc15bec0 con 0x7f65fc1551e0 2024-09-13T18:52:41.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f65fbfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65fc1551e0 0x7f65fc15b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:41.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f66015a0700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f65fc0b2870 0x7f65fc0b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:41.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f6600d9f700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f65fc154e00 0x7f65fc0cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:39820/0 (socket says 172.21.15.7:39820) 2024-09-13T18:52:41.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f6600d9f700 1 -- 172.21.15.7:0/422755479 learned_addr learned my addr 172.21.15.7:0/422755479 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:41.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f65fb7fe700 1 -- 172.21.15.7:0/422755479 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687893749 0 0) 0x7f65fc141020 con 0x7f65fc154e00 2024-09-13T18:52:41.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f65fbfff700 1 -- 172.21.15.7:0/422755479 >> v2:172.21.15.171:3300/0 conn(0x7f65fc0b2870 msgr2=0x7f65fc0b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:41.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f65fbfff700 1 --2- 172.21.15.7:0/422755479 >> v2:172.21.15.171:3300/0 conn(0x7f65fc0b2870 0x7f65fc0b2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f65fbfff700 1 -- 172.21.15.7:0/422755479 >> v1:172.21.15.171:6789/0 conn(0x7f65fc154e00 legacy=0x7f65fc0cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:41.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f65fbfff700 1 -- 172.21.15.7:0/422755479 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65fc1404a0 con 0x7f65fc1551e0 2024-09-13T18:52:41.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f66015a0700 1 --2- 172.21.15.7:0/422755479 >> v2:172.21.15.171:3300/0 conn(0x7f65fc0b2870 0x7f65fc0b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:41.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f65fbfff700 1 --2- 172.21.15.7:0/422755479 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65fc1551e0 0x7f65fc15b980 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009880 tx=0x7f65e4009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=677796cb1abd2a0e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:41.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f65fb7fe700 1 -- 172.21.15.7:0/422755479 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4011830 con 0x7f65fc1551e0 2024-09-13T18:52:41.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f65fb7fe700 1 -- 172.21.15.7:0/422755479 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e4011990 con 0x7f65fc1551e0 2024-09-13T18:52:41.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f65fb7fe700 1 -- 172.21.15.7:0/422755479 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e4011d20 con 0x7f65fc1551e0 2024-09-13T18:52:41.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f6601da1700 1 -- 172.21.15.7:0/422755479 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65fc1551e0 msgr2=0x7f65fc15b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:41.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f6601da1700 1 --2- 172.21.15.7:0/422755479 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65fc1551e0 0x7f65fc15b980 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009880 tx=0x7f65e4009b90 comp rx=0 tx=0).stop 2024-09-13T18:52:41.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f6601da1700 1 -- 172.21.15.7:0/422755479 shutdown_connections 2024-09-13T18:52:41.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f6601da1700 1 --2- 172.21.15.7:0/422755479 >> v2:172.21.15.171:3300/0 conn(0x7f65fc0b2870 0x7f65fc0b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f6601da1700 1 --2- 172.21.15.7:0/422755479 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65fc1551e0 0x7f65fc15b980 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f6601da1700 1 -- 172.21.15.7:0/422755479 >> 172.21.15.7:0/422755479 conn(0x7f65fc06a940 msgr2=0x7f65fc071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:41.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f6601da1700 1 -- 172.21.15.7:0/422755479 shutdown_connections 2024-09-13T18:52:41.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.749+0000 7f6601da1700 1 -- 172.21.15.7:0/422755479 wait complete. 2024-09-13T18:52:41.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.753+0000 7f6601da1700 1 Processor -- start 2024-09-13T18:52:41.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.753+0000 7f6601da1700 1 -- start start 2024-09-13T18:52:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f6601da1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65fc0b2870 0x7f65fc1f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f6601da1700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65fc154e00 0x7f65fc0b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f6601da1700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65fc1551e0 0x7f65fc13ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f6601da1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc06c180 con 0x7f65fc0b2870 2024-09-13T18:52:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f6601da1700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc13b6f0 con 0x7f65fc1551e0 2024-09-13T18:52:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f6601da1700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc13b9d0 con 0x7f65fc154e00 2024-09-13T18:52:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f6600d9f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65fc0b2870 0x7f65fc1f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f6600d9f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65fc0b2870 0x7f65fc1f1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45828/0 (socket says 172.21.15.7:45828) 2024-09-13T18:52:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f6600d9f700 1 -- 172.21.15.7:0/4260290027 learned_addr learned my addr 172.21.15.7:0/4260290027 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f65fbfff700 1 --2- 172.21.15.7:0/4260290027 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65fc154e00 0x7f65fc0b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f66015a0700 1 --2- 172.21.15.7:0/4260290027 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65fc1551e0 0x7f65fc13ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f6600d9f700 1 -- 172.21.15.7:0/4260290027 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65fc154e00 msgr2=0x7f65fc0b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f6600d9f700 1 --2- 172.21.15.7:0/4260290027 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65fc154e00 0x7f65fc0b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f6600d9f700 1 -- 172.21.15.7:0/4260290027 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65fc1551e0 msgr2=0x7f65fc13ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f6600d9f700 1 --2- 172.21.15.7:0/4260290027 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65fc1551e0 0x7f65fc13ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f6600d9f700 1 -- 172.21.15.7:0/4260290027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec009a70 con 0x7f65fc0b2870 2024-09-13T18:52:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f6600d9f700 1 --2- 172.21.15.7:0/4260290027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65fc0b2870 0x7f65fc1f1aa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f65f400c580 tx=0x7f65f400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f65f97fa700 1 -- 172.21.15.7:0/4260290027 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f4004ca0 con 0x7f65fc0b2870 2024-09-13T18:52:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f65f97fa700 1 -- 172.21.15.7:0/4260290027 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f4004e20 con 0x7f65fc0b2870 2024-09-13T18:52:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f6601da1700 1 -- 172.21.15.7:0/4260290027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e40094b0 con 0x7f65fc0b2870 2024-09-13T18:52:41.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f65f97fa700 1 -- 172.21.15.7:0/4260290027 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f4007310 con 0x7f65fc0b2870 2024-09-13T18:52:41.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f6601da1700 1 -- 172.21.15.7:0/4260290027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65fc13bf90 con 0x7f65fc0b2870 2024-09-13T18:52:41.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.757+0000 7f65f97fa700 1 -- 172.21.15.7:0/4260290027 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f65f4007470 con 0x7f65fc0b2870 2024-09-13T18:52:41.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.761+0000 7f6601da1700 1 -- 172.21.15.7:0/4260290027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65fc0b3d40 con 0x7f65fc0b2870 2024-09-13T18:52:41.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.761+0000 7f65f97fa700 1 --2- 172.21.15.7:0/4260290027 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f65e0066730 0x7f65e0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:41.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.761+0000 7f65f97fa700 1 -- 172.21.15.7:0/4260290027 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65f4083d10 con 0x7f65fc0b2870 2024-09-13T18:52:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.761+0000 7f65fbfff700 1 --2- 172.21.15.7:0/4260290027 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f65e0066730 0x7f65e0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.761+0000 7f65fbfff700 1 --2- 172.21.15.7:0/4260290027 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f65e0066730 0x7f65e0068bf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0b0810 tx=0x7f65e4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:41.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.765+0000 7f65f97fa700 1 -- 172.21.15.7:0/4260290027 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f65f4051110 con 0x7f65fc0b2870 2024-09-13T18:52:41.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.949+0000 7f6601da1700 1 -- 172.21.15.7:0/4260290027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65fc0cd820 con 0x7f65fc0b2870 2024-09-13T18:52:41.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.953+0000 7f65f97fa700 1 -- 172.21.15.7:0/4260290027 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f65f4007720 con 0x7f65fc0b2870 2024-09-13T18:52:41.958 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:41.958 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:41.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.957+0000 7f6601da1700 1 -- 172.21.15.7:0/4260290027 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f65e0066730 msgr2=0x7f65e0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:41.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.957+0000 7f6601da1700 1 --2- 172.21.15.7:0/4260290027 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f65e0066730 0x7f65e0068bf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0b0810 tx=0x7f65e4008030 comp rx=0 tx=0).stop 2024-09-13T18:52:41.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.957+0000 7f6601da1700 1 -- 172.21.15.7:0/4260290027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65fc0b2870 msgr2=0x7f65fc1f1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:41.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.957+0000 7f6601da1700 1 --2- 172.21.15.7:0/4260290027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65fc0b2870 0x7f65fc1f1aa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f65f400c580 tx=0x7f65f400c890 comp rx=0 tx=0).stop 2024-09-13T18:52:41.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.957+0000 7f6601da1700 1 -- 172.21.15.7:0/4260290027 shutdown_connections 2024-09-13T18:52:41.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.957+0000 7f6601da1700 1 --2- 172.21.15.7:0/4260290027 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f65e0066730 0x7f65e0068bf0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.957+0000 7f6601da1700 1 --2- 172.21.15.7:0/4260290027 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65fc1551e0 0x7f65fc13ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.957+0000 7f6601da1700 1 --2- 172.21.15.7:0/4260290027 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65fc154e00 0x7f65fc0b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.957+0000 7f6601da1700 1 --2- 172.21.15.7:0/4260290027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65fc0b2870 0x7f65fc1f1aa0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:41.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.957+0000 7f6601da1700 1 -- 172.21.15.7:0/4260290027 >> 172.21.15.7:0/4260290027 conn(0x7f65fc06a940 msgr2=0x7f65fc15e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:41.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.957+0000 7f6601da1700 1 -- 172.21.15.7:0/4260290027 shutdown_connections 2024-09-13T18:52:41.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:41.957+0000 7f6601da1700 1 -- 172.21.15.7:0/4260290027 wait complete. 2024-09-13T18:52:41.977 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:52:15.554683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:52:16.896879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:52:41.977 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:52:42.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.089+0000 7fc97ffff700 1 Processor -- start 2024-09-13T18:52:42.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.089+0000 7fc97ffff700 1 -- start start 2024-09-13T18:52:42.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc97ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc980144340 0x7fc98006d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:42.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc97ffff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc98006bdb0 0x7fc9800cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:42.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc97ffff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc980144720 con 0x7fc98015f140 2024-09-13T18:52:42.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc97ffff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc98006dd40 con 0x7fc980144340 2024-09-13T18:52:42.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc97ffff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98006da60 con 0x7fc98006bdb0 2024-09-13T18:52:42.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc97effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc980144340 0x7fc98006d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:42.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc97effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc980144340 0x7fc98006d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45834/0 (socket says 172.21.15.7:45834) 2024-09-13T18:52:42.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc97effd700 1 -- 172.21.15.7:0/215081115 learned_addr learned my addr 172.21.15.7:0/215081115 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:42.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc9767fc700 1 --2- 172.21.15.7:0/215081115 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc98006bdb0 0x7fc9800cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:42.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc97e7fc700 1 -- 172.21.15.7:0/215081115 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342672871 0 0) 0x7fc980144720 con 0x7fc98015f140 2024-09-13T18:52:42.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc97e7fc700 1 -- 172.21.15.7:0/215081115 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc96c0035d0 con 0x7fc98015f140 2024-09-13T18:52:42.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc97effd700 1 -- 172.21.15.7:0/215081115 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc98006bdb0 msgr2=0x7fc9800cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:42.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc97effd700 1 --2- 172.21.15.7:0/215081115 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc98006bdb0 0x7fc9800cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc97effd700 1 -- 172.21.15.7:0/215081115 >> v1:172.21.15.171:6789/0 conn(0x7fc98015f140 legacy=0x7fc980070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:42.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc97effd700 1 -- 172.21.15.7:0/215081115 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98015e660 con 0x7fc980144340 2024-09-13T18:52:42.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc97effd700 1 --2- 172.21.15.7:0/215081115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc980144340 0x7fc98006d480 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc970009910 tx=0x7fc970009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d675e519d253f3fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:42.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc97e7fc700 1 -- 172.21.15.7:0/215081115 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970011830 con 0x7fc980144340 2024-09-13T18:52:42.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc97e7fc700 1 -- 172.21.15.7:0/215081115 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc970011990 con 0x7fc980144340 2024-09-13T18:52:42.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.093+0000 7fc97e7fc700 1 -- 172.21.15.7:0/215081115 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970011d00 con 0x7fc980144340 2024-09-13T18:52:42.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.097+0000 7fc97ffff700 1 -- 172.21.15.7:0/215081115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc980144340 msgr2=0x7fc98006d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:42.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.097+0000 7fc97ffff700 1 --2- 172.21.15.7:0/215081115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc980144340 0x7fc98006d480 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc970009910 tx=0x7fc970009c20 comp rx=0 tx=0).stop 2024-09-13T18:52:42.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.097+0000 7fc97ffff700 1 -- 172.21.15.7:0/215081115 shutdown_connections 2024-09-13T18:52:42.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.097+0000 7fc97ffff700 1 --2- 172.21.15.7:0/215081115 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc98006bdb0 0x7fc9800cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.097+0000 7fc97ffff700 1 --2- 172.21.15.7:0/215081115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc980144340 0x7fc98006d480 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.097+0000 7fc97ffff700 1 -- 172.21.15.7:0/215081115 >> 172.21.15.7:0/215081115 conn(0x7fc9800b0e10 msgr2=0x7fc9800b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:42.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.097+0000 7fc97ffff700 1 -- 172.21.15.7:0/215081115 shutdown_connections 2024-09-13T18:52:42.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.097+0000 7fc97ffff700 1 -- 172.21.15.7:0/215081115 wait complete. 2024-09-13T18:52:42.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.097+0000 7fc97ffff700 1 Processor -- start 2024-09-13T18:52:42.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.097+0000 7fc97ffff700 1 -- start start 2024-09-13T18:52:42.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc98006bdb0 0x7fc980140970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:42.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97ffff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc980144340 0x7fc98013c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:42.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97ffff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc98015f140 0x7fc9800ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:42.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97ffff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc980143380 con 0x7fc98006bdb0 2024-09-13T18:52:42.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97ffff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc980141000 con 0x7fc98015f140 2024-09-13T18:52:42.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97ffff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9801f2710 con 0x7fc980144340 2024-09-13T18:52:42.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc98006bdb0 0x7fc980140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:42.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc98006bdb0 0x7fc980140970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45848/0 (socket says 172.21.15.7:45848) 2024-09-13T18:52:42.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97effd700 1 -- 172.21.15.7:0/3000451483 learned_addr learned my addr 172.21.15.7:0/3000451483 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:42.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc9767fc700 1 --2- 172.21.15.7:0/3000451483 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc980144340 0x7fc98013c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:42.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97f7fe700 1 --2- 172.21.15.7:0/3000451483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc98015f140 0x7fc9800ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:42.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97effd700 1 -- 172.21.15.7:0/3000451483 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc980144340 msgr2=0x7fc98013c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:42.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97effd700 1 --2- 172.21.15.7:0/3000451483 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc980144340 0x7fc98013c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97effd700 1 -- 172.21.15.7:0/3000451483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc98015f140 msgr2=0x7fc9800ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:42.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97effd700 1 --2- 172.21.15.7:0/3000451483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc98015f140 0x7fc9800ac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97effd700 1 -- 172.21.15.7:0/3000451483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc970009570 con 0x7fc98006bdb0 2024-09-13T18:52:42.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97f7fe700 1 --2- 172.21.15.7:0/3000451483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc98015f140 0x7fc9800ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:42.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97effd700 1 --2- 172.21.15.7:0/3000451483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc98006bdb0 0x7fc980140970 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc970009c60 tx=0x7fc970004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:42.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc977fff700 1 -- 172.21.15.7:0/3000451483 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970004300 con 0x7fc98006bdb0 2024-09-13T18:52:42.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc977fff700 1 -- 172.21.15.7:0/3000451483 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97001f070 con 0x7fc98006bdb0 2024-09-13T18:52:42.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97ffff700 1 -- 172.21.15.7:0/3000451483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9801f29f0 con 0x7fc98006bdb0 2024-09-13T18:52:42.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc977fff700 1 -- 172.21.15.7:0/3000451483 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9700036c0 con 0x7fc98006bdb0 2024-09-13T18:52:42.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.101+0000 7fc97ffff700 1 -- 172.21.15.7:0/3000451483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98013c800 con 0x7fc98006bdb0 2024-09-13T18:52:42.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.105+0000 7fc97ffff700 1 -- 172.21.15.7:0/3000451483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9801f2f10 con 0x7fc98006bdb0 2024-09-13T18:52:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.105+0000 7fc977fff700 1 -- 172.21.15.7:0/3000451483 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc970021030 con 0x7fc98006bdb0 2024-09-13T18:52:42.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.105+0000 7fc977fff700 1 --2- 172.21.15.7:0/3000451483 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc960066730 0x7fc960068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:42.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.105+0000 7fc977fff700 1 -- 172.21.15.7:0/3000451483 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc970085490 con 0x7fc98006bdb0 2024-09-13T18:52:42.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.105+0000 7fc9767fc700 1 --2- 172.21.15.7:0/3000451483 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc960066730 0x7fc960068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:42.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.105+0000 7fc9767fc700 1 --2- 172.21.15.7:0/3000451483 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc960066730 0x7fc960068bf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc98013ffa0 tx=0x7fc968009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:42.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.117+0000 7fc977fff700 1 -- 172.21.15.7:0/3000451483 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc9801f2f10 con 0x7fc98006bdb0 2024-09-13T18:52:42.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.285+0000 7fc97ffff700 1 -- 172.21.15.7:0/3000451483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc9801f2f10 con 0x7fc98006bdb0 2024-09-13T18:52:42.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.289+0000 7fc977fff700 1 -- 172.21.15.7:0/3000451483 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc9801f2f10 con 0x7fc98006bdb0 2024-09-13T18:52:42.293 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:52:42.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.289+0000 7fc97ffff700 1 -- 172.21.15.7:0/3000451483 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc960066730 msgr2=0x7fc960068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:42.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.289+0000 7fc97ffff700 1 --2- 172.21.15.7:0/3000451483 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc960066730 0x7fc960068bf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc98013ffa0 tx=0x7fc968009240 comp rx=0 tx=0).stop 2024-09-13T18:52:42.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.289+0000 7fc97ffff700 1 -- 172.21.15.7:0/3000451483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc98006bdb0 msgr2=0x7fc980140970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:42.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.289+0000 7fc97ffff700 1 --2- 172.21.15.7:0/3000451483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc98006bdb0 0x7fc980140970 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc970009c60 tx=0x7fc970004770 comp rx=0 tx=0).stop 2024-09-13T18:52:42.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.289+0000 7fc97ffff700 1 -- 172.21.15.7:0/3000451483 shutdown_connections 2024-09-13T18:52:42.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.289+0000 7fc97ffff700 1 --2- 172.21.15.7:0/3000451483 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc960066730 0x7fc960068bf0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.289+0000 7fc97ffff700 1 --2- 172.21.15.7:0/3000451483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc98015f140 0x7fc9800ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.289+0000 7fc97ffff700 1 --2- 172.21.15.7:0/3000451483 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc980144340 0x7fc98013c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.289+0000 7fc97ffff700 1 --2- 172.21.15.7:0/3000451483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc98006bdb0 0x7fc980140970 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.289+0000 7fc97ffff700 1 -- 172.21.15.7:0/3000451483 >> 172.21.15.7:0/3000451483 conn(0x7fc9800b0e10 msgr2=0x7fc980070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:42.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.289+0000 7fc97ffff700 1 -- 172.21.15.7:0/3000451483 shutdown_connections 2024-09-13T18:52:42.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.289+0000 7fc97ffff700 1 -- 172.21.15.7:0/3000451483 wait complete. 2024-09-13T18:52:42.309 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:52:42.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.417+0000 7f5349a56700 1 Processor -- start 2024-09-13T18:52:42.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.417+0000 7f5349a56700 1 -- start start 2024-09-13T18:52:42.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5349a56700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5344144340 0x7f534406d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5349a56700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f534406bdb0 0x7f53440cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5349a56700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5344144720 con 0x7f534415f140 2024-09-13T18:52:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5349a56700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f534406dd40 con 0x7f5344144340 2024-09-13T18:52:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5349a56700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534406da60 con 0x7f534406bdb0 2024-09-13T18:52:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5348a54700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5344144340 0x7f534406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5348a54700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5344144340 0x7f534406d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45856/0 (socket says 172.21.15.7:45856) 2024-09-13T18:52:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5348a54700 1 -- 172.21.15.7:0/2379320631 learned_addr learned my addr 172.21.15.7:0/2379320631 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5343fff700 1 --2- 172.21.15.7:0/2379320631 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f534406bdb0 0x7f53440cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5348a54700 1 -- 172.21.15.7:0/2379320631 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f534406bdb0 msgr2=0x7f53440cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5348a54700 1 --2- 172.21.15.7:0/2379320631 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f534406bdb0 0x7f53440cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f53437fe700 1 -- 172.21.15.7:0/2379320631 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585497006 0 0) 0x7f5344144720 con 0x7f534415f140 2024-09-13T18:52:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5348a54700 1 -- 172.21.15.7:0/2379320631 >> v1:172.21.15.171:6789/0 conn(0x7f534415f140 legacy=0x7f5344070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5348a54700 1 -- 172.21.15.7:0/2379320631 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534415e660 con 0x7f5344144340 2024-09-13T18:52:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5348a54700 1 --2- 172.21.15.7:0/2379320631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5344144340 0x7f534406d480 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5334009910 tx=0x7f5334009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9c34bc8027144e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f53437fe700 1 -- 172.21.15.7:0/2379320631 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5334011830 con 0x7f5344144340 2024-09-13T18:52:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f53437fe700 1 -- 172.21.15.7:0/2379320631 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5334011990 con 0x7f5344144340 2024-09-13T18:52:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f53437fe700 1 -- 172.21.15.7:0/2379320631 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5334011d40 con 0x7f5344144340 2024-09-13T18:52:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5349a56700 1 -- 172.21.15.7:0/2379320631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5344144340 msgr2=0x7f534406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5349a56700 1 --2- 172.21.15.7:0/2379320631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5344144340 0x7f534406d480 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5334009910 tx=0x7f5334009c20 comp rx=0 tx=0).stop 2024-09-13T18:52:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5349a56700 1 -- 172.21.15.7:0/2379320631 shutdown_connections 2024-09-13T18:52:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5349a56700 1 --2- 172.21.15.7:0/2379320631 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f534406bdb0 0x7f53440cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5349a56700 1 --2- 172.21.15.7:0/2379320631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5344144340 0x7f534406d480 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5349a56700 1 -- 172.21.15.7:0/2379320631 >> 172.21.15.7:0/2379320631 conn(0x7f53440b0e10 msgr2=0x7f53440b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5349a56700 1 -- 172.21.15.7:0/2379320631 shutdown_connections 2024-09-13T18:52:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.421+0000 7f5349a56700 1 -- 172.21.15.7:0/2379320631 wait complete. 2024-09-13T18:52:42.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.425+0000 7f5349a56700 1 Processor -- start 2024-09-13T18:52:42.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.425+0000 7f5349a56700 1 -- start start 2024-09-13T18:52:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.425+0000 7f5349a56700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f534406bdb0 0x7f53441f0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.425+0000 7f5349a56700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5344144340 0x7f53440ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.425+0000 7f5349a56700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534415f140 0x7f53440abcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.425+0000 7f5349a56700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53441f3af0 con 0x7f534415f140 2024-09-13T18:52:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.425+0000 7f5349a56700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53440aac30 con 0x7f5344144340 2024-09-13T18:52:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.425+0000 7f5349a56700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53440aaf10 con 0x7f534406bdb0 2024-09-13T18:52:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.425+0000 7f5349255700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534415f140 0x7f53440abcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.425+0000 7f5349255700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534415f140 0x7f53440abcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45864/0 (socket says 172.21.15.7:45864) 2024-09-13T18:52:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.425+0000 7f5349255700 1 -- 172.21.15.7:0/3656751289 learned_addr learned my addr 172.21.15.7:0/3656751289 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.425+0000 7f5348a54700 1 --2- 172.21.15.7:0/3656751289 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f534406bdb0 0x7f53441f0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.425+0000 7f5343fff700 1 --2- 172.21.15.7:0/3656751289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5344144340 0x7f53440ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.429+0000 7f5349255700 1 -- 172.21.15.7:0/3656751289 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f534406bdb0 msgr2=0x7f53441f0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.429+0000 7f5349255700 1 --2- 172.21.15.7:0/3656751289 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f534406bdb0 0x7f53441f0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.429+0000 7f5349255700 1 -- 172.21.15.7:0/3656751289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5344144340 msgr2=0x7f53440ad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.429+0000 7f5349255700 1 --2- 172.21.15.7:0/3656751289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5344144340 0x7f53440ad1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.429+0000 7f5349255700 1 -- 172.21.15.7:0/3656751289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5334009570 con 0x7f534415f140 2024-09-13T18:52:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.429+0000 7f5349255700 1 --2- 172.21.15.7:0/3656751289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534415f140 0x7f53440abcd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f533c00c3f0 tx=0x7f533c00c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.429+0000 7f53417fa700 1 -- 172.21.15.7:0/3656751289 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c0052e0 con 0x7f534415f140 2024-09-13T18:52:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.429+0000 7f53417fa700 1 -- 172.21.15.7:0/3656751289 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533c005460 con 0x7f534415f140 2024-09-13T18:52:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.429+0000 7f5349a56700 1 -- 172.21.15.7:0/3656751289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53440ab1f0 con 0x7f534415f140 2024-09-13T18:52:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.429+0000 7f53417fa700 1 -- 172.21.15.7:0/3656751289 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533c0038e0 con 0x7f534415f140 2024-09-13T18:52:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.429+0000 7f5349a56700 1 -- 172.21.15.7:0/3656751289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53440ab7b0 con 0x7f534415f140 2024-09-13T18:52:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.429+0000 7f5349a56700 1 -- 172.21.15.7:0/3656751289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f534406c350 con 0x7f534415f140 2024-09-13T18:52:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.429+0000 7f53417fa700 1 -- 172.21.15.7:0/3656751289 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f533c003a40 con 0x7f534415f140 2024-09-13T18:52:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.433+0000 7f53417fa700 1 --2- 172.21.15.7:0/3656751289 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5324066730 0x7f5324068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.433+0000 7f53417fa700 1 -- 172.21.15.7:0/3656751289 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f533c0833a0 con 0x7f534415f140 2024-09-13T18:52:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.433+0000 7f5348a54700 1 --2- 172.21.15.7:0/3656751289 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5324066730 0x7f5324068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.433+0000 7f5348a54700 1 --2- 172.21.15.7:0/3656751289 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5324066730 0x7f5324068bf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5334000c00 tx=0x7f5334008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.441+0000 7f53417fa700 1 -- 172.21.15.7:0/3656751289 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f533c0507a0 con 0x7f534415f140 2024-09-13T18:52:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.613+0000 7f5349a56700 1 -- 172.21.15.7:0/3656751289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f53440ab9d0 con 0x7f534415f140 2024-09-13T18:52:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.613+0000 7f53417fa700 1 -- 172.21.15.7:0/3656751289 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f533c053dc0 con 0x7f534415f140 2024-09-13T18:52:42.618 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:52:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.613+0000 7f5349a56700 1 -- 172.21.15.7:0/3656751289 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5324066730 msgr2=0x7f5324068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.613+0000 7f5349a56700 1 --2- 172.21.15.7:0/3656751289 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5324066730 0x7f5324068bf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5334000c00 tx=0x7f5334008030 comp rx=0 tx=0).stop 2024-09-13T18:52:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.613+0000 7f5349a56700 1 -- 172.21.15.7:0/3656751289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534415f140 msgr2=0x7f53440abcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.613+0000 7f5349a56700 1 --2- 172.21.15.7:0/3656751289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534415f140 0x7f53440abcd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f533c00c3f0 tx=0x7f533c00c700 comp rx=0 tx=0).stop 2024-09-13T18:52:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.613+0000 7f5349a56700 1 -- 172.21.15.7:0/3656751289 shutdown_connections 2024-09-13T18:52:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.613+0000 7f5349a56700 1 --2- 172.21.15.7:0/3656751289 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5324066730 0x7f5324068bf0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.613+0000 7f5349a56700 1 --2- 172.21.15.7:0/3656751289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534415f140 0x7f53440abcd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.613+0000 7f5349a56700 1 --2- 172.21.15.7:0/3656751289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5344144340 0x7f53440ad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.613+0000 7f5349a56700 1 --2- 172.21.15.7:0/3656751289 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f534406bdb0 0x7f53441f0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.613+0000 7f5349a56700 1 -- 172.21.15.7:0/3656751289 >> 172.21.15.7:0/3656751289 conn(0x7f53440b0e10 msgr2=0x7f53440caa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.613+0000 7f5349a56700 1 -- 172.21.15.7:0/3656751289 shutdown_connections 2024-09-13T18:52:42.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.613+0000 7f5349a56700 1 -- 172.21.15.7:0/3656751289 wait complete. 2024-09-13T18:52:42.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:52:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58fea46700 1 Processor -- start 2024-09-13T18:52:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58fea46700 1 -- start start 2024-09-13T18:52:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58fea46700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f58f81610b0 0x7f58f80cd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58fea46700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f58f8071c80 0x7f58f80720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58fea46700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f8072600 con 0x7f58f80b2bc0 2024-09-13T18:52:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58fea46700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f80b3c00 con 0x7f58f81610b0 2024-09-13T18:52:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58fea46700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80cd660 con 0x7f58f8071c80 2024-09-13T18:52:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58f7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f58f81610b0 0x7f58f80cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58f7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f58f81610b0 0x7f58f80cd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45868/0 (socket says 172.21.15.7:45868) 2024-09-13T18:52:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58f7fff700 1 -- 172.21.15.7:0/3443831867 learned_addr learned my addr 172.21.15.7:0/3443831867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58ef7fe700 1 --2- 172.21.15.7:0/3443831867 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f58f8071c80 0x7f58f80720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58f7fff700 1 -- 172.21.15.7:0/3443831867 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f58f8071c80 msgr2=0x7f58f80720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58f7fff700 1 --2- 172.21.15.7:0/3443831867 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f58f8071c80 0x7f58f80720c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58f7fff700 1 -- 172.21.15.7:0/3443831867 >> v1:172.21.15.171:6789/0 conn(0x7f58f80b2bc0 legacy=0x7f58f80b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58f7fff700 1 -- 172.21.15.7:0/3443831867 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f80b3920 con 0x7f58f81610b0 2024-09-13T18:52:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58f7fff700 1 --2- 172.21.15.7:0/3443831867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f81610b0 0x7f58f80cd120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009910 tx=0x7f58e8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab36c85686808e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58f77fe700 1 -- 172.21.15.7:0/3443831867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e800ba60 con 0x7f58f81610b0 2024-09-13T18:52:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58f77fe700 1 -- 172.21.15.7:0/3443831867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e800bbc0 con 0x7f58f81610b0 2024-09-13T18:52:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.753+0000 7f58f77fe700 1 -- 172.21.15.7:0/3443831867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8009e60 con 0x7f58f81610b0 2024-09-13T18:52:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.757+0000 7f58fea46700 1 -- 172.21.15.7:0/3443831867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f81610b0 msgr2=0x7f58f80cd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.757+0000 7f58fea46700 1 --2- 172.21.15.7:0/3443831867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f81610b0 0x7f58f80cd120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009910 tx=0x7f58e8009c20 comp rx=0 tx=0).stop 2024-09-13T18:52:42.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.757+0000 7f58fea46700 1 -- 172.21.15.7:0/3443831867 shutdown_connections 2024-09-13T18:52:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.757+0000 7f58fea46700 1 --2- 172.21.15.7:0/3443831867 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f58f8071c80 0x7f58f80720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.757+0000 7f58fea46700 1 --2- 172.21.15.7:0/3443831867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f81610b0 0x7f58f80cd120 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.757+0000 7f58fea46700 1 -- 172.21.15.7:0/3443831867 >> 172.21.15.7:0/3443831867 conn(0x7f58f80b1810 msgr2=0x7f58f80b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.757+0000 7f58fea46700 1 -- 172.21.15.7:0/3443831867 shutdown_connections 2024-09-13T18:52:42.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.757+0000 7f58fea46700 1 -- 172.21.15.7:0/3443831867 wait complete. 2024-09-13T18:52:42.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.757+0000 7f58fea46700 1 Processor -- start 2024-09-13T18:52:42.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.757+0000 7f58fea46700 1 -- start start 2024-09-13T18:52:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58fea46700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f8071c80 0x7f58f814e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58fea46700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f58f80b2bc0 0x7f58f80c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58fea46700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58f81610b0 0x7f58f80c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58fea46700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80c60c0 con 0x7f58f8071c80 2024-09-13T18:52:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58fea46700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f814edd0 con 0x7f58f81610b0 2024-09-13T18:52:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58fea46700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f80c5440 con 0x7f58f80b2bc0 2024-09-13T18:52:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58f7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f8071c80 0x7f58f814e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58f7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f8071c80 0x7f58f814e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45884/0 (socket says 172.21.15.7:45884) 2024-09-13T18:52:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58f7fff700 1 -- 172.21.15.7:0/345359536 learned_addr learned my addr 172.21.15.7:0/345359536 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:42.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58fcfe3700 1 --2- 172.21.15.7:0/345359536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58f81610b0 0x7f58f80c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:42.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58ef7fe700 1 --2- 172.21.15.7:0/345359536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f58f80b2bc0 0x7f58f80c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:42.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58f7fff700 1 -- 172.21.15.7:0/345359536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f58f80b2bc0 msgr2=0x7f58f80c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:42.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58f7fff700 1 --2- 172.21.15.7:0/345359536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f58f80b2bc0 0x7f58f80c4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58f7fff700 1 -- 172.21.15.7:0/345359536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58f81610b0 msgr2=0x7f58f80c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:42.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58f7fff700 1 --2- 172.21.15.7:0/345359536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58f81610b0 0x7f58f80c9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58f7fff700 1 -- 172.21.15.7:0/345359536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e8009570 con 0x7f58f8071c80 2024-09-13T18:52:42.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58f7fff700 1 --2- 172.21.15.7:0/345359536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f8071c80 0x7f58f814e890 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009150 tx=0x7f58e8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:42.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58f57fa700 1 -- 172.21.15.7:0/345359536 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8004510 con 0x7f58f8071c80 2024-09-13T18:52:42.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58f57fa700 1 -- 172.21.15.7:0/345359536 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e801e070 con 0x7f58f8071c80 2024-09-13T18:52:42.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58fea46700 1 -- 172.21.15.7:0/345359536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f80c5720 con 0x7f58f8071c80 2024-09-13T18:52:42.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58f57fa700 1 -- 172.21.15.7:0/345359536 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8003900 con 0x7f58f8071c80 2024-09-13T18:52:42.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58fea46700 1 -- 172.21.15.7:0/345359536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f80c5ce0 con 0x7f58f8071c80 2024-09-13T18:52:42.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.761+0000 7f58fea46700 1 -- 172.21.15.7:0/345359536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f80c63e0 con 0x7f58f8071c80 2024-09-13T18:52:42.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.765+0000 7f58f57fa700 1 -- 172.21.15.7:0/345359536 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f58e8003a60 con 0x7f58f8071c80 2024-09-13T18:52:42.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.765+0000 7f58f57fa700 1 --2- 172.21.15.7:0/345359536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f58d8066730 0x7f58d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:42.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.765+0000 7f58f57fa700 1 -- 172.21.15.7:0/345359536 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58e8020070 con 0x7f58f8071c80 2024-09-13T18:52:42.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.765+0000 7f58ef7fe700 1 --2- 172.21.15.7:0/345359536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f58d8066730 0x7f58d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:42.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.765+0000 7f58ef7fe700 1 --2- 172.21.15.7:0/345359536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f58d8066730 0x7f58d8068bf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f58f8163550 tx=0x7f58e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:42.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.773+0000 7f58f57fa700 1 -- 172.21.15.7:0/345359536 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f58e8052da0 con 0x7f58f8071c80 2024-09-13T18:52:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.945+0000 7f58fea46700 1 -- 172.21.15.7:0/345359536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f58f80720c0 con 0x7f58f8071c80 2024-09-13T18:52:42.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.945+0000 7f58f57fa700 1 -- 172.21.15.7:0/345359536 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f58e802a020 con 0x7f58f8071c80 2024-09-13T18:52:42.951 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:52:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.945+0000 7f58fea46700 1 -- 172.21.15.7:0/345359536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f58d8066730 msgr2=0x7f58d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.945+0000 7f58fea46700 1 --2- 172.21.15.7:0/345359536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f58d8066730 0x7f58d8068bf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f58f8163550 tx=0x7f58e0008040 comp rx=0 tx=0).stop 2024-09-13T18:52:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.945+0000 7f58fea46700 1 -- 172.21.15.7:0/345359536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f8071c80 msgr2=0x7f58f814e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.945+0000 7f58fea46700 1 --2- 172.21.15.7:0/345359536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f8071c80 0x7f58f814e890 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009150 tx=0x7f58e8004770 comp rx=0 tx=0).stop 2024-09-13T18:52:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.945+0000 7f58fea46700 1 -- 172.21.15.7:0/345359536 shutdown_connections 2024-09-13T18:52:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.945+0000 7f58fea46700 1 --2- 172.21.15.7:0/345359536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f58d8066730 0x7f58d8068bf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.945+0000 7f58fea46700 1 --2- 172.21.15.7:0/345359536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58f81610b0 0x7f58f80c9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.945+0000 7f58fea46700 1 --2- 172.21.15.7:0/345359536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f58f80b2bc0 0x7f58f80c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.945+0000 7f58fea46700 1 --2- 172.21.15.7:0/345359536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58f8071c80 0x7f58f814e890 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.945+0000 7f58fea46700 1 -- 172.21.15.7:0/345359536 >> 172.21.15.7:0/345359536 conn(0x7f58f80b1810 msgr2=0x7f58f8162960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.949+0000 7f58fea46700 1 -- 172.21.15.7:0/345359536 shutdown_connections 2024-09-13T18:52:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:42.949+0000 7f58fea46700 1 -- 172.21.15.7:0/345359536 wait complete. 2024-09-13T18:52:42.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:43.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.077+0000 7f3e23fff700 1 Processor -- start 2024-09-13T18:52:43.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.077+0000 7f3e23fff700 1 -- start start 2024-09-13T18:52:43.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.077+0000 7f3e23fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3e240b3ae0 0x7f3e24165590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.077+0000 7f3e23fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e24161dd0 con 0x7f3e24160eb0 2024-09-13T18:52:43.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.077+0000 7f3e23fff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e24071f30 con 0x7f3e24161290 2024-09-13T18:52:43.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.077+0000 7f3e23fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e241622e0 con 0x7f3e240b3ae0 2024-09-13T18:52:43.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.077+0000 7f3e227fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3e240b3ae0 0x7f3e24165590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.077+0000 7f3e227fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3e240b3ae0 0x7f3e24165590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45886/0 (socket says 172.21.15.7:45886) 2024-09-13T18:52:43.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.077+0000 7f3e227fc700 1 -- 172.21.15.7:0/94668097 learned_addr learned my addr 172.21.15.7:0/94668097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:43.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.077+0000 7f3e227fc700 1 -- 172.21.15.7:0/94668097 >> v1:172.21.15.171:6789/0 conn(0x7f3e24161290 legacy=0x7f3e241616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.077+0000 7f3e227fc700 1 -- 172.21.15.7:0/94668097 >> v1:172.21.15.7:6789/0 conn(0x7f3e24160eb0 legacy=0x7f3e240b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.077+0000 7f3e227fc700 1 -- 172.21.15.7:0/94668097 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e24162000 con 0x7f3e240b3ae0 2024-09-13T18:52:43.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.077+0000 7f3e227fc700 1 --2- 172.21.15.7:0/94668097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e240b3ae0 0x7f3e24165590 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009e50 tx=0x7f3e1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ed47c5e25386fa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:43.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.077+0000 7f3e21ffb700 1 -- 172.21.15.7:0/94668097 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c00e070 con 0x7f3e240b3ae0 2024-09-13T18:52:43.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.077+0000 7f3e21ffb700 1 -- 172.21.15.7:0/94668097 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1c00b9c0 con 0x7f3e240b3ae0 2024-09-13T18:52:43.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.077+0000 7f3e21ffb700 1 -- 172.21.15.7:0/94668097 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c00bd70 con 0x7f3e240b3ae0 2024-09-13T18:52:43.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.081+0000 7f3e23fff700 1 -- 172.21.15.7:0/94668097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e240b3ae0 msgr2=0x7f3e24165590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.081+0000 7f3e23fff700 1 --2- 172.21.15.7:0/94668097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e240b3ae0 0x7f3e24165590 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c009e50 tx=0x7f3e1c00b330 comp rx=0 tx=0).stop 2024-09-13T18:52:43.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.081+0000 7f3e23fff700 1 -- 172.21.15.7:0/94668097 shutdown_connections 2024-09-13T18:52:43.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.081+0000 7f3e23fff700 1 --2- 172.21.15.7:0/94668097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e240b3ae0 0x7f3e24165590 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.081+0000 7f3e23fff700 1 -- 172.21.15.7:0/94668097 >> 172.21.15.7:0/94668097 conn(0x7f3e240b0f00 msgr2=0x7f3e240b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:43.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.081+0000 7f3e23fff700 1 -- 172.21.15.7:0/94668097 shutdown_connections 2024-09-13T18:52:43.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.081+0000 7f3e23fff700 1 -- 172.21.15.7:0/94668097 wait complete. 2024-09-13T18:52:43.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.081+0000 7f3e23fff700 1 Processor -- start 2024-09-13T18:52:43.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.081+0000 7f3e23fff700 1 -- start start 2024-09-13T18:52:43.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e23fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3e240b3ae0 0x7f3e2406d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e23fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e24160eb0 0x7f3e2414a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e23fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e24161290 0x7f3e2414a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e23fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2414adf0 con 0x7f3e24160eb0 2024-09-13T18:52:43.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e23fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e24157c00 con 0x7f3e24161290 2024-09-13T18:52:43.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e23fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e24155440 con 0x7f3e240b3ae0 2024-09-13T18:52:43.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e227fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e24160eb0 0x7f3e2414a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e227fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e24160eb0 0x7f3e2414a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45888/0 (socket says 172.21.15.7:45888) 2024-09-13T18:52:43.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e22ffd700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3e240b3ae0 0x7f3e2406d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e227fc700 1 -- 172.21.15.7:0/3010006990 learned_addr learned my addr 172.21.15.7:0/3010006990 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:43.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e237fe700 1 --2- 172.21.15.7:0/3010006990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e24161290 0x7f3e2414a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e227fc700 1 -- 172.21.15.7:0/3010006990 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3e240b3ae0 msgr2=0x7f3e2406d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e227fc700 1 --2- 172.21.15.7:0/3010006990 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3e240b3ae0 0x7f3e2406d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e227fc700 1 -- 172.21.15.7:0/3010006990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e24161290 msgr2=0x7f3e2414a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e227fc700 1 --2- 172.21.15.7:0/3010006990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e24161290 0x7f3e2414a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e227fc700 1 -- 172.21.15.7:0/3010006990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1c009b30 con 0x7f3e24160eb0 2024-09-13T18:52:43.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e237fe700 1 --2- 172.21.15.7:0/3010006990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e24161290 0x7f3e2414a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:43.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e227fc700 1 --2- 172.21.15.7:0/3010006990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e24160eb0 0x7f3e2414a370 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c000f80 tx=0x7f3e1c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:43.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e1b7fe700 1 -- 172.21.15.7:0/3010006990 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c004930 con 0x7f3e24160eb0 2024-09-13T18:52:43.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e1b7fe700 1 -- 172.21.15.7:0/3010006990 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1c0092e0 con 0x7f3e24160eb0 2024-09-13T18:52:43.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e23fff700 1 -- 172.21.15.7:0/3010006990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e24155720 con 0x7f3e24160eb0 2024-09-13T18:52:43.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e1b7fe700 1 -- 172.21.15.7:0/3010006990 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c01c680 con 0x7f3e24160eb0 2024-09-13T18:52:43.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.085+0000 7f3e23fff700 1 -- 172.21.15.7:0/3010006990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e24155ce0 con 0x7f3e24160eb0 2024-09-13T18:52:43.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.089+0000 7f3e23fff700 1 -- 172.21.15.7:0/3010006990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e24165590 con 0x7f3e24160eb0 2024-09-13T18:52:43.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.089+0000 7f3e1b7fe700 1 -- 172.21.15.7:0/3010006990 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3e1c01c7e0 con 0x7f3e24160eb0 2024-09-13T18:52:43.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.089+0000 7f3e1b7fe700 1 --2- 172.21.15.7:0/3010006990 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3e08066730 0x7f3e08068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.089+0000 7f3e1b7fe700 1 -- 172.21.15.7:0/3010006990 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e1c085c00 con 0x7f3e24160eb0 2024-09-13T18:52:43.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.089+0000 7f3e22ffd700 1 --2- 172.21.15.7:0/3010006990 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3e08066730 0x7f3e08068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.089+0000 7f3e22ffd700 1 --2- 172.21.15.7:0/3010006990 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3e08066730 0x7f3e08068bf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c000f60 tx=0x7f3e0c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:43.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.093+0000 7f3e1b7fe700 1 -- 172.21.15.7:0/3010006990 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3e1c052f60 con 0x7f3e24160eb0 2024-09-13T18:52:43.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.261+0000 7f3e23fff700 1 -- 172.21.15.7:0/3010006990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e24161760 con 0x7f3e24160eb0 2024-09-13T18:52:43.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.265+0000 7f3e1b7fe700 1 -- 172.21.15.7:0/3010006990 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f3e1c056580 con 0x7f3e24160eb0 2024-09-13T18:52:43.271 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:43.271 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:43.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.265+0000 7f3e23fff700 1 -- 172.21.15.7:0/3010006990 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3e08066730 msgr2=0x7f3e08068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.265+0000 7f3e23fff700 1 --2- 172.21.15.7:0/3010006990 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3e08066730 0x7f3e08068bf0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c000f60 tx=0x7f3e0c009040 comp rx=0 tx=0).stop 2024-09-13T18:52:43.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.265+0000 7f3e23fff700 1 -- 172.21.15.7:0/3010006990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e24160eb0 msgr2=0x7f3e2414a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.265+0000 7f3e23fff700 1 --2- 172.21.15.7:0/3010006990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e24160eb0 0x7f3e2414a370 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c000f80 tx=0x7f3e1c004af0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.269+0000 7f3e23fff700 1 -- 172.21.15.7:0/3010006990 shutdown_connections 2024-09-13T18:52:43.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.269+0000 7f3e23fff700 1 --2- 172.21.15.7:0/3010006990 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3e08066730 0x7f3e08068bf0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.269+0000 7f3e23fff700 1 --2- 172.21.15.7:0/3010006990 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e24161290 0x7f3e2414a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.269+0000 7f3e23fff700 1 --2- 172.21.15.7:0/3010006990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e24160eb0 0x7f3e2414a370 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.269+0000 7f3e23fff700 1 --2- 172.21.15.7:0/3010006990 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3e240b3ae0 0x7f3e2406d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.269+0000 7f3e23fff700 1 -- 172.21.15.7:0/3010006990 >> 172.21.15.7:0/3010006990 conn(0x7f3e240b0f00 msgr2=0x7f3e240ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:43.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.269+0000 7f3e23fff700 1 -- 172.21.15.7:0/3010006990 shutdown_connections 2024-09-13T18:52:43.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.269+0000 7f3e23fff700 1 -- 172.21.15.7:0/3010006990 wait complete. 2024-09-13T18:52:43.287 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:52:15.554683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:52:16.896879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:52:43.287 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:52:43.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.405+0000 7f8f8af0c700 1 Processor -- start 2024-09-13T18:52:43.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.405+0000 7f8f8af0c700 1 -- start start 2024-09-13T18:52:43.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f8af0c700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f8f8406b900 0x7f8f840b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f8af0c700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8f8406a520 0x7f8f8406a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f8af0c700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f840b8430 con 0x7f8f8415cda0 2024-09-13T18:52:43.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f8af0c700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f8406ae90 con 0x7f8f8406b900 2024-09-13T18:52:43.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f8af0c700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8406bd60 con 0x7f8f8406a520 2024-09-13T18:52:43.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f8a70b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8f8406a520 0x7f8f8406a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f89709700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f8f8406b900 0x7f8f840b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f8a70b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8f8406a520 0x7f8f8406a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:42322/0 (socket says 172.21.15.7:42322) 2024-09-13T18:52:43.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f89f0a700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f8f8415cda0 0x7f8f840ceb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:39868/0 (socket says 172.21.15.7:39868) 2024-09-13T18:52:43.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f8a70b700 1 -- 172.21.15.7:0/2718248295 learned_addr learned my addr 172.21.15.7:0/2718248295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:43.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f89709700 1 -- 172.21.15.7:0/2718248295 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8f8406a520 msgr2=0x7f8f8406a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:43.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f89709700 1 --2- 172.21.15.7:0/2718248295 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8f8406a520 0x7f8f8406a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f89709700 1 -- 172.21.15.7:0/2718248295 >> v1:172.21.15.171:6789/0 conn(0x7f8f8415cda0 legacy=0x7f8f840ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f88f08700 1 -- 172.21.15.7:0/2718248295 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 906337575 0 0) 0x7f8f840b8430 con 0x7f8f8415cda0 2024-09-13T18:52:43.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f89709700 1 -- 172.21.15.7:0/2718248295 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f840cf350 con 0x7f8f8406b900 2024-09-13T18:52:43.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f89709700 1 --2- 172.21.15.7:0/2718248295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f8406b900 0x7f8f840b5de0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009850 tx=0x7f8f78009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3aa5c237c68886d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:43.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f88f08700 1 -- 172.21.15.7:0/2718248295 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f78011830 con 0x7f8f8406b900 2024-09-13T18:52:43.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f88f08700 1 -- 172.21.15.7:0/2718248295 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f78011990 con 0x7f8f8406b900 2024-09-13T18:52:43.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f88f08700 1 -- 172.21.15.7:0/2718248295 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f78011d40 con 0x7f8f8406b900 2024-09-13T18:52:43.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f8af0c700 1 -- 172.21.15.7:0/2718248295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f8406b900 msgr2=0x7f8f840b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f8af0c700 1 --2- 172.21.15.7:0/2718248295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f8406b900 0x7f8f840b5de0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8f78009850 tx=0x7f8f78009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:43.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f8af0c700 1 -- 172.21.15.7:0/2718248295 shutdown_connections 2024-09-13T18:52:43.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f8af0c700 1 --2- 172.21.15.7:0/2718248295 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8f8406a520 0x7f8f8406a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f8af0c700 1 --2- 172.21.15.7:0/2718248295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f8406b900 0x7f8f840b5de0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f8af0c700 1 -- 172.21.15.7:0/2718248295 >> 172.21.15.7:0/2718248295 conn(0x7f8f840d2cb0 msgr2=0x7f8f840d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:43.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f8af0c700 1 -- 172.21.15.7:0/2718248295 shutdown_connections 2024-09-13T18:52:43.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.409+0000 7f8f8af0c700 1 -- 172.21.15.7:0/2718248295 wait complete. 2024-09-13T18:52:43.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.413+0000 7f8f8af0c700 1 Processor -- start 2024-09-13T18:52:43.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.413+0000 7f8f8af0c700 1 -- start start 2024-09-13T18:52:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f8af0c700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8f8406a520 0x7f8f84155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f8af0c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f8406b900 0x7f8f841567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f8af0c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f8415cda0 0x7f8f841f2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f8af0c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f840c80c0 con 0x7f8f8415cda0 2024-09-13T18:52:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f8af0c700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f840c83a0 con 0x7f8f8406b900 2024-09-13T18:52:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f8af0c700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f840c8680 con 0x7f8f8406a520 2024-09-13T18:52:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f8a70b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f8415cda0 0x7f8f841f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f89709700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f8406b900 0x7f8f841567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f8a70b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f8415cda0 0x7f8f841f2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45892/0 (socket says 172.21.15.7:45892) 2024-09-13T18:52:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f8a70b700 1 -- 172.21.15.7:0/2659366624 learned_addr learned my addr 172.21.15.7:0/2659366624 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f89f0a700 1 --2- 172.21.15.7:0/2659366624 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8f8406a520 0x7f8f84155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f89709700 1 -- 172.21.15.7:0/2659366624 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8f8406a520 msgr2=0x7f8f84155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f89709700 1 --2- 172.21.15.7:0/2659366624 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8f8406a520 0x7f8f84155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f89709700 1 -- 172.21.15.7:0/2659366624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f8415cda0 msgr2=0x7f8f841f2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f89709700 1 --2- 172.21.15.7:0/2659366624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f8415cda0 0x7f8f841f2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f89709700 1 -- 172.21.15.7:0/2659366624 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f780094b0 con 0x7f8f8406b900 2024-09-13T18:52:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f89709700 1 --2- 172.21.15.7:0/2659366624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f8406b900 0x7f8f841567f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8f78016000 tx=0x7f8f78004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f767fc700 1 -- 172.21.15.7:0/2659366624 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7801a990 con 0x7f8f8406b900 2024-09-13T18:52:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f767fc700 1 -- 172.21.15.7:0/2659366624 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7801f070 con 0x7f8f8406b900 2024-09-13T18:52:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f767fc700 1 -- 172.21.15.7:0/2659366624 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7801ad30 con 0x7f8f8406b900 2024-09-13T18:52:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f8af0c700 1 -- 172.21.15.7:0/2659366624 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f840c8960 con 0x7f8f8406b900 2024-09-13T18:52:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.417+0000 7f8f8af0c700 1 -- 172.21.15.7:0/2659366624 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f840c7d50 con 0x7f8f8406b900 2024-09-13T18:52:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.421+0000 7f8f8af0c700 1 -- 172.21.15.7:0/2659366624 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f840c8e80 con 0x7f8f8406b900 2024-09-13T18:52:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.421+0000 7f8f767fc700 1 -- 172.21.15.7:0/2659366624 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8f7802e450 con 0x7f8f8406b900 2024-09-13T18:52:43.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.421+0000 7f8f767fc700 1 --2- 172.21.15.7:0/2659366624 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8f68066730 0x7f8f68068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.421+0000 7f8f767fc700 1 -- 172.21.15.7:0/2659366624 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f78085530 con 0x7f8f8406b900 2024-09-13T18:52:43.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.421+0000 7f8f89f0a700 1 --2- 172.21.15.7:0/2659366624 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8f68066730 0x7f8f68068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.421+0000 7f8f89f0a700 1 --2- 172.21.15.7:0/2659366624 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8f68066730 0x7f8f68068bf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8f80000f60 tx=0x7f8f80007e90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:43.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.433+0000 7f8f767fc700 1 -- 172.21.15.7:0/2659366624 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8f78052890 con 0x7f8f8406b900 2024-09-13T18:52:43.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.605+0000 7f8f8af0c700 1 -- 172.21.15.7:0/2659366624 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8f840c8e80 con 0x7f8f8406b900 2024-09-13T18:52:43.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.605+0000 7f8f767fc700 1 -- 172.21.15.7:0/2659366624 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8f840c8e80 con 0x7f8f8406b900 2024-09-13T18:52:43.610 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:52:43.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.605+0000 7f8f8af0c700 1 -- 172.21.15.7:0/2659366624 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8f68066730 msgr2=0x7f8f68068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.605+0000 7f8f8af0c700 1 --2- 172.21.15.7:0/2659366624 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8f68066730 0x7f8f68068bf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8f80000f60 tx=0x7f8f80007e90 comp rx=0 tx=0).stop 2024-09-13T18:52:43.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.605+0000 7f8f8af0c700 1 -- 172.21.15.7:0/2659366624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f8406b900 msgr2=0x7f8f841567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.605+0000 7f8f8af0c700 1 --2- 172.21.15.7:0/2659366624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f8406b900 0x7f8f841567f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8f78016000 tx=0x7f8f78004770 comp rx=0 tx=0).stop 2024-09-13T18:52:43.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.605+0000 7f8f8af0c700 1 -- 172.21.15.7:0/2659366624 shutdown_connections 2024-09-13T18:52:43.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.605+0000 7f8f8af0c700 1 --2- 172.21.15.7:0/2659366624 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8f68066730 0x7f8f68068bf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.605+0000 7f8f8af0c700 1 --2- 172.21.15.7:0/2659366624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f8415cda0 0x7f8f841f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.605+0000 7f8f8af0c700 1 --2- 172.21.15.7:0/2659366624 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8f8406b900 0x7f8f841567f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.605+0000 7f8f8af0c700 1 --2- 172.21.15.7:0/2659366624 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8f8406a520 0x7f8f84155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.605+0000 7f8f8af0c700 1 -- 172.21.15.7:0/2659366624 >> 172.21.15.7:0/2659366624 conn(0x7f8f840d2cb0 msgr2=0x7f8f840d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:43.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.605+0000 7f8f8af0c700 1 -- 172.21.15.7:0/2659366624 shutdown_connections 2024-09-13T18:52:43.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.605+0000 7f8f8af0c700 1 -- 172.21.15.7:0/2659366624 wait complete. 2024-09-13T18:52:43.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:52:43.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.737+0000 7f16a50bf700 1 Processor -- start 2024-09-13T18:52:43.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.737+0000 7f16a50bf700 1 -- start start 2024-09-13T18:52:43.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.737+0000 7f16a50bf700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f16a015d200 0x7f16a0163c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.737+0000 7f16a50bf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a006dad0 con 0x7f16a00ce270 2024-09-13T18:52:43.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.737+0000 7f16a50bf700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a00ae3c0 con 0x7f16a006d6f0 2024-09-13T18:52:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.737+0000 7f16a50bf700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a00705b0 con 0x7f16a015d200 2024-09-13T18:52:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.737+0000 7f169f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f16a00ce270 0x7f16a015caf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57790/0 (socket says 172.21.15.7:57790) 2024-09-13T18:52:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.737+0000 7f169f7fe700 1 -- 172.21.15.7:0/906756966 learned_addr learned my addr 172.21.15.7:0/906756966 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.737+0000 7f16a48be700 1 --2- 172.21.15.7:0/906756966 >> v2:172.21.15.171:3300/0 conn(0x7f16a015d200 0x7f16a0163c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.737+0000 7f169effd700 1 -- 172.21.15.7:0/906756966 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409622443 0 0) 0x7f16a006dad0 con 0x7f16a00ce270 2024-09-13T18:52:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.737+0000 7f169effd700 1 -- 172.21.15.7:0/906756966 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16880035d0 con 0x7f16a00ce270 2024-09-13T18:52:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.741+0000 7f169effd700 1 -- 172.21.15.7:0/906756966 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645807510 0 0) 0x7f16a00ae3c0 con 0x7f16a006d6f0 2024-09-13T18:52:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.741+0000 7f169effd700 1 -- 172.21.15.7:0/906756966 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16a006dad0 con 0x7f16a006d6f0 2024-09-13T18:52:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.741+0000 7f16a48be700 1 -- 172.21.15.7:0/906756966 >> v1:172.21.15.171:6789/0 conn(0x7f16a006d6f0 legacy=0x7f16a00adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.741+0000 7f16a48be700 1 -- 172.21.15.7:0/906756966 >> v1:172.21.15.7:6789/0 conn(0x7f16a00ce270 legacy=0x7f16a015caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.741+0000 7f169effd700 1 -- 172.21.15.7:0/906756966 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3455698965 0 0) 0x7f16880035d0 con 0x7f16a00ce270 2024-09-13T18:52:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.741+0000 7f16a48be700 1 -- 172.21.15.7:0/906756966 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a0070890 con 0x7f16a015d200 2024-09-13T18:52:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.741+0000 7f16a48be700 1 --2- 172.21.15.7:0/906756966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16a015d200 0x7f16a0163c50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1698009dc0 tx=0x7f169800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2540bd82e5b138bb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.741+0000 7f169effd700 1 -- 172.21.15.7:0/906756966 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800e070 con 0x7f16a015d200 2024-09-13T18:52:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.741+0000 7f169effd700 1 -- 172.21.15.7:0/906756966 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169800bae0 con 0x7f16a015d200 2024-09-13T18:52:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.741+0000 7f169effd700 1 -- 172.21.15.7:0/906756966 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800be90 con 0x7f16a015d200 2024-09-13T18:52:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.741+0000 7f16a50bf700 1 -- 172.21.15.7:0/906756966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16a015d200 msgr2=0x7f16a0163c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.741+0000 7f16a50bf700 1 --2- 172.21.15.7:0/906756966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16a015d200 0x7f16a0163c50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1698009dc0 tx=0x7f169800b330 comp rx=0 tx=0).stop 2024-09-13T18:52:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.741+0000 7f16a50bf700 1 -- 172.21.15.7:0/906756966 shutdown_connections 2024-09-13T18:52:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.741+0000 7f16a50bf700 1 --2- 172.21.15.7:0/906756966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16a015d200 0x7f16a0163c50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.741+0000 7f16a50bf700 1 -- 172.21.15.7:0/906756966 >> 172.21.15.7:0/906756966 conn(0x7f16a00b0e10 msgr2=0x7f16a00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.741+0000 7f16a50bf700 1 -- 172.21.15.7:0/906756966 shutdown_connections 2024-09-13T18:52:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.741+0000 7f16a50bf700 1 -- 172.21.15.7:0/906756966 wait complete. 2024-09-13T18:52:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f16a50bf700 1 Processor -- start 2024-09-13T18:52:43.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f16a50bf700 1 -- start start 2024-09-13T18:52:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f16a50bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a006d6f0 0x7f16a014d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f16a50bf700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16a00ce270 0x7f16a014c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f16a50bf700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f16a015d200 0x7f16a00ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f16a50bf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a014e790 con 0x7f16a006d6f0 2024-09-13T18:52:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f16a50bf700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a014ebb0 con 0x7f16a00ce270 2024-09-13T18:52:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f16a50bf700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00ac890 con 0x7f16a015d200 2024-09-13T18:52:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f169ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a006d6f0 0x7f16a014d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f169ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a006d6f0 0x7f16a014d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45894/0 (socket says 172.21.15.7:45894) 2024-09-13T18:52:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f169ffff700 1 -- 172.21.15.7:0/1484803420 learned_addr learned my addr 172.21.15.7:0/1484803420 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f169f7fe700 1 --2- 172.21.15.7:0/1484803420 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16a00ce270 0x7f16a014c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f16a48be700 1 --2- 172.21.15.7:0/1484803420 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f16a015d200 0x7f16a00ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f169ffff700 1 -- 172.21.15.7:0/1484803420 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f16a015d200 msgr2=0x7f16a00ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f169ffff700 1 --2- 172.21.15.7:0/1484803420 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f16a015d200 0x7f16a00ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f169ffff700 1 -- 172.21.15.7:0/1484803420 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16a00ce270 msgr2=0x7f16a014c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f169ffff700 1 --2- 172.21.15.7:0/1484803420 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16a00ce270 0x7f16a014c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f169ffff700 1 -- 172.21.15.7:0/1484803420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1698009a70 con 0x7f16a006d6f0 2024-09-13T18:52:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f169f7fe700 1 --2- 172.21.15.7:0/1484803420 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16a00ce270 0x7f16a014c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.745+0000 7f169ffff700 1 --2- 172.21.15.7:0/1484803420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a006d6f0 0x7f16a014d430 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f169400c760 tx=0x7f169400ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.749+0000 7f169cff9700 1 -- 172.21.15.7:0/1484803420 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16940052d0 con 0x7f16a006d6f0 2024-09-13T18:52:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.749+0000 7f169cff9700 1 -- 172.21.15.7:0/1484803420 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1694005450 con 0x7f16a006d6f0 2024-09-13T18:52:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.749+0000 7f169cff9700 1 -- 172.21.15.7:0/1484803420 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16940038e0 con 0x7f16a006d6f0 2024-09-13T18:52:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.749+0000 7f16a50bf700 1 -- 172.21.15.7:0/1484803420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a014db10 con 0x7f16a006d6f0 2024-09-13T18:52:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.749+0000 7f16a50bf700 1 -- 172.21.15.7:0/1484803420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a014e0d0 con 0x7f16a006d6f0 2024-09-13T18:52:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.749+0000 7f16a50bf700 1 -- 172.21.15.7:0/1484803420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a00adbd0 con 0x7f16a006d6f0 2024-09-13T18:52:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.749+0000 7f169cff9700 1 -- 172.21.15.7:0/1484803420 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1694003a40 con 0x7f16a006d6f0 2024-09-13T18:52:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.749+0000 7f169cff9700 1 --2- 172.21.15.7:0/1484803420 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f168c066730 0x7f168c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.753+0000 7f169cff9700 1 -- 172.21.15.7:0/1484803420 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1694083470 con 0x7f16a006d6f0 2024-09-13T18:52:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.753+0000 7f169f7fe700 1 --2- 172.21.15.7:0/1484803420 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f168c066730 0x7f168c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:43.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.753+0000 7f169f7fe700 1 --2- 172.21.15.7:0/1484803420 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f168c066730 0x7f168c068bf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f16a01f3950 tx=0x7f1690009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.761+0000 7f169cff9700 1 -- 172.21.15.7:0/1484803420 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1694050870 con 0x7f16a006d6f0 2024-09-13T18:52:43.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.925+0000 7f16a50bf700 1 -- 172.21.15.7:0/1484803420 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f16a014ea00 con 0x7f16a006d6f0 2024-09-13T18:52:43.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.925+0000 7f169cff9700 1 -- 172.21.15.7:0/1484803420 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1694053e90 con 0x7f16a006d6f0 2024-09-13T18:52:43.933 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:52:43.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.929+0000 7f16a50bf700 1 -- 172.21.15.7:0/1484803420 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f168c066730 msgr2=0x7f168c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.929+0000 7f16a50bf700 1 --2- 172.21.15.7:0/1484803420 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f168c066730 0x7f168c068bf0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f16a01f3950 tx=0x7f1690009040 comp rx=0 tx=0).stop 2024-09-13T18:52:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.929+0000 7f16a50bf700 1 -- 172.21.15.7:0/1484803420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a006d6f0 msgr2=0x7f16a014d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.929+0000 7f16a50bf700 1 --2- 172.21.15.7:0/1484803420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a006d6f0 0x7f16a014d430 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f169400c760 tx=0x7f169400ca70 comp rx=0 tx=0).stop 2024-09-13T18:52:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.929+0000 7f16a50bf700 1 -- 172.21.15.7:0/1484803420 shutdown_connections 2024-09-13T18:52:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.929+0000 7f16a50bf700 1 --2- 172.21.15.7:0/1484803420 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f168c066730 0x7f168c068bf0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.929+0000 7f16a50bf700 1 --2- 172.21.15.7:0/1484803420 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f16a015d200 0x7f16a00ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.929+0000 7f16a50bf700 1 --2- 172.21.15.7:0/1484803420 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16a00ce270 0x7f16a014c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.929+0000 7f16a50bf700 1 --2- 172.21.15.7:0/1484803420 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16a006d6f0 0x7f16a014d430 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.929+0000 7f16a50bf700 1 -- 172.21.15.7:0/1484803420 >> 172.21.15.7:0/1484803420 conn(0x7f16a00b0e10 msgr2=0x7f16a00ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.929+0000 7f16a50bf700 1 -- 172.21.15.7:0/1484803420 shutdown_connections 2024-09-13T18:52:43.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:43.929+0000 7f16a50bf700 1 -- 172.21.15.7:0/1484803420 wait complete. 2024-09-13T18:52:43.948 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:52:44.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf80e86700 1 Processor -- start 2024-09-13T18:52:44.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf80e86700 1 -- start start 2024-09-13T18:52:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf80e86700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fcf7c0ce300 0x7fcf7c0ad270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf80e86700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcf7c0ad7b0 0x7fcf7c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf80e86700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf7c0ae000 con 0x7fcf7c146160 2024-09-13T18:52:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf80e86700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf7c0cc820 con 0x7fcf7c0ad7b0 2024-09-13T18:52:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf80e86700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf7c070070 con 0x7fcf7c0ce300 2024-09-13T18:52:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf7bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcf7c0ad7b0 0x7fcf7c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf7affd700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fcf7c0ce300 0x7fcf7c0ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf7bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcf7c0ad7b0 0x7fcf7c070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45898/0 (socket says 172.21.15.7:45898) 2024-09-13T18:52:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf7bfff700 1 -- 172.21.15.7:0/3388104316 learned_addr learned my addr 172.21.15.7:0/3388104316 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:44.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf7bfff700 1 -- 172.21.15.7:0/3388104316 >> v2:172.21.15.171:3300/0 conn(0x7fcf7c0ce300 msgr2=0x7fcf7c0ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:44.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf7bfff700 1 --2- 172.21.15.7:0/3388104316 >> v2:172.21.15.171:3300/0 conn(0x7fcf7c0ce300 0x7fcf7c0ad270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf7bfff700 1 -- 172.21.15.7:0/3388104316 >> v1:172.21.15.171:6789/0 conn(0x7fcf7c146160 legacy=0x7fcf7c0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:44.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf7bfff700 1 -- 172.21.15.7:0/3388104316 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c070350 con 0x7fcf7c0ad7b0 2024-09-13T18:52:44.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf7bfff700 1 --2- 172.21.15.7:0/3388104316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c0ad7b0 0x7fcf7c070b50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0cd730 tx=0x7fcf7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce941deb02234b51 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:44.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf7a7fc700 1 -- 172.21.15.7:0/3388104316 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7400e070 con 0x7fcf7c0ad7b0 2024-09-13T18:52:44.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf7a7fc700 1 -- 172.21.15.7:0/3388104316 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7400b910 con 0x7fcf7c0ad7b0 2024-09-13T18:52:44.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.065+0000 7fcf7a7fc700 1 -- 172.21.15.7:0/3388104316 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7400bcc0 con 0x7fcf7c0ad7b0 2024-09-13T18:52:44.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf80e86700 1 -- 172.21.15.7:0/3388104316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c0ad7b0 msgr2=0x7fcf7c070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:44.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf80e86700 1 --2- 172.21.15.7:0/3388104316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c0ad7b0 0x7fcf7c070b50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0cd730 tx=0x7fcf7400b330 comp rx=0 tx=0).stop 2024-09-13T18:52:44.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf80e86700 1 -- 172.21.15.7:0/3388104316 shutdown_connections 2024-09-13T18:52:44.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf80e86700 1 --2- 172.21.15.7:0/3388104316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c0ad7b0 0x7fcf7c070b50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf80e86700 1 --2- 172.21.15.7:0/3388104316 >> v2:172.21.15.171:3300/0 conn(0x7fcf7c0ce300 0x7fcf7c0ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf80e86700 1 -- 172.21.15.7:0/3388104316 >> 172.21.15.7:0/3388104316 conn(0x7fcf7c0b0e10 msgr2=0x7fcf7c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:44.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf80e86700 1 -- 172.21.15.7:0/3388104316 shutdown_connections 2024-09-13T18:52:44.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf80e86700 1 -- 172.21.15.7:0/3388104316 wait complete. 2024-09-13T18:52:44.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf80e86700 1 Processor -- start 2024-09-13T18:52:44.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf80e86700 1 -- start start 2024-09-13T18:52:44.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf80e86700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf7c0ad7b0 0x7fcf7c13d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:44.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf80e86700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c0ce300 0x7fcf7c15c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:44.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf80e86700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcf7c146160 0x7fcf7c13b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:44.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf80e86700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c13cfa0 con 0x7fcf7c0ce300 2024-09-13T18:52:44.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf80e86700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c13c390 con 0x7fcf7c0ad7b0 2024-09-13T18:52:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf80e86700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c13c670 con 0x7fcf7c146160 2024-09-13T18:52:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf7affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c0ce300 0x7fcf7c15c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf7affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c0ce300 0x7fcf7c15c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45914/0 (socket says 172.21.15.7:45914) 2024-09-13T18:52:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.069+0000 7fcf7affd700 1 -- 172.21.15.7:0/3839552028 learned_addr learned my addr 172.21.15.7:0/3839552028 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.073+0000 7fcf7bfff700 1 --2- 172.21.15.7:0/3839552028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcf7c146160 0x7fcf7c13b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.073+0000 7fcf7b7fe700 1 --2- 172.21.15.7:0/3839552028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf7c0ad7b0 0x7fcf7c13d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.073+0000 7fcf7affd700 1 -- 172.21.15.7:0/3839552028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcf7c146160 msgr2=0x7fcf7c13b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.073+0000 7fcf7affd700 1 --2- 172.21.15.7:0/3839552028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcf7c146160 0x7fcf7c13b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.073+0000 7fcf7affd700 1 -- 172.21.15.7:0/3839552028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf7c0ad7b0 msgr2=0x7fcf7c13d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.073+0000 7fcf7affd700 1 --2- 172.21.15.7:0/3839552028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf7c0ad7b0 0x7fcf7c13d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.073+0000 7fcf7affd700 1 -- 172.21.15.7:0/3839552028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf74009b30 con 0x7fcf7c0ce300 2024-09-13T18:52:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.073+0000 7fcf7affd700 1 --2- 172.21.15.7:0/3839552028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c0ce300 0x7fcf7c15c410 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000e9d0 tx=0x7fcf7000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.073+0000 7fcf63fff700 1 -- 172.21.15.7:0/3839552028 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf70009720 con 0x7fcf7c0ce300 2024-09-13T18:52:44.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.073+0000 7fcf63fff700 1 -- 172.21.15.7:0/3839552028 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7000ee70 con 0x7fcf7c0ce300 2024-09-13T18:52:44.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.073+0000 7fcf80e86700 1 -- 172.21.15.7:0/3839552028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7c13c950 con 0x7fcf7c0ce300 2024-09-13T18:52:44.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.073+0000 7fcf63fff700 1 -- 172.21.15.7:0/3839552028 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7000ca50 con 0x7fcf7c0ce300 2024-09-13T18:52:44.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.073+0000 7fcf80e86700 1 -- 172.21.15.7:0/3839552028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7c15c950 con 0x7fcf7c0ce300 2024-09-13T18:52:44.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.073+0000 7fcf80e86700 1 -- 172.21.15.7:0/3839552028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7c0b2fa0 con 0x7fcf7c0ce300 2024-09-13T18:52:44.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.073+0000 7fcf63fff700 1 -- 172.21.15.7:0/3839552028 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcf7000cbb0 con 0x7fcf7c0ce300 2024-09-13T18:52:44.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.077+0000 7fcf63fff700 1 --2- 172.21.15.7:0/3839552028 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcf68066730 0x7fcf68068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:44.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.077+0000 7fcf63fff700 1 -- 172.21.15.7:0/3839552028 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf70084180 con 0x7fcf7c0ce300 2024-09-13T18:52:44.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.077+0000 7fcf7b7fe700 1 --2- 172.21.15.7:0/3839552028 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcf68066730 0x7fcf68068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:44.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.077+0000 7fcf7b7fe700 1 --2- 172.21.15.7:0/3839552028 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcf68066730 0x7fcf68068bf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c000f60 tx=0x7fcf6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:44.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.085+0000 7fcf63fff700 1 -- 172.21.15.7:0/3839552028 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcf700514f0 con 0x7fcf7c0ce300 2024-09-13T18:52:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.249+0000 7fcf80e86700 1 -- 172.21.15.7:0/3839552028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcf7c0ad270 con 0x7fcf7c0ce300 2024-09-13T18:52:44.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.253+0000 7fcf63fff700 1 -- 172.21.15.7:0/3839552028 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcf7000ce60 con 0x7fcf7c0ce300 2024-09-13T18:52:44.258 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:52:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.253+0000 7fcf80e86700 1 -- 172.21.15.7:0/3839552028 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcf68066730 msgr2=0x7fcf68068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.253+0000 7fcf80e86700 1 --2- 172.21.15.7:0/3839552028 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcf68066730 0x7fcf68068bf0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c000f60 tx=0x7fcf6c009040 comp rx=0 tx=0).stop 2024-09-13T18:52:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.253+0000 7fcf80e86700 1 -- 172.21.15.7:0/3839552028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c0ce300 msgr2=0x7fcf7c15c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.253+0000 7fcf80e86700 1 --2- 172.21.15.7:0/3839552028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c0ce300 0x7fcf7c15c410 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000e9d0 tx=0x7fcf7000ece0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.253+0000 7fcf80e86700 1 -- 172.21.15.7:0/3839552028 shutdown_connections 2024-09-13T18:52:44.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.253+0000 7fcf80e86700 1 --2- 172.21.15.7:0/3839552028 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcf68066730 0x7fcf68068bf0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.253+0000 7fcf80e86700 1 --2- 172.21.15.7:0/3839552028 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcf7c146160 0x7fcf7c13b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.253+0000 7fcf80e86700 1 --2- 172.21.15.7:0/3839552028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf7c0ce300 0x7fcf7c15c410 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.253+0000 7fcf80e86700 1 --2- 172.21.15.7:0/3839552028 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf7c0ad7b0 0x7fcf7c13d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.253+0000 7fcf80e86700 1 -- 172.21.15.7:0/3839552028 >> 172.21.15.7:0/3839552028 conn(0x7fcf7c0b0e10 msgr2=0x7fcf7c06a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.253+0000 7fcf80e86700 1 -- 172.21.15.7:0/3839552028 shutdown_connections 2024-09-13T18:52:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.253+0000 7fcf80e86700 1 -- 172.21.15.7:0/3839552028 wait complete. 2024-09-13T18:52:44.273 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:44.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.381+0000 7f94e1389700 1 Processor -- start 2024-09-13T18:52:44.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.381+0000 7f94e1389700 1 -- start start 2024-09-13T18:52:44.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.381+0000 7f94e1389700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94dc159720 0x7f94dc06cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:44.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.381+0000 7f94e1389700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f94dc0b3ae0 0x7f94dc0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:44.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.381+0000 7f94e1389700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94dc06aed0 con 0x7f94dc06aa90 2024-09-13T18:52:44.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.381+0000 7f94e1389700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94dc06d690 con 0x7f94dc0b3ae0 2024-09-13T18:52:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.381+0000 7f94e1389700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc0cbfe0 con 0x7f94dc159720 2024-09-13T18:52:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94e0b88700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f94dc0b3ae0 0x7f94dc0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94db7fe700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f94dc06aa90 0x7f94dc0b33d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:39906/0 (socket says 172.21.15.7:39906) 2024-09-13T18:52:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94db7fe700 1 -- 172.21.15.7:0/2106672190 learned_addr learned my addr 172.21.15.7:0/2106672190 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94dbfff700 1 --2- 172.21.15.7:0/2106672190 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94dc159720 0x7f94dc06cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94e0b88700 1 -- 172.21.15.7:0/2106672190 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94dc159720 msgr2=0x7f94dc06cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94e0b88700 1 --2- 172.21.15.7:0/2106672190 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94dc159720 0x7f94dc06cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94e0b88700 1 -- 172.21.15.7:0/2106672190 >> v1:172.21.15.171:6789/0 conn(0x7f94dc06aa90 legacy=0x7f94dc0b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94e0b88700 1 -- 172.21.15.7:0/2106672190 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc0cc2c0 con 0x7f94dc0b3ae0 2024-09-13T18:52:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94daffd700 1 -- 172.21.15.7:0/2106672190 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154844705 0 0) 0x7f94dc06aed0 con 0x7f94dc06aa90 2024-09-13T18:52:44.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94e0b88700 1 --2- 172.21.15.7:0/2106672190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94dc0b3ae0 0x7f94dc0ccac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009d90 tx=0x7f94d400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7924d8f1dfac5076 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:44.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94daffd700 1 -- 172.21.15.7:0/2106672190 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d400e070 con 0x7f94dc0b3ae0 2024-09-13T18:52:44.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94daffd700 1 -- 172.21.15.7:0/2106672190 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d400bb20 con 0x7f94dc0b3ae0 2024-09-13T18:52:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94daffd700 1 -- 172.21.15.7:0/2106672190 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4012430 con 0x7f94dc0b3ae0 2024-09-13T18:52:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94e1389700 1 -- 172.21.15.7:0/2106672190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94dc0b3ae0 msgr2=0x7f94dc0ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94e1389700 1 --2- 172.21.15.7:0/2106672190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94dc0b3ae0 0x7f94dc0ccac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009d90 tx=0x7f94d400b330 comp rx=0 tx=0).stop 2024-09-13T18:52:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94e1389700 1 -- 172.21.15.7:0/2106672190 shutdown_connections 2024-09-13T18:52:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94e1389700 1 --2- 172.21.15.7:0/2106672190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94dc0b3ae0 0x7f94dc0ccac0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94e1389700 1 --2- 172.21.15.7:0/2106672190 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94dc159720 0x7f94dc06cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94e1389700 1 -- 172.21.15.7:0/2106672190 >> 172.21.15.7:0/2106672190 conn(0x7f94dc0b0f00 msgr2=0x7f94dc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94e1389700 1 -- 172.21.15.7:0/2106672190 shutdown_connections 2024-09-13T18:52:44.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.385+0000 7f94e1389700 1 -- 172.21.15.7:0/2106672190 wait complete. 2024-09-13T18:52:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94e1389700 1 Processor -- start 2024-09-13T18:52:44.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94e1389700 1 -- start start 2024-09-13T18:52:44.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94e1389700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94dc06aa90 0x7f94dc14c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:44.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94e1389700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94dc0b3ae0 0x7f94dc151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:44.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94e1389700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94dc159720 0x7f94dc15c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:44.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94e1389700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc14c7a0 con 0x7f94dc06aa90 2024-09-13T18:52:44.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94e1389700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc14e890 con 0x7f94dc159720 2024-09-13T18:52:44.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94e1389700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc159d20 con 0x7f94dc0b3ae0 2024-09-13T18:52:44.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94dc06aa90 0x7f94dc14c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:44.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94dc06aa90 0x7f94dc14c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45928/0 (socket says 172.21.15.7:45928) 2024-09-13T18:52:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94dbfff700 1 -- 172.21.15.7:0/1229665707 learned_addr learned my addr 172.21.15.7:0/1229665707 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94db7fe700 1 --2- 172.21.15.7:0/1229665707 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94dc0b3ae0 0x7f94dc151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94e0b88700 1 --2- 172.21.15.7:0/1229665707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94dc159720 0x7f94dc15c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94dbfff700 1 -- 172.21.15.7:0/1229665707 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94dc0b3ae0 msgr2=0x7f94dc151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94dbfff700 1 --2- 172.21.15.7:0/1229665707 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94dc0b3ae0 0x7f94dc151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94dbfff700 1 -- 172.21.15.7:0/1229665707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94dc159720 msgr2=0x7f94dc15c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94dbfff700 1 --2- 172.21.15.7:0/1229665707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94dc159720 0x7f94dc15c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94dbfff700 1 -- 172.21.15.7:0/1229665707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d4009a70 con 0x7f94dc06aa90 2024-09-13T18:52:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94dbfff700 1 --2- 172.21.15.7:0/1229665707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94dc06aa90 0x7f94dc14c260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00e870 tx=0x7f94cc00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94d8ff9700 1 -- 172.21.15.7:0/1229665707 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0095a0 con 0x7f94dc06aa90 2024-09-13T18:52:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94d8ff9700 1 -- 172.21.15.7:0/1229665707 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94cc013680 con 0x7f94dc06aa90 2024-09-13T18:52:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94e1389700 1 -- 172.21.15.7:0/1229665707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94dc159fc0 con 0x7f94dc06aa90 2024-09-13T18:52:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94d8ff9700 1 -- 172.21.15.7:0/1229665707 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc0139f0 con 0x7f94dc06aa90 2024-09-13T18:52:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.389+0000 7f94e1389700 1 -- 172.21.15.7:0/1229665707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94dc15a580 con 0x7f94dc06aa90 2024-09-13T18:52:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.393+0000 7f94d8ff9700 1 -- 172.21.15.7:0/1229665707 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f94cc013b50 con 0x7f94dc06aa90 2024-09-13T18:52:44.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.393+0000 7f94e1389700 1 -- 172.21.15.7:0/1229665707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94dc0ccac0 con 0x7f94dc06aa90 2024-09-13T18:52:44.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.393+0000 7f94d8ff9700 1 --2- 172.21.15.7:0/1229665707 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f94c8066730 0x7f94c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:44.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.393+0000 7f94d8ff9700 1 -- 172.21.15.7:0/1229665707 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94cc0840a0 con 0x7f94dc06aa90 2024-09-13T18:52:44.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.393+0000 7f94db7fe700 1 --2- 172.21.15.7:0/1229665707 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f94c8066730 0x7f94c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.397+0000 7f94db7fe700 1 --2- 172.21.15.7:0/1229665707 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f94c8066730 0x7f94c8068bf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0ad8a0 tx=0x7f94d0007e00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.401+0000 7f94d8ff9700 1 -- 172.21.15.7:0/1229665707 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f94cc051370 con 0x7f94dc06aa90 2024-09-13T18:52:44.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.577+0000 7f94e1389700 1 -- 172.21.15.7:0/1229665707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94dc0cb550 con 0x7f94dc06aa90 2024-09-13T18:52:44.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.581+0000 7f94d8ff9700 1 -- 172.21.15.7:0/1229665707 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f94cc054990 con 0x7f94dc06aa90 2024-09-13T18:52:44.587 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:44.587 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:44.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.581+0000 7f94e1389700 1 -- 172.21.15.7:0/1229665707 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f94c8066730 msgr2=0x7f94c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:44.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.581+0000 7f94e1389700 1 --2- 172.21.15.7:0/1229665707 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f94c8066730 0x7f94c8068bf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0ad8a0 tx=0x7f94d0007e00 comp rx=0 tx=0).stop 2024-09-13T18:52:44.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.581+0000 7f94e1389700 1 -- 172.21.15.7:0/1229665707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94dc06aa90 msgr2=0x7f94dc14c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:44.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.581+0000 7f94e1389700 1 --2- 172.21.15.7:0/1229665707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94dc06aa90 0x7f94dc14c260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00e870 tx=0x7f94cc00eb80 comp rx=0 tx=0).stop 2024-09-13T18:52:44.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.581+0000 7f94e1389700 1 -- 172.21.15.7:0/1229665707 shutdown_connections 2024-09-13T18:52:44.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.581+0000 7f94e1389700 1 --2- 172.21.15.7:0/1229665707 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f94c8066730 0x7f94c8068bf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.581+0000 7f94e1389700 1 --2- 172.21.15.7:0/1229665707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f94dc159720 0x7f94dc15c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.581+0000 7f94e1389700 1 --2- 172.21.15.7:0/1229665707 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f94dc0b3ae0 0x7f94dc151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.581+0000 7f94e1389700 1 --2- 172.21.15.7:0/1229665707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94dc06aa90 0x7f94dc14c260 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.581+0000 7f94e1389700 1 -- 172.21.15.7:0/1229665707 >> 172.21.15.7:0/1229665707 conn(0x7f94dc0b0f00 msgr2=0x7f94dc0afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:44.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.585+0000 7f94e1389700 1 -- 172.21.15.7:0/1229665707 shutdown_connections 2024-09-13T18:52:44.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.585+0000 7f94e1389700 1 -- 172.21.15.7:0/1229665707 wait complete. 2024-09-13T18:52:44.602 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:52:15.554683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:52:16.896879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:52:44.604 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:52:44.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.721+0000 7f73730d9700 1 Processor -- start 2024-09-13T18:52:44.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.721+0000 7f73730d9700 1 -- start start 2024-09-13T18:52:44.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73730d9700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f736c0cd420 0x7f736c0cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:44.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73730d9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f736c06aba0 0x7f736c06afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:44.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73730d9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f736c15ad10 con 0x7f736c15a930 2024-09-13T18:52:44.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73730d9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f736c0cd0a0 con 0x7f736c06aba0 2024-09-13T18:52:44.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73730d9700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f736c0718e0 con 0x7f736c0cd420 2024-09-13T18:52:44.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73728d8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f736c06aba0 0x7f736c06afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:44.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73720d7700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f736c15a930 0x7f736c071d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57794/0 (socket says 172.21.15.7:57794) 2024-09-13T18:52:44.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73720d7700 1 -- 172.21.15.7:0/583592069 learned_addr learned my addr 172.21.15.7:0/583592069 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:44.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73718d6700 1 --2- 172.21.15.7:0/583592069 >> v2:172.21.15.171:3300/0 conn(0x7f736c0cd420 0x7f736c0cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:44.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73728d8700 1 -- 172.21.15.7:0/583592069 >> v2:172.21.15.171:3300/0 conn(0x7f736c0cd420 msgr2=0x7f736c0cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:44.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73728d8700 1 --2- 172.21.15.7:0/583592069 >> v2:172.21.15.171:3300/0 conn(0x7f736c0cd420 0x7f736c0cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73728d8700 1 -- 172.21.15.7:0/583592069 >> v1:172.21.15.7:6789/0 conn(0x7f736c15a930 legacy=0x7f736c071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:44.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73728d8700 1 -- 172.21.15.7:0/583592069 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736c06b510 con 0x7f736c06aba0 2024-09-13T18:52:44.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73728d8700 1 --2- 172.21.15.7:0/583592069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f736c06aba0 0x7f736c06afd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f736c072870 tx=0x7f736400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aec7084a5ee6c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:44.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73710d5700 1 -- 172.21.15.7:0/583592069 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736400e070 con 0x7f736c06aba0 2024-09-13T18:52:44.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73710d5700 1 -- 172.21.15.7:0/583592069 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f736400bb20 con 0x7f736c06aba0 2024-09-13T18:52:44.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73710d5700 1 -- 172.21.15.7:0/583592069 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7364012430 con 0x7f736c06aba0 2024-09-13T18:52:44.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73730d9700 1 -- 172.21.15.7:0/583592069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f736c06aba0 msgr2=0x7f736c06afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:44.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73730d9700 1 --2- 172.21.15.7:0/583592069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f736c06aba0 0x7f736c06afd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f736c072870 tx=0x7f736400b330 comp rx=0 tx=0).stop 2024-09-13T18:52:44.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73730d9700 1 -- 172.21.15.7:0/583592069 shutdown_connections 2024-09-13T18:52:44.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73730d9700 1 --2- 172.21.15.7:0/583592069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f736c06aba0 0x7f736c06afd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73730d9700 1 --2- 172.21.15.7:0/583592069 >> v2:172.21.15.171:3300/0 conn(0x7f736c0cd420 0x7f736c0cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.725+0000 7f73730d9700 1 -- 172.21.15.7:0/583592069 >> 172.21.15.7:0/583592069 conn(0x7f736c0b1810 msgr2=0x7f736c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:44.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.729+0000 7f73730d9700 1 -- 172.21.15.7:0/583592069 shutdown_connections 2024-09-13T18:52:44.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.729+0000 7f73730d9700 1 -- 172.21.15.7:0/583592069 wait complete. 2024-09-13T18:52:44.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.729+0000 7f73730d9700 1 Processor -- start 2024-09-13T18:52:44.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.729+0000 7f73730d9700 1 -- start start 2024-09-13T18:52:44.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73730d9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736c06aba0 0x7f736c14ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73730d9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f736c0cd420 0x7f736c153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73730d9700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f736c15a930 0x7f736c153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73730d9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736c159ab0 con 0x7f736c0cd420 2024-09-13T18:52:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73730d9700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736c159210 con 0x7f736c06aba0 2024-09-13T18:52:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73730d9700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736c1594f0 con 0x7f736c15a930 2024-09-13T18:52:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73718d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f736c0cd420 0x7f736c153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73718d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f736c0cd420 0x7f736c153d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45946/0 (socket says 172.21.15.7:45946) 2024-09-13T18:52:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73718d6700 1 -- 172.21.15.7:0/1607899508 learned_addr learned my addr 172.21.15.7:0/1607899508 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73728d8700 1 --2- 172.21.15.7:0/1607899508 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f736c15a930 0x7f736c153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73720d7700 1 --2- 172.21.15.7:0/1607899508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736c06aba0 0x7f736c14ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73718d6700 1 -- 172.21.15.7:0/1607899508 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f736c15a930 msgr2=0x7f736c153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73718d6700 1 --2- 172.21.15.7:0/1607899508 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f736c15a930 0x7f736c153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73718d6700 1 -- 172.21.15.7:0/1607899508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736c06aba0 msgr2=0x7f736c14ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73718d6700 1 --2- 172.21.15.7:0/1607899508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736c06aba0 0x7f736c14ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73718d6700 1 -- 172.21.15.7:0/1607899508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7364009a70 con 0x7f736c0cd420 2024-09-13T18:52:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73718d6700 1 --2- 172.21.15.7:0/1607899508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f736c0cd420 0x7f736c153d30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f735c00e840 tx=0x7f735c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f7362ffd700 1 -- 172.21.15.7:0/1607899508 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c0095a0 con 0x7f736c0cd420 2024-09-13T18:52:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f7362ffd700 1 -- 172.21.15.7:0/1607899508 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735c00c690 con 0x7f736c0cd420 2024-09-13T18:52:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73730d9700 1 -- 172.21.15.7:0/1607899508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f736c1597d0 con 0x7f736c0cd420 2024-09-13T18:52:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f7362ffd700 1 -- 172.21.15.7:0/1607899508 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c00ca00 con 0x7f736c0cd420 2024-09-13T18:52:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73730d9700 1 -- 172.21.15.7:0/1607899508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f736c154270 con 0x7f736c0cd420 2024-09-13T18:52:44.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.733+0000 7f73730d9700 1 -- 172.21.15.7:0/1607899508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f736c15c080 con 0x7f736c0cd420 2024-09-13T18:52:44.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.737+0000 7f7362ffd700 1 -- 172.21.15.7:0/1607899508 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f735c00cb60 con 0x7f736c0cd420 2024-09-13T18:52:44.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.737+0000 7f7362ffd700 1 --2- 172.21.15.7:0/1607899508 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7350066730 0x7f7350068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:44.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.737+0000 7f7362ffd700 1 -- 172.21.15.7:0/1607899508 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f735c0840f0 con 0x7f736c0cd420 2024-09-13T18:52:44.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.737+0000 7f73720d7700 1 --2- 172.21.15.7:0/1607899508 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7350066730 0x7f7350068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:44.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.737+0000 7f73720d7700 1 --2- 172.21.15.7:0/1607899508 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7350066730 0x7f7350068bf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7368005930 tx=0x7f7368009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:44.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.741+0000 7f7362ffd700 1 -- 172.21.15.7:0/1607899508 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f736c15c080 con 0x7f736c0cd420 2024-09-13T18:52:44.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.913+0000 7f73730d9700 1 -- 172.21.15.7:0/1607899508 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f736c15c080 con 0x7f736c0cd420 2024-09-13T18:52:44.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.913+0000 7f7362ffd700 1 -- 172.21.15.7:0/1607899508 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f736c15c080 con 0x7f736c0cd420 2024-09-13T18:52:44.921 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:52:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.917+0000 7f73730d9700 1 -- 172.21.15.7:0/1607899508 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7350066730 msgr2=0x7f7350068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.917+0000 7f73730d9700 1 --2- 172.21.15.7:0/1607899508 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7350066730 0x7f7350068bf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7368005930 tx=0x7f7368009040 comp rx=0 tx=0).stop 2024-09-13T18:52:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.917+0000 7f73730d9700 1 -- 172.21.15.7:0/1607899508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f736c0cd420 msgr2=0x7f736c153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.917+0000 7f73730d9700 1 --2- 172.21.15.7:0/1607899508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f736c0cd420 0x7f736c153d30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f735c00e840 tx=0x7f735c00ec00 comp rx=0 tx=0).stop 2024-09-13T18:52:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.917+0000 7f73730d9700 1 -- 172.21.15.7:0/1607899508 shutdown_connections 2024-09-13T18:52:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.917+0000 7f73730d9700 1 --2- 172.21.15.7:0/1607899508 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7350066730 0x7f7350068bf0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.917+0000 7f73730d9700 1 --2- 172.21.15.7:0/1607899508 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f736c15a930 0x7f736c153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.917+0000 7f73730d9700 1 --2- 172.21.15.7:0/1607899508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f736c0cd420 0x7f736c153d30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.917+0000 7f73730d9700 1 --2- 172.21.15.7:0/1607899508 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736c06aba0 0x7f736c14ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:44.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.917+0000 7f73730d9700 1 -- 172.21.15.7:0/1607899508 >> 172.21.15.7:0/1607899508 conn(0x7f736c0b1810 msgr2=0x7f736c0cf910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:44.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.917+0000 7f73730d9700 1 -- 172.21.15.7:0/1607899508 shutdown_connections 2024-09-13T18:52:44.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:44.917+0000 7f73730d9700 1 -- 172.21.15.7:0/1607899508 wait complete. 2024-09-13T18:52:44.936 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:52:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f5440e1a700 1 Processor -- start 2024-09-13T18:52:45.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f5440e1a700 1 -- start start 2024-09-13T18:52:45.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f5440e1a700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f543c15cda0 0x7f543c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:45.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f5440e1a700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f543c0ceb70 0x7f543c0b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:45.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f5440e1a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543c0b8430 con 0x7f543c06a520 2024-09-13T18:52:45.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f5440e1a700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f543c06be40 con 0x7f543c0ceb70 2024-09-13T18:52:45.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f5440e1a700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543c0cf350 con 0x7f543c15cda0 2024-09-13T18:52:45.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f543bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f543c06a520 0x7f543c0d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57802/0 (socket says 172.21.15.7:57802) 2024-09-13T18:52:45.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f543bfff700 1 -- 172.21.15.7:0/1386551258 learned_addr learned my addr 172.21.15.7:0/1386551258 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:45.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f543affd700 1 --2- 172.21.15.7:0/1386551258 >> v2:172.21.15.171:3300/0 conn(0x7f543c0ceb70 0x7f543c0b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:45.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f543b7fe700 1 --2- 172.21.15.7:0/1386551258 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f543c15cda0 0x7f543c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:45.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f543a7fc700 1 -- 172.21.15.7:0/1386551258 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1608830501 0 0) 0x7f543c0b8430 con 0x7f543c06a520 2024-09-13T18:52:45.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f543a7fc700 1 -- 172.21.15.7:0/1386551258 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54280035d0 con 0x7f543c06a520 2024-09-13T18:52:45.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f543affd700 1 -- 172.21.15.7:0/1386551258 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f543c15cda0 msgr2=0x7f543c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:45.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f543affd700 1 --2- 172.21.15.7:0/1386551258 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f543c15cda0 0x7f543c06b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f543affd700 1 -- 172.21.15.7:0/1386551258 >> v1:172.21.15.7:6789/0 conn(0x7f543c06a520 legacy=0x7f543c0d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f543affd700 1 -- 172.21.15.7:0/1386551258 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c0d0880 con 0x7f543c0ceb70 2024-09-13T18:52:45.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f543b7fe700 1 --2- 172.21.15.7:0/1386551258 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f543c15cda0 0x7f543c06b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:45.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f543affd700 1 --2- 172.21.15.7:0/1386551258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f543c0ceb70 0x7f543c0b5de0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5424009850 tx=0x7f5424009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bea1081401b16282 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:45.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f543a7fc700 1 -- 172.21.15.7:0/1386551258 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424011830 con 0x7f543c0ceb70 2024-09-13T18:52:45.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f543a7fc700 1 -- 172.21.15.7:0/1386551258 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5424011990 con 0x7f543c0ceb70 2024-09-13T18:52:45.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.049+0000 7f543a7fc700 1 -- 172.21.15.7:0/1386551258 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424011d00 con 0x7f543c0ceb70 2024-09-13T18:52:45.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.053+0000 7f5440e1a700 1 -- 172.21.15.7:0/1386551258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f543c0ceb70 msgr2=0x7f543c0b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.053+0000 7f5440e1a700 1 --2- 172.21.15.7:0/1386551258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f543c0ceb70 0x7f543c0b5de0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5424009850 tx=0x7f5424009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:45.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.053+0000 7f5440e1a700 1 -- 172.21.15.7:0/1386551258 shutdown_connections 2024-09-13T18:52:45.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.053+0000 7f5440e1a700 1 --2- 172.21.15.7:0/1386551258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f543c0ceb70 0x7f543c0b5de0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.053+0000 7f5440e1a700 1 --2- 172.21.15.7:0/1386551258 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f543c15cda0 0x7f543c06b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.053+0000 7f5440e1a700 1 -- 172.21.15.7:0/1386551258 >> 172.21.15.7:0/1386551258 conn(0x7f543c0d2cb0 msgr2=0x7f543c0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:45.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.053+0000 7f5440e1a700 1 -- 172.21.15.7:0/1386551258 shutdown_connections 2024-09-13T18:52:45.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.053+0000 7f5440e1a700 1 -- 172.21.15.7:0/1386551258 wait complete. 2024-09-13T18:52:45.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.053+0000 7f5440e1a700 1 Processor -- start 2024-09-13T18:52:45.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.053+0000 7f5440e1a700 1 -- start start 2024-09-13T18:52:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f5440e1a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f543c06a520 0x7f543c152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f5440e1a700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f543c0ceb70 0x7f543c15f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f5440e1a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f543c15cda0 0x7f543c15d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f5440e1a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c15f960 con 0x7f543c06a520 2024-09-13T18:52:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f5440e1a700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c15fc40 con 0x7f543c15cda0 2024-09-13T18:52:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f5440e1a700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543c15ff20 con 0x7f543c0ceb70 2024-09-13T18:52:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f543b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f543c06a520 0x7f543c152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f543b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f543c06a520 0x7f543c152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45958/0 (socket says 172.21.15.7:45958) 2024-09-13T18:52:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f543b7fe700 1 -- 172.21.15.7:0/3188938968 learned_addr learned my addr 172.21.15.7:0/3188938968 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:45.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f543affd700 1 --2- 172.21.15.7:0/3188938968 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f543c0ceb70 0x7f543c15f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:45.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f543bfff700 1 --2- 172.21.15.7:0/3188938968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f543c15cda0 0x7f543c15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:45.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f543b7fe700 1 -- 172.21.15.7:0/3188938968 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f543c0ceb70 msgr2=0x7f543c15f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f543b7fe700 1 --2- 172.21.15.7:0/3188938968 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f543c0ceb70 0x7f543c15f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f543b7fe700 1 -- 172.21.15.7:0/3188938968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f543c15cda0 msgr2=0x7f543c15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f543b7fe700 1 --2- 172.21.15.7:0/3188938968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f543c15cda0 0x7f543c15d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f543b7fe700 1 -- 172.21.15.7:0/3188938968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c0094b0 con 0x7f543c06a520 2024-09-13T18:52:45.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f543b7fe700 1 --2- 172.21.15.7:0/3188938968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f543c06a520 0x7f543c152340 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f542c00ebe0 tx=0x7f542c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:45.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f5433fff700 1 -- 172.21.15.7:0/3188938968 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c009770 con 0x7f543c06a520 2024-09-13T18:52:45.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f5440e1a700 1 -- 172.21.15.7:0/3188938968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54240094b0 con 0x7f543c06a520 2024-09-13T18:52:45.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f5433fff700 1 -- 172.21.15.7:0/3188938968 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542c00c850 con 0x7f543c06a520 2024-09-13T18:52:45.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f5433fff700 1 -- 172.21.15.7:0/3188938968 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c00cbe0 con 0x7f543c06a520 2024-09-13T18:52:45.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f5440e1a700 1 -- 172.21.15.7:0/3188938968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f543c15f5c0 con 0x7f543c06a520 2024-09-13T18:52:45.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.057+0000 7f5440e1a700 1 -- 172.21.15.7:0/3188938968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f543c06b900 con 0x7f543c06a520 2024-09-13T18:52:45.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.061+0000 7f5433fff700 1 -- 172.21.15.7:0/3188938968 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f542c00cd40 con 0x7f543c06a520 2024-09-13T18:52:45.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.061+0000 7f5433fff700 1 --2- 172.21.15.7:0/3188938968 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f541c066730 0x7f541c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:45.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.061+0000 7f5433fff700 1 -- 172.21.15.7:0/3188938968 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f542c0841a0 con 0x7f543c06a520 2024-09-13T18:52:45.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.061+0000 7f543affd700 1 --2- 172.21.15.7:0/3188938968 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f541c066730 0x7f541c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:45.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.061+0000 7f543affd700 1 --2- 172.21.15.7:0/3188938968 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f541c066730 0x7f541c068bf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5424009850 tx=0x7f5424008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:45.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.069+0000 7f5433fff700 1 -- 172.21.15.7:0/3188938968 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f542c051520 con 0x7f543c06a520 2024-09-13T18:52:45.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.245+0000 7f5440e1a700 1 -- 172.21.15.7:0/3188938968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f543c0c69a0 con 0x7f543c06a520 2024-09-13T18:52:45.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.245+0000 7f5433fff700 1 -- 172.21.15.7:0/3188938968 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f542c054b40 con 0x7f543c06a520 2024-09-13T18:52:45.250 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:52:45.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.249+0000 7f5440e1a700 1 -- 172.21.15.7:0/3188938968 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f541c066730 msgr2=0x7f541c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.249+0000 7f5440e1a700 1 --2- 172.21.15.7:0/3188938968 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f541c066730 0x7f541c068bf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5424009850 tx=0x7f5424008030 comp rx=0 tx=0).stop 2024-09-13T18:52:45.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.249+0000 7f5440e1a700 1 -- 172.21.15.7:0/3188938968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f543c06a520 msgr2=0x7f543c152340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.249+0000 7f5440e1a700 1 --2- 172.21.15.7:0/3188938968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f543c06a520 0x7f543c152340 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f542c00ebe0 tx=0x7f542c00eef0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.249+0000 7f5440e1a700 1 -- 172.21.15.7:0/3188938968 shutdown_connections 2024-09-13T18:52:45.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.249+0000 7f5440e1a700 1 --2- 172.21.15.7:0/3188938968 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f541c066730 0x7f541c068bf0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.249+0000 7f5440e1a700 1 --2- 172.21.15.7:0/3188938968 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f543c15cda0 0x7f543c15d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.249+0000 7f5440e1a700 1 --2- 172.21.15.7:0/3188938968 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f543c0ceb70 0x7f543c15f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.249+0000 7f5440e1a700 1 --2- 172.21.15.7:0/3188938968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f543c06a520 0x7f543c152340 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.249+0000 7f5440e1a700 1 -- 172.21.15.7:0/3188938968 >> 172.21.15.7:0/3188938968 conn(0x7f543c0d2cb0 msgr2=0x7f543c15b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:45.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.249+0000 7f5440e1a700 1 -- 172.21.15.7:0/3188938968 shutdown_connections 2024-09-13T18:52:45.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.249+0000 7f5440e1a700 1 -- 172.21.15.7:0/3188938968 wait complete. 2024-09-13T18:52:45.271 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:52:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.385+0000 7f293bfff700 1 Processor -- start 2024-09-13T18:52:45.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.385+0000 7f293bfff700 1 -- start start 2024-09-13T18:52:45.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.385+0000 7f293bfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f293c1573e0 0x7f293c070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:45.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.385+0000 7f293bfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f293c071540 con 0x7f293c15a630 2024-09-13T18:52:45.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.385+0000 7f293bfff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f293c15ac50 con 0x7f293c0710e0 2024-09-13T18:52:45.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.385+0000 7f293bfff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293c0cc1b0 con 0x7f293c1573e0 2024-09-13T18:52:45.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.385+0000 7f293b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f293c15a630 0x7f293c06c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57804/0 (socket says 172.21.15.7:57804) 2024-09-13T18:52:45.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.385+0000 7f293b7fe700 1 -- 172.21.15.7:0/62611102 learned_addr learned my addr 172.21.15.7:0/62611102 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:45.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.385+0000 7f293affd700 1 --2- 172.21.15.7:0/62611102 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f293c1573e0 0x7f293c070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.385+0000 7f2939ffb700 1 -- 172.21.15.7:0/62611102 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720289293 0 0) 0x7f293c15ac50 con 0x7f293c0710e0 2024-09-13T18:52:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.385+0000 7f2939ffb700 1 -- 172.21.15.7:0/62611102 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29280035d0 con 0x7f293c0710e0 2024-09-13T18:52:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.385+0000 7f2939ffb700 1 -- 172.21.15.7:0/62611102 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634756797 0 0) 0x7f293c071540 con 0x7f293c15a630 2024-09-13T18:52:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.385+0000 7f2939ffb700 1 -- 172.21.15.7:0/62611102 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f293c15ac50 con 0x7f293c15a630 2024-09-13T18:52:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.385+0000 7f293affd700 1 -- 172.21.15.7:0/62611102 >> v1:172.21.15.171:6789/0 conn(0x7f293c0710e0 legacy=0x7f293c159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.385+0000 7f293affd700 1 -- 172.21.15.7:0/62611102 >> v1:172.21.15.7:6789/0 conn(0x7f293c15a630 legacy=0x7f293c06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.385+0000 7f293affd700 1 -- 172.21.15.7:0/62611102 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293c0cb3f0 con 0x7f293c1573e0 2024-09-13T18:52:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.389+0000 7f293affd700 1 --2- 172.21.15.7:0/62611102 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f293c1573e0 0x7f293c070ba0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2924009df0 tx=0x7f292400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa38dc0f4d0e0521 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:45.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.389+0000 7f2939ffb700 1 -- 172.21.15.7:0/62611102 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292400e070 con 0x7f293c1573e0 2024-09-13T18:52:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.389+0000 7f2939ffb700 1 -- 172.21.15.7:0/62611102 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292400bae0 con 0x7f293c1573e0 2024-09-13T18:52:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.389+0000 7f2939ffb700 1 -- 172.21.15.7:0/62611102 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292400be90 con 0x7f293c1573e0 2024-09-13T18:52:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.389+0000 7f293bfff700 1 -- 172.21.15.7:0/62611102 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f293c1573e0 msgr2=0x7f293c070ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.389+0000 7f293bfff700 1 --2- 172.21.15.7:0/62611102 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f293c1573e0 0x7f293c070ba0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2924009df0 tx=0x7f292400b330 comp rx=0 tx=0).stop 2024-09-13T18:52:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.389+0000 7f293bfff700 1 -- 172.21.15.7:0/62611102 shutdown_connections 2024-09-13T18:52:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.389+0000 7f293bfff700 1 --2- 172.21.15.7:0/62611102 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f293c1573e0 0x7f293c070ba0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.389+0000 7f293bfff700 1 -- 172.21.15.7:0/62611102 >> 172.21.15.7:0/62611102 conn(0x7f293c0b0e10 msgr2=0x7f293c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.389+0000 7f293bfff700 1 -- 172.21.15.7:0/62611102 shutdown_connections 2024-09-13T18:52:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.389+0000 7f293bfff700 1 -- 172.21.15.7:0/62611102 wait complete. 2024-09-13T18:52:45.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.389+0000 7f293bfff700 1 Processor -- start 2024-09-13T18:52:45.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.389+0000 7f293bfff700 1 -- start start 2024-09-13T18:52:45.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f293c0710e0 0x7f293c159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:45.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293bfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f293c1573e0 0x7f293c142200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:45.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f293c15a630 0x7f293c14fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293bfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c142740 con 0x7f293c15a630 2024-09-13T18:52:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293bfff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f293c1501e0 con 0x7f293c0710e0 2024-09-13T18:52:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293bfff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f293c150480 con 0x7f293c1573e0 2024-09-13T18:52:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f293c15a630 0x7f293c14fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f293c15a630 0x7f293c14fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45960/0 (socket says 172.21.15.7:45960) 2024-09-13T18:52:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293b7fe700 1 -- 172.21.15.7:0/737967443 learned_addr learned my addr 172.21.15.7:0/737967443 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293affd700 1 --2- 172.21.15.7:0/737967443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f293c0710e0 0x7f293c159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293a7fc700 1 --2- 172.21.15.7:0/737967443 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f293c1573e0 0x7f293c142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293b7fe700 1 -- 172.21.15.7:0/737967443 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f293c1573e0 msgr2=0x7f293c142200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293b7fe700 1 --2- 172.21.15.7:0/737967443 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f293c1573e0 0x7f293c142200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293b7fe700 1 -- 172.21.15.7:0/737967443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f293c0710e0 msgr2=0x7f293c159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293b7fe700 1 --2- 172.21.15.7:0/737967443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f293c0710e0 0x7f293c159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293b7fe700 1 -- 172.21.15.7:0/737967443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2924009a70 con 0x7f293c15a630 2024-09-13T18:52:45.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293b7fe700 1 --2- 172.21.15.7:0/737967443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f293c15a630 0x7f293c14fca0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f293400c4d0 tx=0x7f293400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:45.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f2933fff700 1 -- 172.21.15.7:0/737967443 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2934005d90 con 0x7f293c15a630 2024-09-13T18:52:45.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f2933fff700 1 -- 172.21.15.7:0/737967443 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29340070d0 con 0x7f293c15a630 2024-09-13T18:52:45.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293bfff700 1 -- 172.21.15.7:0/737967443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f293c150760 con 0x7f293c15a630 2024-09-13T18:52:45.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f2933fff700 1 -- 172.21.15.7:0/737967443 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2934007440 con 0x7f293c15a630 2024-09-13T18:52:45.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293bfff700 1 -- 172.21.15.7:0/737967443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f293c150d20 con 0x7f293c15a630 2024-09-13T18:52:45.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.393+0000 7f293bfff700 1 -- 172.21.15.7:0/737967443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f293c159f20 con 0x7f293c15a630 2024-09-13T18:52:45.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.397+0000 7f2933fff700 1 -- 172.21.15.7:0/737967443 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f29340075a0 con 0x7f293c15a630 2024-09-13T18:52:45.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.397+0000 7f2933fff700 1 --2- 172.21.15.7:0/737967443 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f291c066730 0x7f291c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:45.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.397+0000 7f2933fff700 1 -- 172.21.15.7:0/737967443 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2934083560 con 0x7f293c15a630 2024-09-13T18:52:45.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.397+0000 7f293affd700 1 --2- 172.21.15.7:0/737967443 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f291c066730 0x7f291c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:45.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.397+0000 7f293affd700 1 --2- 172.21.15.7:0/737967443 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f291c066730 0x7f291c068bf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2924014040 tx=0x7f292401d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:45.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.405+0000 7f2933fff700 1 -- 172.21.15.7:0/737967443 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2934050900 con 0x7f293c15a630 2024-09-13T18:52:45.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.577+0000 7f293bfff700 1 -- 172.21.15.7:0/737967443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f293c0cadf0 con 0x7f293c15a630 2024-09-13T18:52:45.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.577+0000 7f2933fff700 1 -- 172.21.15.7:0/737967443 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2934053f20 con 0x7f293c15a630 2024-09-13T18:52:45.584 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:52:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.581+0000 7f293bfff700 1 -- 172.21.15.7:0/737967443 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f291c066730 msgr2=0x7f291c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.581+0000 7f293bfff700 1 --2- 172.21.15.7:0/737967443 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f291c066730 0x7f291c068bf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2924014040 tx=0x7f292401d040 comp rx=0 tx=0).stop 2024-09-13T18:52:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.581+0000 7f293bfff700 1 -- 172.21.15.7:0/737967443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f293c15a630 msgr2=0x7f293c14fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.581+0000 7f293bfff700 1 --2- 172.21.15.7:0/737967443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f293c15a630 0x7f293c14fca0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f293400c4d0 tx=0x7f293400c890 comp rx=0 tx=0).stop 2024-09-13T18:52:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.581+0000 7f293bfff700 1 -- 172.21.15.7:0/737967443 shutdown_connections 2024-09-13T18:52:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.581+0000 7f293bfff700 1 --2- 172.21.15.7:0/737967443 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f291c066730 0x7f291c068bf0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.581+0000 7f293bfff700 1 --2- 172.21.15.7:0/737967443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f293c15a630 0x7f293c14fca0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.581+0000 7f293bfff700 1 --2- 172.21.15.7:0/737967443 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f293c1573e0 0x7f293c142200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.581+0000 7f293bfff700 1 --2- 172.21.15.7:0/737967443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f293c0710e0 0x7f293c159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.581+0000 7f293bfff700 1 -- 172.21.15.7:0/737967443 >> 172.21.15.7:0/737967443 conn(0x7f293c0b0e10 msgr2=0x7f293c070810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.581+0000 7f293bfff700 1 -- 172.21.15.7:0/737967443 shutdown_connections 2024-09-13T18:52:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.581+0000 7f293bfff700 1 -- 172.21.15.7:0/737967443 wait complete. 2024-09-13T18:52:45.602 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:45.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.713+0000 7f06b22c1700 1 Processor -- start 2024-09-13T18:52:45.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.713+0000 7f06b22c1700 1 -- start start 2024-09-13T18:52:45.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.713+0000 7f06b22c1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06ac06fb30 0x7f06ac0adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:45.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.713+0000 7f06b22c1700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06ac06d150 0x7f06ac06d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:45.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.713+0000 7f06b22c1700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06ac06a780 con 0x7f06ac06bb90 2024-09-13T18:52:45.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.713+0000 7f06b22c1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06ac06cd30 con 0x7f06ac06fb30 2024-09-13T18:52:45.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.713+0000 7f06b22c1700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac159470 con 0x7f06ac06d150 2024-09-13T18:52:45.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.713+0000 7f06b0abe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f06ac06bb90 0x7f06ac0cbf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57818/0 (socket says 172.21.15.7:57818) 2024-09-13T18:52:45.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.713+0000 7f06b12bf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06ac06fb30 0x7f06ac0adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:45.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.713+0000 7f06b0abe700 1 -- 172.21.15.7:0/2984157702 learned_addr learned my addr 172.21.15.7:0/2984157702 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:45.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.713+0000 7f06b1ac0700 1 --2- 172.21.15.7:0/2984157702 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06ac06d150 0x7f06ac06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:45.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.713+0000 7f06b12bf700 1 -- 172.21.15.7:0/2984157702 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06ac06d150 msgr2=0x7f06ac06d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:45.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.713+0000 7f06b12bf700 1 --2- 172.21.15.7:0/2984157702 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06ac06d150 0x7f06ac06d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.713+0000 7f06b12bf700 1 -- 172.21.15.7:0/2984157702 >> v1:172.21.15.7:6789/0 conn(0x7f06ac06bb90 legacy=0x7f06ac0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.713+0000 7f06a3fff700 1 -- 172.21.15.7:0/2984157702 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2365888367 0 0) 0x7f06ac06a780 con 0x7f06ac06bb90 2024-09-13T18:52:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.713+0000 7f06b12bf700 1 -- 172.21.15.7:0/2984157702 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac0cc610 con 0x7f06ac06fb30 2024-09-13T18:52:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.717+0000 7f06b12bf700 1 --2- 172.21.15.7:0/2984157702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06ac06fb30 0x7f06ac0adbd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f069c009850 tx=0x7f069c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac2dcda73590a5b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.717+0000 7f06a3fff700 1 -- 172.21.15.7:0/2984157702 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c011830 con 0x7f06ac06fb30 2024-09-13T18:52:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.717+0000 7f06a3fff700 1 -- 172.21.15.7:0/2984157702 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069c011990 con 0x7f06ac06fb30 2024-09-13T18:52:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.717+0000 7f06a3fff700 1 -- 172.21.15.7:0/2984157702 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c011d40 con 0x7f06ac06fb30 2024-09-13T18:52:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.717+0000 7f06b22c1700 1 -- 172.21.15.7:0/2984157702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06ac06fb30 msgr2=0x7f06ac0adbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.717+0000 7f06b22c1700 1 --2- 172.21.15.7:0/2984157702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06ac06fb30 0x7f06ac0adbd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f069c009850 tx=0x7f069c009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.717+0000 7f06b22c1700 1 -- 172.21.15.7:0/2984157702 shutdown_connections 2024-09-13T18:52:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.717+0000 7f06b22c1700 1 --2- 172.21.15.7:0/2984157702 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06ac06d150 0x7f06ac06d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.717+0000 7f06b22c1700 1 --2- 172.21.15.7:0/2984157702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06ac06fb30 0x7f06ac0adbd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.717+0000 7f06b22c1700 1 -- 172.21.15.7:0/2984157702 >> 172.21.15.7:0/2984157702 conn(0x7f06ac0b0f00 msgr2=0x7f06ac0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.717+0000 7f06b22c1700 1 -- 172.21.15.7:0/2984157702 shutdown_connections 2024-09-13T18:52:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.717+0000 7f06b22c1700 1 -- 172.21.15.7:0/2984157702 wait complete. 2024-09-13T18:52:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.721+0000 7f06b22c1700 1 Processor -- start 2024-09-13T18:52:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.721+0000 7f06b22c1700 1 -- start start 2024-09-13T18:52:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.721+0000 7f06b22c1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06ac06bb90 0x7f06ac144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.721+0000 7f06b22c1700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06ac06d150 0x7f06ac14a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.721+0000 7f06b22c1700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06ac06fb30 0x7f06ac15aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.721+0000 7f06b22c1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac14ac30 con 0x7f06ac06bb90 2024-09-13T18:52:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.721+0000 7f06b22c1700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac14a260 con 0x7f06ac06fb30 2024-09-13T18:52:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.721+0000 7f06b22c1700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac159e60 con 0x7f06ac06d150 2024-09-13T18:52:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.721+0000 7f06b12bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06ac06bb90 0x7f06ac144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.721+0000 7f06b12bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06ac06bb90 0x7f06ac144b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45978/0 (socket says 172.21.15.7:45978) 2024-09-13T18:52:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.721+0000 7f06b12bf700 1 -- 172.21.15.7:0/1264527302 learned_addr learned my addr 172.21.15.7:0/1264527302 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.721+0000 7f06b1ac0700 1 --2- 172.21.15.7:0/1264527302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06ac06fb30 0x7f06ac15aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.721+0000 7f06b0abe700 1 --2- 172.21.15.7:0/1264527302 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06ac06d150 0x7f06ac14a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.721+0000 7f06b12bf700 1 -- 172.21.15.7:0/1264527302 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06ac06d150 msgr2=0x7f06ac14a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.721+0000 7f06b12bf700 1 --2- 172.21.15.7:0/1264527302 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06ac06d150 0x7f06ac14a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.721+0000 7f06b12bf700 1 -- 172.21.15.7:0/1264527302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06ac06fb30 msgr2=0x7f06ac15aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.721+0000 7f06b12bf700 1 --2- 172.21.15.7:0/1264527302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06ac06fb30 0x7f06ac15aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.721+0000 7f06b12bf700 1 -- 172.21.15.7:0/1264527302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c0094b0 con 0x7f06ac06bb90 2024-09-13T18:52:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.725+0000 7f06b12bf700 1 --2- 172.21.15.7:0/1264527302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06ac06bb90 0x7f06ac144b40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f069c014040 tx=0x7f069c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.725+0000 7f06a1ffb700 1 -- 172.21.15.7:0/1264527302 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c019980 con 0x7f06ac06bb90 2024-09-13T18:52:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.725+0000 7f06a1ffb700 1 -- 172.21.15.7:0/1264527302 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069c01e070 con 0x7f06ac06bb90 2024-09-13T18:52:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.725+0000 7f06b22c1700 1 -- 172.21.15.7:0/1264527302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ac149650 con 0x7f06ac06bb90 2024-09-13T18:52:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.725+0000 7f06a1ffb700 1 -- 172.21.15.7:0/1264527302 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c019d20 con 0x7f06ac06bb90 2024-09-13T18:52:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.725+0000 7f06b22c1700 1 -- 172.21.15.7:0/1264527302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ac149c10 con 0x7f06ac06bb90 2024-09-13T18:52:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.725+0000 7f06b22c1700 1 -- 172.21.15.7:0/1264527302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06ac06d570 con 0x7f06ac06bb90 2024-09-13T18:52:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.725+0000 7f06a1ffb700 1 -- 172.21.15.7:0/1264527302 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f069c02d450 con 0x7f06ac06bb90 2024-09-13T18:52:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.725+0000 7f06a1ffb700 1 --2- 172.21.15.7:0/1264527302 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0690066730 0x7f0690068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.725+0000 7f06a1ffb700 1 -- 172.21.15.7:0/1264527302 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f069c085000 con 0x7f06ac06bb90 2024-09-13T18:52:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.725+0000 7f06b0abe700 1 --2- 172.21.15.7:0/1264527302 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0690066730 0x7f0690068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.729+0000 7f06b0abe700 1 --2- 172.21.15.7:0/1264527302 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0690066730 0x7f0690068bf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f06ac164c50 tx=0x7f06a8001b70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.733+0000 7f06a1ffb700 1 -- 172.21.15.7:0/1264527302 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f069c052400 con 0x7f06ac06bb90 2024-09-13T18:52:45.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.909+0000 7f06b22c1700 1 -- 172.21.15.7:0/1264527302 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06ac14ed50 con 0x7f06ac06bb90 2024-09-13T18:52:45.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.913+0000 7f06a1ffb700 1 -- 172.21.15.7:0/1264527302 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f069c020090 con 0x7f06ac06bb90 2024-09-13T18:52:45.919 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:45.919 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:45.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.917+0000 7f06b22c1700 1 -- 172.21.15.7:0/1264527302 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0690066730 msgr2=0x7f0690068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.917+0000 7f06b22c1700 1 --2- 172.21.15.7:0/1264527302 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0690066730 0x7f0690068bf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f06ac164c50 tx=0x7f06a8001b70 comp rx=0 tx=0).stop 2024-09-13T18:52:45.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.917+0000 7f06b22c1700 1 -- 172.21.15.7:0/1264527302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06ac06bb90 msgr2=0x7f06ac144b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:45.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.917+0000 7f06b22c1700 1 --2- 172.21.15.7:0/1264527302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06ac06bb90 0x7f06ac144b40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f069c014040 tx=0x7f069c004770 comp rx=0 tx=0).stop 2024-09-13T18:52:45.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.917+0000 7f06b22c1700 1 -- 172.21.15.7:0/1264527302 shutdown_connections 2024-09-13T18:52:45.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.917+0000 7f06b22c1700 1 --2- 172.21.15.7:0/1264527302 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0690066730 0x7f0690068bf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.917+0000 7f06b22c1700 1 --2- 172.21.15.7:0/1264527302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06ac06fb30 0x7f06ac15aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.917+0000 7f06b22c1700 1 --2- 172.21.15.7:0/1264527302 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f06ac06d150 0x7f06ac14a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.917+0000 7f06b22c1700 1 --2- 172.21.15.7:0/1264527302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06ac06bb90 0x7f06ac144b40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:45.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.917+0000 7f06b22c1700 1 -- 172.21.15.7:0/1264527302 >> 172.21.15.7:0/1264527302 conn(0x7f06ac0b0f00 msgr2=0x7f06ac0af540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:45.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.917+0000 7f06b22c1700 1 -- 172.21.15.7:0/1264527302 shutdown_connections 2024-09-13T18:52:45.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:45.917+0000 7f06b22c1700 1 -- 172.21.15.7:0/1264527302 wait complete. 2024-09-13T18:52:45.939 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:52:15.554683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:52:16.896879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:52:45.939 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:52:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.049+0000 7f57fb59e700 1 Processor -- start 2024-09-13T18:52:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.049+0000 7f57fb59e700 1 -- start start 2024-09-13T18:52:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fb59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57fc0cdc90 0x7f57fc06d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fb59e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f57fc06bdb0 0x7f57fc06fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fb59e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57fc144720 con 0x7f57fc144340 2024-09-13T18:52:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fb59e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57fc06dd40 con 0x7f57fc06bdb0 2024-09-13T18:52:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fb59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc06da60 con 0x7f57fc0cdc90 2024-09-13T18:52:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fa59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f57fc144340 0x7f57fc0cd580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57830/0 (socket says 172.21.15.7:57830) 2024-09-13T18:52:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fa59c700 1 -- 172.21.15.7:0/4223419368 learned_addr learned my addr 172.21.15.7:0/4223419368 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fad9d700 1 --2- 172.21.15.7:0/4223419368 >> v2:172.21.15.171:3300/0 conn(0x7f57fc06bdb0 0x7f57fc06fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57f9d9b700 1 --2- 172.21.15.7:0/4223419368 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57fc0cdc90 0x7f57fc06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57f959a700 1 -- 172.21.15.7:0/4223419368 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594576600 0 0) 0x7f57fc144720 con 0x7f57fc144340 2024-09-13T18:52:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57f959a700 1 -- 172.21.15.7:0/4223419368 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57dc0035d0 con 0x7f57fc144340 2024-09-13T18:52:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fad9d700 1 -- 172.21.15.7:0/4223419368 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57fc0cdc90 msgr2=0x7f57fc06d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fad9d700 1 --2- 172.21.15.7:0/4223419368 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57fc0cdc90 0x7f57fc06d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fad9d700 1 -- 172.21.15.7:0/4223419368 >> v1:172.21.15.7:6789/0 conn(0x7f57fc144340 legacy=0x7f57fc0cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fad9d700 1 -- 172.21.15.7:0/4223419368 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57fc15e660 con 0x7f57fc06bdb0 2024-09-13T18:52:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57f9d9b700 1 --2- 172.21.15.7:0/4223419368 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57fc0cdc90 0x7f57fc06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fad9d700 1 --2- 172.21.15.7:0/4223419368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57fc06bdb0 0x7f57fc06fa90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009d90 tx=0x7f57ec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6d98f288f27d38b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57f959a700 1 -- 172.21.15.7:0/4223419368 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec00e070 con 0x7f57fc06bdb0 2024-09-13T18:52:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57f959a700 1 -- 172.21.15.7:0/4223419368 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec00bb20 con 0x7f57fc06bdb0 2024-09-13T18:52:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57f959a700 1 -- 172.21.15.7:0/4223419368 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec012430 con 0x7f57fc06bdb0 2024-09-13T18:52:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fb59e700 1 -- 172.21.15.7:0/4223419368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57fc06bdb0 msgr2=0x7f57fc06fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fb59e700 1 --2- 172.21.15.7:0/4223419368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57fc06bdb0 0x7f57fc06fa90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009d90 tx=0x7f57ec00b330 comp rx=0 tx=0).stop 2024-09-13T18:52:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fb59e700 1 -- 172.21.15.7:0/4223419368 shutdown_connections 2024-09-13T18:52:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fb59e700 1 --2- 172.21.15.7:0/4223419368 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57fc06bdb0 0x7f57fc06fa90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fb59e700 1 --2- 172.21.15.7:0/4223419368 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57fc0cdc90 0x7f57fc06d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fb59e700 1 -- 172.21.15.7:0/4223419368 >> 172.21.15.7:0/4223419368 conn(0x7f57fc0b0e10 msgr2=0x7f57fc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fb59e700 1 -- 172.21.15.7:0/4223419368 shutdown_connections 2024-09-13T18:52:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.053+0000 7f57fb59e700 1 -- 172.21.15.7:0/4223419368 wait complete. 2024-09-13T18:52:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.057+0000 7f57fb59e700 1 Processor -- start 2024-09-13T18:52:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.057+0000 7f57fb59e700 1 -- start start 2024-09-13T18:52:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57fb59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57fc06bdb0 0x7f57fc12df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57fb59e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57fc0cdc90 0x7f57fc15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57fb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc144340 0x7f57fc12f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57fb59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc12e4b0 con 0x7f57fc144340 2024-09-13T18:52:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57fb59e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc15abe0 con 0x7f57fc0cdc90 2024-09-13T18:52:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57fb59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc12f2b0 con 0x7f57fc06bdb0 2024-09-13T18:52:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57fad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc144340 0x7f57fc12f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57fad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc144340 0x7f57fc12f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45982/0 (socket says 172.21.15.7:45982) 2024-09-13T18:52:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57fad9d700 1 -- 172.21.15.7:0/23031903 learned_addr learned my addr 172.21.15.7:0/23031903 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57fa59c700 1 --2- 172.21.15.7:0/23031903 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57fc06bdb0 0x7f57fc12df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57f9d9b700 1 --2- 172.21.15.7:0/23031903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57fc0cdc90 0x7f57fc15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57fad9d700 1 -- 172.21.15.7:0/23031903 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57fc06bdb0 msgr2=0x7f57fc12df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57fad9d700 1 --2- 172.21.15.7:0/23031903 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57fc06bdb0 0x7f57fc12df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57fad9d700 1 -- 172.21.15.7:0/23031903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57fc0cdc90 msgr2=0x7f57fc15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57fad9d700 1 --2- 172.21.15.7:0/23031903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57fc0cdc90 0x7f57fc15d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57fad9d700 1 -- 172.21.15.7:0/23031903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f40094b0 con 0x7f57fc144340 2024-09-13T18:52:46.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57fad9d700 1 --2- 172.21.15.7:0/23031903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc144340 0x7f57fc12f740 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f57ec014040 tx=0x7f57ec004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:46.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57f2ffd700 1 -- 172.21.15.7:0/23031903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec012430 con 0x7f57fc144340 2024-09-13T18:52:46.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57f2ffd700 1 -- 172.21.15.7:0/23031903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec020070 con 0x7f57fc144340 2024-09-13T18:52:46.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57fb59e700 1 -- 172.21.15.7:0/23031903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ec009a70 con 0x7f57fc144340 2024-09-13T18:52:46.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57f2ffd700 1 -- 172.21.15.7:0/23031903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0098f0 con 0x7f57fc144340 2024-09-13T18:52:46.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.061+0000 7f57fb59e700 1 -- 172.21.15.7:0/23031903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc12e750 con 0x7f57fc144340 2024-09-13T18:52:46.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.065+0000 7f57f2ffd700 1 -- 172.21.15.7:0/23031903 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f57ec004040 con 0x7f57fc144340 2024-09-13T18:52:46.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.065+0000 7f57fb59e700 1 -- 172.21.15.7:0/23031903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57fc12f3f0 con 0x7f57fc144340 2024-09-13T18:52:46.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.065+0000 7f57f2ffd700 1 --2- 172.21.15.7:0/23031903 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f57e0066760 0x7f57e0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.065+0000 7f57f2ffd700 1 -- 172.21.15.7:0/23031903 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57ec022070 con 0x7f57fc144340 2024-09-13T18:52:46.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.065+0000 7f57fa59c700 1 --2- 172.21.15.7:0/23031903 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f57e0066760 0x7f57e0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.065+0000 7f57fa59c700 1 --2- 172.21.15.7:0/23031903 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f57e0066760 0x7f57e0068c20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f57e4007fd0 tx=0x7f57e4007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:46.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.073+0000 7f57f2ffd700 1 -- 172.21.15.7:0/23031903 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f57ec087e60 con 0x7f57fc144340 2024-09-13T18:52:46.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.245+0000 7f57fb59e700 1 -- 172.21.15.7:0/23031903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57fc1437a0 con 0x7f57fc144340 2024-09-13T18:52:46.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.245+0000 7f57f2ffd700 1 -- 172.21.15.7:0/23031903 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57ec02c0a0 con 0x7f57fc144340 2024-09-13T18:52:46.253 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:52:46.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.249+0000 7f57fb59e700 1 -- 172.21.15.7:0/23031903 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f57e0066760 msgr2=0x7f57e0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.249+0000 7f57fb59e700 1 --2- 172.21.15.7:0/23031903 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f57e0066760 0x7f57e0068c20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f57e4007fd0 tx=0x7f57e4007de0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.249+0000 7f57fb59e700 1 -- 172.21.15.7:0/23031903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc144340 msgr2=0x7f57fc12f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.249+0000 7f57fb59e700 1 --2- 172.21.15.7:0/23031903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc144340 0x7f57fc12f740 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f57ec014040 tx=0x7f57ec004af0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.249+0000 7f57fb59e700 1 -- 172.21.15.7:0/23031903 shutdown_connections 2024-09-13T18:52:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.249+0000 7f57fb59e700 1 --2- 172.21.15.7:0/23031903 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f57e0066760 0x7f57e0068c20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.249+0000 7f57fb59e700 1 --2- 172.21.15.7:0/23031903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57fc144340 0x7f57fc12f740 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.249+0000 7f57fb59e700 1 --2- 172.21.15.7:0/23031903 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57fc0cdc90 0x7f57fc15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.249+0000 7f57fb59e700 1 --2- 172.21.15.7:0/23031903 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f57fc06bdb0 0x7f57fc12df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.249+0000 7f57fb59e700 1 -- 172.21.15.7:0/23031903 >> 172.21.15.7:0/23031903 conn(0x7f57fc0b0e10 msgr2=0x7f57fc0cd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.249+0000 7f57fb59e700 1 -- 172.21.15.7:0/23031903 shutdown_connections 2024-09-13T18:52:46.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.249+0000 7f57fb59e700 1 -- 172.21.15.7:0/23031903 wait complete. 2024-09-13T18:52:46.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:52:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129bfff700 1 Processor -- start 2024-09-13T18:52:46.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129bfff700 1 -- start start 2024-09-13T18:52:46.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f129c146160 0x7f129c0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129bfff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f129c0ce130 0x7f129c0ad270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129bfff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129c0ae000 con 0x7f129c0ad7b0 2024-09-13T18:52:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129bfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f129c0cc820 con 0x7f129c146160 2024-09-13T18:52:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129bfff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f129c070070 con 0x7f129c0ce130 2024-09-13T18:52:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f129c146160 0x7f129c0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f129c146160 0x7f129c0cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45990/0 (socket says 172.21.15.7:45990) 2024-09-13T18:52:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129affd700 1 -- 172.21.15.7:0/2109373276 learned_addr learned my addr 172.21.15.7:0/2109373276 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129a7fc700 1 --2- 172.21.15.7:0/2109373276 >> v2:172.21.15.171:3300/0 conn(0x7f129c0ce130 0x7f129c0ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129affd700 1 -- 172.21.15.7:0/2109373276 >> v2:172.21.15.171:3300/0 conn(0x7f129c0ce130 msgr2=0x7f129c0ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129affd700 1 --2- 172.21.15.7:0/2109373276 >> v2:172.21.15.171:3300/0 conn(0x7f129c0ce130 0x7f129c0ad270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129affd700 1 -- 172.21.15.7:0/2109373276 >> v1:172.21.15.171:6789/0 conn(0x7f129c0ad7b0 legacy=0x7f129c160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129affd700 1 -- 172.21.15.7:0/2109373276 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c070350 con 0x7f129c146160 2024-09-13T18:52:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129affd700 1 --2- 172.21.15.7:0/2109373276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f129c146160 0x7f129c0cdbf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f129c0cd730 tx=0x7f128c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6baef62ba8c485e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f1299ffb700 1 -- 172.21.15.7:0/2109373276 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c00b8a0 con 0x7f129c146160 2024-09-13T18:52:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f1299ffb700 1 -- 172.21.15.7:0/2109373276 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128c00ba00 con 0x7f129c146160 2024-09-13T18:52:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f1299ffb700 1 -- 172.21.15.7:0/2109373276 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c00bd90 con 0x7f129c146160 2024-09-13T18:52:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129bfff700 1 -- 172.21.15.7:0/2109373276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f129c146160 msgr2=0x7f129c0cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129bfff700 1 --2- 172.21.15.7:0/2109373276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f129c146160 0x7f129c0cdbf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f129c0cd730 tx=0x7f128c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129bfff700 1 -- 172.21.15.7:0/2109373276 shutdown_connections 2024-09-13T18:52:46.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129bfff700 1 --2- 172.21.15.7:0/2109373276 >> v2:172.21.15.171:3300/0 conn(0x7f129c0ce130 0x7f129c0ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129bfff700 1 --2- 172.21.15.7:0/2109373276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f129c146160 0x7f129c0cdbf0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129bfff700 1 -- 172.21.15.7:0/2109373276 >> 172.21.15.7:0/2109373276 conn(0x7f129c0b0e10 msgr2=0x7f129c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129bfff700 1 -- 172.21.15.7:0/2109373276 shutdown_connections 2024-09-13T18:52:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.381+0000 7f129bfff700 1 -- 172.21.15.7:0/2109373276 wait complete. 2024-09-13T18:52:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129bfff700 1 Processor -- start 2024-09-13T18:52:46.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129bfff700 1 -- start start 2024-09-13T18:52:46.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f129c0ad7b0 0x7f129c15c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129bfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f129c0ce130 0x7f129c13c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f129c146160 0x7f129c13ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129bfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c1f6f10 con 0x7f129c0ad7b0 2024-09-13T18:52:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129bfff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c13ed80 con 0x7f129c146160 2024-09-13T18:52:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129bfff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129c140fa0 con 0x7f129c0ce130 2024-09-13T18:52:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f129c0ad7b0 0x7f129c15c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f129c0ad7b0 0x7f129c15c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45992/0 (socket says 172.21.15.7:45992) 2024-09-13T18:52:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129affd700 1 -- 172.21.15.7:0/3431474382 learned_addr learned my addr 172.21.15.7:0/3431474382 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129b7fe700 1 --2- 172.21.15.7:0/3431474382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f129c146160 0x7f129c13ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129a7fc700 1 --2- 172.21.15.7:0/3431474382 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f129c0ce130 0x7f129c13c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129affd700 1 -- 172.21.15.7:0/3431474382 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f129c0ce130 msgr2=0x7f129c13c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129affd700 1 --2- 172.21.15.7:0/3431474382 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f129c0ce130 0x7f129c13c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129affd700 1 -- 172.21.15.7:0/3431474382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f129c146160 msgr2=0x7f129c13ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129affd700 1 --2- 172.21.15.7:0/3431474382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f129c146160 0x7f129c13ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129affd700 1 -- 172.21.15.7:0/3431474382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c009570 con 0x7f129c0ad7b0 2024-09-13T18:52:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129a7fc700 1 --2- 172.21.15.7:0/3431474382 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f129c0ce130 0x7f129c13c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.385+0000 7f129affd700 1 --2- 172.21.15.7:0/3431474382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f129c0ad7b0 0x7f129c15c310 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f129c0b3d60 tx=0x7f128c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.389+0000 7f12837fe700 1 -- 172.21.15.7:0/3431474382 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c0038c0 con 0x7f129c0ad7b0 2024-09-13T18:52:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.389+0000 7f12837fe700 1 -- 172.21.15.7:0/3431474382 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128c01e070 con 0x7f129c0ad7b0 2024-09-13T18:52:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.389+0000 7f129bfff700 1 -- 172.21.15.7:0/3431474382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129c1f6390 con 0x7f129c0ad7b0 2024-09-13T18:52:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.389+0000 7f12837fe700 1 -- 172.21.15.7:0/3431474382 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c003c80 con 0x7f129c0ad7b0 2024-09-13T18:52:46.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.389+0000 7f129bfff700 1 -- 172.21.15.7:0/3431474382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129c1f6950 con 0x7f129c0ad7b0 2024-09-13T18:52:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.389+0000 7f129bfff700 1 -- 172.21.15.7:0/3431474382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f129c141180 con 0x7f129c0ad7b0 2024-09-13T18:52:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.389+0000 7f12837fe700 1 -- 172.21.15.7:0/3431474382 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f128c011420 con 0x7f129c0ad7b0 2024-09-13T18:52:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.389+0000 7f12837fe700 1 --2- 172.21.15.7:0/3431474382 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1288066730 0x7f1288068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.389+0000 7f12837fe700 1 -- 172.21.15.7:0/3431474382 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f128c020070 con 0x7f129c0ad7b0 2024-09-13T18:52:46.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.389+0000 7f129a7fc700 1 --2- 172.21.15.7:0/3431474382 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1288066730 0x7f1288068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.393+0000 7f129a7fc700 1 --2- 172.21.15.7:0/3431474382 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1288066730 0x7f1288068bf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f129c06d390 tx=0x7f1290008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:46.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.397+0000 7f12837fe700 1 -- 172.21.15.7:0/3431474382 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f128c052c40 con 0x7f129c0ad7b0 2024-09-13T18:52:46.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.573+0000 7f129bfff700 1 -- 172.21.15.7:0/3431474382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f129c0ad270 con 0x7f129c0ad7b0 2024-09-13T18:52:46.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.573+0000 7f12837fe700 1 -- 172.21.15.7:0/3431474382 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f128c056260 con 0x7f129c0ad7b0 2024-09-13T18:52:46.581 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:52:46.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.577+0000 7f129bfff700 1 -- 172.21.15.7:0/3431474382 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1288066730 msgr2=0x7f1288068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.577+0000 7f129bfff700 1 --2- 172.21.15.7:0/3431474382 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1288066730 0x7f1288068bf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f129c06d390 tx=0x7f1290008040 comp rx=0 tx=0).stop 2024-09-13T18:52:46.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.577+0000 7f129bfff700 1 -- 172.21.15.7:0/3431474382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f129c0ad7b0 msgr2=0x7f129c15c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.577+0000 7f129bfff700 1 --2- 172.21.15.7:0/3431474382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f129c0ad7b0 0x7f129c15c310 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f129c0b3d60 tx=0x7f128c004770 comp rx=0 tx=0).stop 2024-09-13T18:52:46.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.577+0000 7f129bfff700 1 -- 172.21.15.7:0/3431474382 shutdown_connections 2024-09-13T18:52:46.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.577+0000 7f129bfff700 1 --2- 172.21.15.7:0/3431474382 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1288066730 0x7f1288068bf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.577+0000 7f129bfff700 1 --2- 172.21.15.7:0/3431474382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f129c146160 0x7f129c13ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.577+0000 7f129bfff700 1 --2- 172.21.15.7:0/3431474382 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f129c0ce130 0x7f129c13c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.577+0000 7f129bfff700 1 --2- 172.21.15.7:0/3431474382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f129c0ad7b0 0x7f129c15c310 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.577+0000 7f129bfff700 1 -- 172.21.15.7:0/3431474382 >> 172.21.15.7:0/3431474382 conn(0x7f129c0b0e10 msgr2=0x7f129c144600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:46.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.577+0000 7f129bfff700 1 -- 172.21.15.7:0/3431474382 shutdown_connections 2024-09-13T18:52:46.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.581+0000 7f129bfff700 1 -- 172.21.15.7:0/3431474382 wait complete. 2024-09-13T18:52:46.602 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:52:46.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.717+0000 7f09e9865700 1 Processor -- start 2024-09-13T18:52:46.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.717+0000 7f09e9865700 1 -- start start 2024-09-13T18:52:46.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e9865700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f09e4157270 0x7f09e40b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e9865700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09e40cd800 0x7f09e40b5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e9865700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09e4071dc0 0x7f09e4072270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e9865700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e41614e0 con 0x7f09e4071dc0 2024-09-13T18:52:46.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e9865700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e41608f0 con 0x7f09e4157270 2024-09-13T18:52:46.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e9865700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e4160bb0 con 0x7f09e40cd800 2024-09-13T18:52:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09e4071dc0 0x7f09e4072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09e4071dc0 0x7f09e4072270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46008/0 (socket says 172.21.15.7:46008) 2024-09-13T18:52:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e37fe700 1 -- 172.21.15.7:0/1094795819 learned_addr learned my addr 172.21.15.7:0/1094795819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e27fc700 1 --2- 172.21.15.7:0/1094795819 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09e40cd800 0x7f09e40b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e2ffd700 1 --2- 172.21.15.7:0/1094795819 >> v2:172.21.15.171:3300/0 conn(0x7f09e4157270 0x7f09e40b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e37fe700 1 -- 172.21.15.7:0/1094795819 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09e40cd800 msgr2=0x7f09e40b5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e37fe700 1 --2- 172.21.15.7:0/1094795819 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09e40cd800 0x7f09e40b5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e37fe700 1 -- 172.21.15.7:0/1094795819 >> v2:172.21.15.171:3300/0 conn(0x7f09e4157270 msgr2=0x7f09e40b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e37fe700 1 --2- 172.21.15.7:0/1094795819 >> v2:172.21.15.171:3300/0 conn(0x7f09e4157270 0x7f09e40b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e37fe700 1 -- 172.21.15.7:0/1094795819 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e4160e90 con 0x7f09e4071dc0 2024-09-13T18:52:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e37fe700 1 --2- 172.21.15.7:0/1094795819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e4071dc0 0x7f09e4072270 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f09e4066fc0 tx=0x7f09dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1a98c54fdea0b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:46.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e1ffb700 1 -- 172.21.15.7:0/1094795819 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc00e070 con 0x7f09e4071dc0 2024-09-13T18:52:46.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e1ffb700 1 -- 172.21.15.7:0/1094795819 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09dc00b910 con 0x7f09e4071dc0 2024-09-13T18:52:46.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e1ffb700 1 -- 172.21.15.7:0/1094795819 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc00bcc0 con 0x7f09e4071dc0 2024-09-13T18:52:46.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e9865700 1 -- 172.21.15.7:0/1094795819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e4071dc0 msgr2=0x7f09e4072270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e9865700 1 --2- 172.21.15.7:0/1094795819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e4071dc0 0x7f09e4072270 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f09e4066fc0 tx=0x7f09dc00b330 comp rx=0 tx=0).stop 2024-09-13T18:52:46.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e9865700 1 -- 172.21.15.7:0/1094795819 shutdown_connections 2024-09-13T18:52:46.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e9865700 1 --2- 172.21.15.7:0/1094795819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e4071dc0 0x7f09e4072270 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e9865700 1 --2- 172.21.15.7:0/1094795819 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09e40cd800 0x7f09e40b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e9865700 1 --2- 172.21.15.7:0/1094795819 >> v2:172.21.15.171:3300/0 conn(0x7f09e4157270 0x7f09e40b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e9865700 1 -- 172.21.15.7:0/1094795819 >> 172.21.15.7:0/1094795819 conn(0x7f09e40cc4a0 msgr2=0x7f09e40cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:46.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e9865700 1 -- 172.21.15.7:0/1094795819 shutdown_connections 2024-09-13T18:52:46.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.721+0000 7f09e9865700 1 -- 172.21.15.7:0/1094795819 wait complete. 2024-09-13T18:52:46.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e9865700 1 Processor -- start 2024-09-13T18:52:46.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e9865700 1 -- start start 2024-09-13T18:52:46.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e9865700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09e4071dc0 0x7f09e414e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e9865700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e40cd800 0x7f09e406c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e9865700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09e4157270 0x7f09e4153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e9865700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e41f7380 con 0x7f09e40cd800 2024-09-13T18:52:46.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e9865700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e414ebb0 con 0x7f09e4071dc0 2024-09-13T18:52:46.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e9865700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e41f64b0 con 0x7f09e4157270 2024-09-13T18:52:46.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e40cd800 0x7f09e406c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e40cd800 0x7f09e406c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46024/0 (socket says 172.21.15.7:46024) 2024-09-13T18:52:46.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e27fc700 1 -- 172.21.15.7:0/1568524400 learned_addr learned my addr 172.21.15.7:0/1568524400 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:46.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e37fe700 1 --2- 172.21.15.7:0/1568524400 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09e4157270 0x7f09e4153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e2ffd700 1 --2- 172.21.15.7:0/1568524400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09e4071dc0 0x7f09e414e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e27fc700 1 -- 172.21.15.7:0/1568524400 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09e4157270 msgr2=0x7f09e4153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e27fc700 1 --2- 172.21.15.7:0/1568524400 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09e4157270 0x7f09e4153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e27fc700 1 -- 172.21.15.7:0/1568524400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09e4071dc0 msgr2=0x7f09e414e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e27fc700 1 --2- 172.21.15.7:0/1568524400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09e4071dc0 0x7f09e414e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e27fc700 1 -- 172.21.15.7:0/1568524400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09dc009b30 con 0x7f09e40cd800 2024-09-13T18:52:46.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e27fc700 1 --2- 172.21.15.7:0/1568524400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e40cd800 0x7f09e406c760 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f09d400e940 tx=0x7f09d400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:46.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09cbfff700 1 -- 172.21.15.7:0/1568524400 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d4013680 con 0x7f09e40cd800 2024-09-13T18:52:46.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09cbfff700 1 -- 172.21.15.7:0/1568524400 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d40137e0 con 0x7f09e40cd800 2024-09-13T18:52:46.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e9865700 1 -- 172.21.15.7:0/1568524400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e414e620 con 0x7f09e40cd800 2024-09-13T18:52:46.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09cbfff700 1 -- 172.21.15.7:0/1568524400 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d4013b90 con 0x7f09e40cd800 2024-09-13T18:52:46.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.725+0000 7f09e9865700 1 -- 172.21.15.7:0/1568524400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e41f69e0 con 0x7f09e40cd800 2024-09-13T18:52:46.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.729+0000 7f09e9865700 1 -- 172.21.15.7:0/1568524400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e4072270 con 0x7f09e40cd800 2024-09-13T18:52:46.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.729+0000 7f09cbfff700 1 -- 172.21.15.7:0/1568524400 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f09d4013cf0 con 0x7f09e40cd800 2024-09-13T18:52:46.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.729+0000 7f09cbfff700 1 --2- 172.21.15.7:0/1568524400 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f09d0066730 0x7f09d0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:46.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.729+0000 7f09cbfff700 1 -- 172.21.15.7:0/1568524400 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09d4083de0 con 0x7f09e40cd800 2024-09-13T18:52:46.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.729+0000 7f09e2ffd700 1 --2- 172.21.15.7:0/1568524400 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f09d0066730 0x7f09d0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:46.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.729+0000 7f09e2ffd700 1 --2- 172.21.15.7:0/1568524400 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f09d0066730 0x7f09d0068bf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009b50 tx=0x7f09d8009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:46.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.737+0000 7f09cbfff700 1 -- 172.21.15.7:0/1568524400 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f09d40510d0 con 0x7f09e40cd800 2024-09-13T18:52:46.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.905+0000 7f09e9865700 1 -- 172.21.15.7:0/1568524400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09e4160440 con 0x7f09e40cd800 2024-09-13T18:52:46.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.905+0000 7f09cbfff700 1 -- 172.21.15.7:0/1568524400 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f09d40546f0 con 0x7f09e40cd800 2024-09-13T18:52:46.912 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:52:46.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.909+0000 7f09e9865700 1 -- 172.21.15.7:0/1568524400 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f09d0066730 msgr2=0x7f09d0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.909+0000 7f09e9865700 1 --2- 172.21.15.7:0/1568524400 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f09d0066730 0x7f09d0068bf0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009b50 tx=0x7f09d8009240 comp rx=0 tx=0).stop 2024-09-13T18:52:46.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.909+0000 7f09e9865700 1 -- 172.21.15.7:0/1568524400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e40cd800 msgr2=0x7f09e406c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:46.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.909+0000 7f09e9865700 1 --2- 172.21.15.7:0/1568524400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e40cd800 0x7f09e406c760 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f09d400e940 tx=0x7f09d400ec50 comp rx=0 tx=0).stop 2024-09-13T18:52:46.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.909+0000 7f09e9865700 1 -- 172.21.15.7:0/1568524400 shutdown_connections 2024-09-13T18:52:46.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.909+0000 7f09e9865700 1 --2- 172.21.15.7:0/1568524400 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f09d0066730 0x7f09d0068bf0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.909+0000 7f09e9865700 1 --2- 172.21.15.7:0/1568524400 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f09e4157270 0x7f09e4153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.909+0000 7f09e9865700 1 --2- 172.21.15.7:0/1568524400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09e40cd800 0x7f09e406c760 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.909+0000 7f09e9865700 1 --2- 172.21.15.7:0/1568524400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f09e4071dc0 0x7f09e414e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:46.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.909+0000 7f09e9865700 1 -- 172.21.15.7:0/1568524400 >> 172.21.15.7:0/1568524400 conn(0x7f09e40cc4a0 msgr2=0x7f09e40cacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:46.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.909+0000 7f09e9865700 1 -- 172.21.15.7:0/1568524400 shutdown_connections 2024-09-13T18:52:46.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:46.909+0000 7f09e9865700 1 -- 172.21.15.7:0/1568524400 wait complete. 2024-09-13T18:52:46.948 INFO:tasks.cephfs_test_runner:Starting test: test_max_items_per_obj (tasks.cephfs.test_openfiletable.OpenFileTable) The maximum number of openfiles omap objects keys are now equal to 2024-09-13T18:52:46.948 INFO:tasks.cephfs_test_runner:test_max_items_per_obj (tasks.cephfs.test_openfiletable.OpenFileTable) 2024-09-13T18:52:46.949 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:47.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe72109c700 1 Processor -- start 2024-09-13T18:52:47.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe72109c700 1 -- start start 2024-09-13T18:52:47.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe72109c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe71c15c8c0 0x7fe71c0d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:47.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe72109c700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fe71c0cddb0 0x7fe71c0d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:47.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe72109c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71c0b5a60 con 0x7fe71c15cca0 2024-09-13T18:52:47.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe72109c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe71c0b6680 con 0x7fe71c15c8c0 2024-09-13T18:52:47.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe72109c700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe71c0ce430 con 0x7fe71c0cddb0 2024-09-13T18:52:47.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe71bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe71c15c8c0 0x7fe71c0d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:47.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe71bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe71c15c8c0 0x7fe71c0d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46038/0 (socket says 172.21.15.7:46038) 2024-09-13T18:52:47.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe71bfff700 1 -- 172.21.15.7:0/4272195161 learned_addr learned my addr 172.21.15.7:0/4272195161 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:47.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe72089b700 1 --2- 172.21.15.7:0/4272195161 >> v2:172.21.15.171:3300/0 conn(0x7fe71c0cddb0 0x7fe71c0d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:47.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe71affd700 1 -- 172.21.15.7:0/4272195161 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2451848030 0 0) 0x7fe71c0b5a60 con 0x7fe71c15cca0 2024-09-13T18:52:47.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe71affd700 1 -- 172.21.15.7:0/4272195161 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7080035d0 con 0x7fe71c15cca0 2024-09-13T18:52:47.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe71bfff700 1 -- 172.21.15.7:0/4272195161 >> v2:172.21.15.171:3300/0 conn(0x7fe71c0cddb0 msgr2=0x7fe71c0d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:47.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe71bfff700 1 --2- 172.21.15.7:0/4272195161 >> v2:172.21.15.171:3300/0 conn(0x7fe71c0cddb0 0x7fe71c0d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe71bfff700 1 -- 172.21.15.7:0/4272195161 >> v1:172.21.15.7:6789/0 conn(0x7fe71c15cca0 legacy=0x7fe71c0cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:47.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe71bfff700 1 -- 172.21.15.7:0/4272195161 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c0cc160 con 0x7fe71c15c8c0 2024-09-13T18:52:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe71bfff700 1 --2- 172.21.15.7:0/4272195161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c15c8c0 0x7fe71c0d0a90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0cc980 tx=0x7fe70c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951c9f52bfcfeb7d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe71affd700 1 -- 172.21.15.7:0/4272195161 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c00e070 con 0x7fe71c15c8c0 2024-09-13T18:52:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe71affd700 1 -- 172.21.15.7:0/4272195161 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70c00b910 con 0x7fe71c15c8c0 2024-09-13T18:52:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.061+0000 7fe71affd700 1 -- 172.21.15.7:0/4272195161 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c00bc80 con 0x7fe71c15c8c0 2024-09-13T18:52:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.065+0000 7fe72109c700 1 -- 172.21.15.7:0/4272195161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c15c8c0 msgr2=0x7fe71c0d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.065+0000 7fe72109c700 1 --2- 172.21.15.7:0/4272195161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c15c8c0 0x7fe71c0d0a90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0cc980 tx=0x7fe70c00b330 comp rx=0 tx=0).stop 2024-09-13T18:52:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.065+0000 7fe72109c700 1 -- 172.21.15.7:0/4272195161 shutdown_connections 2024-09-13T18:52:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.065+0000 7fe72109c700 1 --2- 172.21.15.7:0/4272195161 >> v2:172.21.15.171:3300/0 conn(0x7fe71c0cddb0 0x7fe71c0d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.065+0000 7fe72109c700 1 --2- 172.21.15.7:0/4272195161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c15c8c0 0x7fe71c0d0a90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.065+0000 7fe72109c700 1 -- 172.21.15.7:0/4272195161 >> 172.21.15.7:0/4272195161 conn(0x7fe71c071ee0 msgr2=0x7fe71c0722c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.065+0000 7fe72109c700 1 -- 172.21.15.7:0/4272195161 shutdown_connections 2024-09-13T18:52:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.065+0000 7fe72109c700 1 -- 172.21.15.7:0/4272195161 wait complete. 2024-09-13T18:52:47.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.065+0000 7fe72109c700 1 Processor -- start 2024-09-13T18:52:47.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.065+0000 7fe72109c700 1 -- start start 2024-09-13T18:52:47.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe72109c700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe71c0cddb0 0x7fe71c1454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:47.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe72109c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c15c8c0 0x7fe71c06bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:47.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe72109c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe71c15cca0 0x7fe71c158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:47.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe72109c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c146100 con 0x7fe71c15c8c0 2024-09-13T18:52:47.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe72109c700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c1463e0 con 0x7fe71c15cca0 2024-09-13T18:52:47.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe72109c700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c1466c0 con 0x7fe71c0cddb0 2024-09-13T18:52:47.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe71b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c15c8c0 0x7fe71c06bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:47.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe71b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c15c8c0 0x7fe71c06bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46044/0 (socket says 172.21.15.7:46044) 2024-09-13T18:52:47.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe71b7fe700 1 -- 172.21.15.7:0/2330117617 learned_addr learned my addr 172.21.15.7:0/2330117617 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:47.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe71bfff700 1 --2- 172.21.15.7:0/2330117617 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe71c0cddb0 0x7fe71c1454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:47.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe72089b700 1 --2- 172.21.15.7:0/2330117617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe71c15cca0 0x7fe71c158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:47.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe71b7fe700 1 -- 172.21.15.7:0/2330117617 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe71c0cddb0 msgr2=0x7fe71c1454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:47.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe71b7fe700 1 --2- 172.21.15.7:0/2330117617 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe71c0cddb0 0x7fe71c1454e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe71b7fe700 1 -- 172.21.15.7:0/2330117617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe71c15cca0 msgr2=0x7fe71c158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:47.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe71b7fe700 1 --2- 172.21.15.7:0/2330117617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe71c15cca0 0x7fe71c158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe71b7fe700 1 -- 172.21.15.7:0/2330117617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c009b30 con 0x7fe71c15c8c0 2024-09-13T18:52:47.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe71b7fe700 1 --2- 172.21.15.7:0/2330117617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c15c8c0 0x7fe71c06bf10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe70400c500 tx=0x7fe70400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:47.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe718ff9700 1 -- 172.21.15.7:0/2330117617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040052d0 con 0x7fe71c15c8c0 2024-09-13T18:52:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe718ff9700 1 -- 172.21.15.7:0/2330117617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe704005450 con 0x7fe71c15c8c0 2024-09-13T18:52:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe72109c700 1 -- 172.21.15.7:0/2330117617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71c1469a0 con 0x7fe71c15c8c0 2024-09-13T18:52:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe718ff9700 1 -- 172.21.15.7:0/2330117617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040038e0 con 0x7fe71c15c8c0 2024-09-13T18:52:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.069+0000 7fe72109c700 1 -- 172.21.15.7:0/2330117617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71c145a20 con 0x7fe71c15c8c0 2024-09-13T18:52:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.073+0000 7fe72109c700 1 -- 172.21.15.7:0/2330117617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71c0b6a80 con 0x7fe71c15c8c0 2024-09-13T18:52:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.073+0000 7fe718ff9700 1 -- 172.21.15.7:0/2330117617 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe704003a40 con 0x7fe71c15c8c0 2024-09-13T18:52:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.073+0000 7fe718ff9700 1 --2- 172.21.15.7:0/2330117617 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe6fc066730 0x7fe6fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.073+0000 7fe718ff9700 1 -- 172.21.15.7:0/2330117617 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe704083460 con 0x7fe71c15c8c0 2024-09-13T18:52:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.073+0000 7fe71bfff700 1 --2- 172.21.15.7:0/2330117617 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe6fc066730 0x7fe6fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.073+0000 7fe71bfff700 1 --2- 172.21.15.7:0/2330117617 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe6fc066730 0x7fe6fc068bf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe70c000f80 tx=0x7fe70c009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:47.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.077+0000 7fe718ff9700 1 -- 172.21.15.7:0/2330117617 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe71c0b6a80 con 0x7fe71c15c8c0 2024-09-13T18:52:47.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.253+0000 7fe72109c700 1 -- 172.21.15.7:0/2330117617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe71c146ec0 con 0x7fe71c15c8c0 2024-09-13T18:52:47.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.257+0000 7fe718ff9700 1 -- 172.21.15.7:0/2330117617 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7fe71c146ec0 con 0x7fe71c15c8c0 2024-09-13T18:52:47.264 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:47.264 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.261+0000 7fe72109c700 1 -- 172.21.15.7:0/2330117617 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe6fc066730 msgr2=0x7fe6fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.261+0000 7fe72109c700 1 --2- 172.21.15.7:0/2330117617 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe6fc066730 0x7fe6fc068bf0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe70c000f80 tx=0x7fe70c009000 comp rx=0 tx=0).stop 2024-09-13T18:52:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.261+0000 7fe72109c700 1 -- 172.21.15.7:0/2330117617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c15c8c0 msgr2=0x7fe71c06bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.261+0000 7fe72109c700 1 --2- 172.21.15.7:0/2330117617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c15c8c0 0x7fe71c06bf10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe70400c500 tx=0x7fe70400c810 comp rx=0 tx=0).stop 2024-09-13T18:52:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.261+0000 7fe72109c700 1 -- 172.21.15.7:0/2330117617 shutdown_connections 2024-09-13T18:52:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.261+0000 7fe72109c700 1 --2- 172.21.15.7:0/2330117617 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe6fc066730 0x7fe6fc068bf0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.261+0000 7fe72109c700 1 --2- 172.21.15.7:0/2330117617 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe71c15cca0 0x7fe71c158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.261+0000 7fe72109c700 1 --2- 172.21.15.7:0/2330117617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe71c15c8c0 0x7fe71c06bf10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.261+0000 7fe72109c700 1 --2- 172.21.15.7:0/2330117617 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe71c0cddb0 0x7fe71c1454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.261+0000 7fe72109c700 1 -- 172.21.15.7:0/2330117617 >> 172.21.15.7:0/2330117617 conn(0x7fe71c071ee0 msgr2=0x7fe71c0cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:47.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.261+0000 7fe72109c700 1 -- 172.21.15.7:0/2330117617 shutdown_connections 2024-09-13T18:52:47.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.261+0000 7fe72109c700 1 -- 172.21.15.7:0/2330117617 wait complete. 2024-09-13T18:52:47.281 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:52:15.554683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:52:16.896879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:52:47.282 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:52:47.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b8b55700 1 Processor -- start 2024-09-13T18:52:47.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b8b55700 1 -- start start 2024-09-13T18:52:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b8b55700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f21b415ce40 0x7f21b406aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b8b55700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f21b406b400 0x7f21b40cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b8b55700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21b40ce830 con 0x7f21b40ce000 2024-09-13T18:52:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b8b55700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b406b870 con 0x7f21b415ce40 2024-09-13T18:52:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b8b55700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b40ceb10 con 0x7f21b406b400 2024-09-13T18:52:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b2d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f21b40ce000 0x7f21b40d0920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57860/0 (socket says 172.21.15.7:57860) 2024-09-13T18:52:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b2d9d700 1 -- 172.21.15.7:0/1704536512 learned_addr learned my addr 172.21.15.7:0/1704536512 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b259c700 1 --2- 172.21.15.7:0/1704536512 >> v2:172.21.15.7:3300/0 conn(0x7f21b415ce40 0x7f21b406aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b1d9b700 1 --2- 172.21.15.7:0/1704536512 >> v2:172.21.15.171:3300/0 conn(0x7f21b406b400 0x7f21b40cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b159a700 1 -- 172.21.15.7:0/1704536512 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455204871 0 0) 0x7f21b40ce830 con 0x7f21b40ce000 2024-09-13T18:52:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b159a700 1 -- 172.21.15.7:0/1704536512 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f219c0035d0 con 0x7f21b40ce000 2024-09-13T18:52:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b259c700 1 -- 172.21.15.7:0/1704536512 >> v2:172.21.15.171:3300/0 conn(0x7f21b406b400 msgr2=0x7f21b40cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b259c700 1 --2- 172.21.15.7:0/1704536512 >> v2:172.21.15.171:3300/0 conn(0x7f21b406b400 0x7f21b40cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b259c700 1 -- 172.21.15.7:0/1704536512 >> v1:172.21.15.7:6789/0 conn(0x7f21b40ce000 legacy=0x7f21b40d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:47.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b259c700 1 -- 172.21.15.7:0/1704536512 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b40cedf0 con 0x7f21b415ce40 2024-09-13T18:52:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b259c700 1 --2- 172.21.15.7:0/1704536512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21b415ce40 0x7f21b406aec0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f21b40cd330 tx=0x7f21a4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b49ca53a611615 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b159a700 1 -- 172.21.15.7:0/1704536512 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4011830 con 0x7f21b415ce40 2024-09-13T18:52:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b159a700 1 -- 172.21.15.7:0/1704536512 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a4011990 con 0x7f21b415ce40 2024-09-13T18:52:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.401+0000 7f21b159a700 1 -- 172.21.15.7:0/1704536512 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4011d20 con 0x7f21b415ce40 2024-09-13T18:52:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.405+0000 7f21b8b55700 1 -- 172.21.15.7:0/1704536512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21b415ce40 msgr2=0x7f21b406aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.405+0000 7f21b8b55700 1 --2- 172.21.15.7:0/1704536512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21b415ce40 0x7f21b406aec0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f21b40cd330 tx=0x7f21a4009b30 comp rx=0 tx=0).stop 2024-09-13T18:52:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.405+0000 7f21b8b55700 1 -- 172.21.15.7:0/1704536512 shutdown_connections 2024-09-13T18:52:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.405+0000 7f21b8b55700 1 --2- 172.21.15.7:0/1704536512 >> v2:172.21.15.171:3300/0 conn(0x7f21b406b400 0x7f21b40cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.405+0000 7f21b8b55700 1 --2- 172.21.15.7:0/1704536512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21b415ce40 0x7f21b406aec0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.405+0000 7f21b8b55700 1 -- 172.21.15.7:0/1704536512 >> 172.21.15.7:0/1704536512 conn(0x7f21b40d2f30 msgr2=0x7f21b40d3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.405+0000 7f21b8b55700 1 -- 172.21.15.7:0/1704536512 shutdown_connections 2024-09-13T18:52:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.405+0000 7f21b8b55700 1 -- 172.21.15.7:0/1704536512 wait complete. 2024-09-13T18:52:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.405+0000 7f21b8b55700 1 Processor -- start 2024-09-13T18:52:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.405+0000 7f21b8b55700 1 -- start start 2024-09-13T18:52:47.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b8b55700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21b406b400 0x7f21b4156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b8b55700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21b40ce000 0x7f21b4158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b8b55700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f21b415ce40 0x7f21b4158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b8b55700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b40cb2f0 con 0x7f21b40ce000 2024-09-13T18:52:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b8b55700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b4159d40 con 0x7f21b406b400 2024-09-13T18:52:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b8b55700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41594d0 con 0x7f21b415ce40 2024-09-13T18:52:47.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21b40ce000 0x7f21b4158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:47.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21b40ce000 0x7f21b4158a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46052/0 (socket says 172.21.15.7:46052) 2024-09-13T18:52:47.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b1d9b700 1 -- 172.21.15.7:0/3837349052 learned_addr learned my addr 172.21.15.7:0/3837349052 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:47.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b2d9d700 1 --2- 172.21.15.7:0/3837349052 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f21b415ce40 0x7f21b4158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:47.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b259c700 1 --2- 172.21.15.7:0/3837349052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21b406b400 0x7f21b4156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:47.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b1d9b700 1 -- 172.21.15.7:0/3837349052 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f21b415ce40 msgr2=0x7f21b4158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:47.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b1d9b700 1 --2- 172.21.15.7:0/3837349052 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f21b415ce40 0x7f21b4158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b1d9b700 1 -- 172.21.15.7:0/3837349052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21b406b400 msgr2=0x7f21b4156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:47.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b1d9b700 1 --2- 172.21.15.7:0/3837349052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21b406b400 0x7f21b4156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b1d9b700 1 -- 172.21.15.7:0/3837349052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a40094b0 con 0x7f21b40ce000 2024-09-13T18:52:47.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b1d9b700 1 --2- 172.21.15.7:0/3837349052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21b40ce000 0x7f21b4158a50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f21a800e9d0 tx=0x7f21a800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:47.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21a2ffd700 1 -- 172.21.15.7:0/3837349052 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a8009720 con 0x7f21b40ce000 2024-09-13T18:52:47.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21a2ffd700 1 -- 172.21.15.7:0/3837349052 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a800ee70 con 0x7f21b40ce000 2024-09-13T18:52:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b8b55700 1 -- 172.21.15.7:0/3837349052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b4159780 con 0x7f21b40ce000 2024-09-13T18:52:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21a2ffd700 1 -- 172.21.15.7:0/3837349052 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a800ca50 con 0x7f21b40ce000 2024-09-13T18:52:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.409+0000 7f21b8b55700 1 -- 172.21.15.7:0/3837349052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b40caf80 con 0x7f21b40ce000 2024-09-13T18:52:47.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.413+0000 7f21a2ffd700 1 -- 172.21.15.7:0/3837349052 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f21a800cbb0 con 0x7f21b40ce000 2024-09-13T18:52:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.413+0000 7f21b8b55700 1 -- 172.21.15.7:0/3837349052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b40cb460 con 0x7f21b40ce000 2024-09-13T18:52:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.413+0000 7f21a2ffd700 1 --2- 172.21.15.7:0/3837349052 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f21940667b0 0x7f2194068c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.413+0000 7f21a2ffd700 1 -- 172.21.15.7:0/3837349052 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21a8084030 con 0x7f21b40ce000 2024-09-13T18:52:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.413+0000 7f21b259c700 1 --2- 172.21.15.7:0/3837349052 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f21940667b0 0x7f2194068c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.413+0000 7f21b259c700 1 --2- 172.21.15.7:0/3837349052 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f21940667b0 0x7f2194068c70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f21a4000c00 tx=0x7f21a4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:47.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.421+0000 7f21a2ffd700 1 -- 172.21.15.7:0/3837349052 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f21a8051480 con 0x7f21b40ce000 2024-09-13T18:52:47.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.593+0000 7f21b8b55700 1 -- 172.21.15.7:0/3837349052 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21b40ce410 con 0x7f21b40ce000 2024-09-13T18:52:47.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.593+0000 7f21a2ffd700 1 -- 172.21.15.7:0/3837349052 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f21a800ce60 con 0x7f21b40ce000 2024-09-13T18:52:47.600 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:52:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.597+0000 7f21b8b55700 1 -- 172.21.15.7:0/3837349052 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f21940667b0 msgr2=0x7f2194068c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.597+0000 7f21b8b55700 1 --2- 172.21.15.7:0/3837349052 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f21940667b0 0x7f2194068c70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f21a4000c00 tx=0x7f21a4008030 comp rx=0 tx=0).stop 2024-09-13T18:52:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.597+0000 7f21b8b55700 1 -- 172.21.15.7:0/3837349052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21b40ce000 msgr2=0x7f21b4158a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.597+0000 7f21b8b55700 1 --2- 172.21.15.7:0/3837349052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21b40ce000 0x7f21b4158a50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f21a800e9d0 tx=0x7f21a800ece0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.597+0000 7f21b8b55700 1 -- 172.21.15.7:0/3837349052 shutdown_connections 2024-09-13T18:52:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.597+0000 7f21b8b55700 1 --2- 172.21.15.7:0/3837349052 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f21940667b0 0x7f2194068c70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.597+0000 7f21b8b55700 1 --2- 172.21.15.7:0/3837349052 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f21b415ce40 0x7f21b4158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.597+0000 7f21b8b55700 1 --2- 172.21.15.7:0/3837349052 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21b40ce000 0x7f21b4158a50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.597+0000 7f21b8b55700 1 --2- 172.21.15.7:0/3837349052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f21b406b400 0x7f21b4156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.597+0000 7f21b8b55700 1 -- 172.21.15.7:0/3837349052 >> 172.21.15.7:0/3837349052 conn(0x7f21b40d2f30 msgr2=0x7f21b40d3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.597+0000 7f21b8b55700 1 -- 172.21.15.7:0/3837349052 shutdown_connections 2024-09-13T18:52:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.597+0000 7f21b8b55700 1 -- 172.21.15.7:0/3837349052 wait complete. 2024-09-13T18:52:47.615 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:52:47.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.729+0000 7f06a473c700 1 Processor -- start 2024-09-13T18:52:47.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.729+0000 7f06a473c700 1 -- start start 2024-09-13T18:52:47.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.729+0000 7f06a473c700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f069c1467c0 0x7f069c0ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:47.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.729+0000 7f06a473c700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f069c0b6380 0x7f069c166790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:47.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.729+0000 7f06a473c700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069c0ceaf0 con 0x7f069c0ce5e0 2024-09-13T18:52:47.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.729+0000 7f06a473c700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069c0cedd0 con 0x7f069c0b6380 2024-09-13T18:52:47.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.729+0000 7f06a473c700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c0cf0b0 con 0x7f069c1467c0 2024-09-13T18:52:47.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.729+0000 7f06a24d8700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f069c1467c0 0x7f069c0ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:47.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.729+0000 7f06a2cd9700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f069c0b6380 0x7f069c166790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:47.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.729+0000 7f06a24d8700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f069c1467c0 0x7f069c0ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:42492/0 (socket says 172.21.15.7:42492) 2024-09-13T18:52:47.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.729+0000 7f06a24d8700 1 -- 172.21.15.7:0/3143888605 learned_addr learned my addr 172.21.15.7:0/3143888605 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:47.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.733+0000 7f06a2cd9700 1 -- 172.21.15.7:0/3143888605 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f069c1467c0 msgr2=0x7f069c0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:47.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.733+0000 7f06a2cd9700 1 --2- 172.21.15.7:0/3143888605 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f069c1467c0 0x7f069c0ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.733+0000 7f06a2cd9700 1 -- 172.21.15.7:0/3143888605 >> v1:172.21.15.171:6789/0 conn(0x7f069c0ce5e0 legacy=0x7f069c0b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:47.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.733+0000 7f06a2cd9700 1 -- 172.21.15.7:0/3143888605 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c0cf390 con 0x7f069c0b6380 2024-09-13T18:52:47.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.733+0000 7f06a2cd9700 1 --2- 172.21.15.7:0/3143888605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f069c0b6380 0x7f069c166790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0694009d90 tx=0x7f069400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0c6f689e9afab01 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:47.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.733+0000 7f06a14d6700 1 -- 172.21.15.7:0/3143888605 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069400e070 con 0x7f069c0b6380 2024-09-13T18:52:47.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.733+0000 7f06a14d6700 1 -- 172.21.15.7:0/3143888605 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069400bb20 con 0x7f069c0b6380 2024-09-13T18:52:47.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.733+0000 7f06a14d6700 1 -- 172.21.15.7:0/3143888605 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694012430 con 0x7f069c0b6380 2024-09-13T18:52:47.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.733+0000 7f06a473c700 1 -- 172.21.15.7:0/3143888605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f069c0b6380 msgr2=0x7f069c166790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:47.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.733+0000 7f06a473c700 1 --2- 172.21.15.7:0/3143888605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f069c0b6380 0x7f069c166790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0694009d90 tx=0x7f069400b330 comp rx=0 tx=0).stop 2024-09-13T18:52:47.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.733+0000 7f06a473c700 1 -- 172.21.15.7:0/3143888605 shutdown_connections 2024-09-13T18:52:47.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.733+0000 7f06a473c700 1 --2- 172.21.15.7:0/3143888605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f069c0b6380 0x7f069c166790 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.733+0000 7f06a473c700 1 --2- 172.21.15.7:0/3143888605 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f069c1467c0 0x7f069c0ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.733+0000 7f06a473c700 1 -- 172.21.15.7:0/3143888605 >> 172.21.15.7:0/3143888605 conn(0x7f069c071da0 msgr2=0x7f069c072180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:47.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.733+0000 7f06a473c700 1 -- 172.21.15.7:0/3143888605 shutdown_connections 2024-09-13T18:52:47.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.733+0000 7f06a473c700 1 -- 172.21.15.7:0/3143888605 wait complete. 2024-09-13T18:52:47.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a473c700 1 Processor -- start 2024-09-13T18:52:47.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a473c700 1 -- start start 2024-09-13T18:52:47.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a473c700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f069c0b6380 0x7f069c06c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:47.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a473c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f069c0ce5e0 0x7f069c13c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:47.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a473c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f069c1467c0 0x7f069c15e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:47.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a473c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c13d1d0 con 0x7f069c0ce5e0 2024-09-13T18:52:47.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a473c700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c06cd30 con 0x7f069c1467c0 2024-09-13T18:52:47.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a473c700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069c06d010 con 0x7f069c0b6380 2024-09-13T18:52:47.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a1cd7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f069c0ce5e0 0x7f069c13c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:47.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a1cd7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f069c0ce5e0 0x7f069c13c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46062/0 (socket says 172.21.15.7:46062) 2024-09-13T18:52:47.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a1cd7700 1 -- 172.21.15.7:0/911052016 learned_addr learned my addr 172.21.15.7:0/911052016 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:47.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a24d8700 1 --2- 172.21.15.7:0/911052016 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f069c0b6380 0x7f069c06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:47.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a2cd9700 1 --2- 172.21.15.7:0/911052016 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f069c1467c0 0x7f069c15e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:47.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a1cd7700 1 -- 172.21.15.7:0/911052016 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f069c0b6380 msgr2=0x7f069c06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:47.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a1cd7700 1 --2- 172.21.15.7:0/911052016 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f069c0b6380 0x7f069c06c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a1cd7700 1 -- 172.21.15.7:0/911052016 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f069c1467c0 msgr2=0x7f069c15e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:47.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a1cd7700 1 --2- 172.21.15.7:0/911052016 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f069c1467c0 0x7f069c15e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a1cd7700 1 -- 172.21.15.7:0/911052016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0694009a70 con 0x7f069c0ce5e0 2024-09-13T18:52:47.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a2cd9700 1 --2- 172.21.15.7:0/911052016 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f069c1467c0 0x7f069c15e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:47.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a1cd7700 1 --2- 172.21.15.7:0/911052016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f069c0ce5e0 0x7f069c13c420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f069800c4d0 tx=0x7f069800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:47.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f0692ffd700 1 -- 172.21.15.7:0/911052016 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06980052d0 con 0x7f069c0ce5e0 2024-09-13T18:52:47.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f0692ffd700 1 -- 172.21.15.7:0/911052016 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0698005430 con 0x7f069c0ce5e0 2024-09-13T18:52:47.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a473c700 1 -- 172.21.15.7:0/911052016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c06d2f0 con 0x7f069c0ce5e0 2024-09-13T18:52:47.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f0692ffd700 1 -- 172.21.15.7:0/911052016 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06980055a0 con 0x7f069c0ce5e0 2024-09-13T18:52:47.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.737+0000 7f06a473c700 1 -- 172.21.15.7:0/911052016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c06e870 con 0x7f069c0ce5e0 2024-09-13T18:52:47.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.741+0000 7f06a473c700 1 -- 172.21.15.7:0/911052016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069c0cd630 con 0x7f069c0ce5e0 2024-09-13T18:52:47.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.741+0000 7f0692ffd700 1 -- 172.21.15.7:0/911052016 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f06980038c0 con 0x7f069c0ce5e0 2024-09-13T18:52:47.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.741+0000 7f0692ffd700 1 --2- 172.21.15.7:0/911052016 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0680066730 0x7f0680068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:47.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.741+0000 7f0692ffd700 1 -- 172.21.15.7:0/911052016 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06980835d0 con 0x7f069c0ce5e0 2024-09-13T18:52:47.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.741+0000 7f06a24d8700 1 --2- 172.21.15.7:0/911052016 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0680066730 0x7f0680068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:47.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.745+0000 7f06a24d8700 1 --2- 172.21.15.7:0/911052016 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0680066730 0x7f0680068bf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f069c140270 tx=0x7f068c0092e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:47.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.753+0000 7f0692ffd700 1 -- 172.21.15.7:0/911052016 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0698050930 con 0x7f069c0ce5e0 2024-09-13T18:52:47.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.917+0000 7f06a473c700 1 -- 172.21.15.7:0/911052016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f069c0ce0a0 con 0x7f069c0ce5e0 2024-09-13T18:52:47.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.921+0000 7f0692ffd700 1 -- 172.21.15.7:0/911052016 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0698053f50 con 0x7f069c0ce5e0 2024-09-13T18:52:47.925 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:52:47.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.921+0000 7f06a473c700 1 -- 172.21.15.7:0/911052016 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0680066730 msgr2=0x7f0680068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:47.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.921+0000 7f06a473c700 1 --2- 172.21.15.7:0/911052016 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0680066730 0x7f0680068bf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f069c140270 tx=0x7f068c0092e0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.921+0000 7f06a473c700 1 -- 172.21.15.7:0/911052016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f069c0ce5e0 msgr2=0x7f069c13c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:47.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.921+0000 7f06a473c700 1 --2- 172.21.15.7:0/911052016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f069c0ce5e0 0x7f069c13c420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f069800c4d0 tx=0x7f069800c7e0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.921+0000 7f06a473c700 1 -- 172.21.15.7:0/911052016 shutdown_connections 2024-09-13T18:52:47.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.921+0000 7f06a473c700 1 --2- 172.21.15.7:0/911052016 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0680066730 0x7f0680068bf0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.921+0000 7f06a473c700 1 --2- 172.21.15.7:0/911052016 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f069c1467c0 0x7f069c15e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.921+0000 7f06a473c700 1 --2- 172.21.15.7:0/911052016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f069c0ce5e0 0x7f069c13c420 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.921+0000 7f06a473c700 1 --2- 172.21.15.7:0/911052016 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f069c0b6380 0x7f069c06c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:47.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.921+0000 7f06a473c700 1 -- 172.21.15.7:0/911052016 >> 172.21.15.7:0/911052016 conn(0x7f069c071da0 msgr2=0x7f069c0b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:47.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.921+0000 7f06a473c700 1 -- 172.21.15.7:0/911052016 shutdown_connections 2024-09-13T18:52:47.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:47.921+0000 7f06a473c700 1 -- 172.21.15.7:0/911052016 wait complete. 2024-09-13T18:52:47.941 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:52:48.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.061+0000 7fb0c9d40700 1 Processor -- start 2024-09-13T18:52:48.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.061+0000 7fb0c9d40700 1 -- start start 2024-09-13T18:52:48.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.061+0000 7fb0c9d40700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fb0bc03d590 0x7fb0bc0ed770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.061+0000 7fb0c9d40700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0bc053d60 con 0x7fb0bc057cc0 2024-09-13T18:52:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.061+0000 7fb0c9d40700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0bc03bb50 con 0x7fb0bc005e10 2024-09-13T18:52:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.061+0000 7fb0c9d40700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0bc03c0f0 con 0x7fb0bc03d590 2024-09-13T18:52:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.061+0000 7fb0c3fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb0bc057cc0 0x7fb0bc0061f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57864/0 (socket says 172.21.15.7:57864) 2024-09-13T18:52:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.061+0000 7fb0c3fff700 1 -- 172.21.15.7:0/2340886364 learned_addr learned my addr 172.21.15.7:0/2340886364 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:48.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.061+0000 7fb0c953f700 1 --2- 172.21.15.7:0/2340886364 >> v2:172.21.15.171:3300/0 conn(0x7fb0bc03d590 0x7fb0bc0ed770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.061+0000 7fb0c37fe700 1 -- 172.21.15.7:0/2340886364 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211957149 0 0) 0x7fb0bc053d60 con 0x7fb0bc057cc0 2024-09-13T18:52:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.061+0000 7fb0c37fe700 1 -- 172.21.15.7:0/2340886364 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0ac0035d0 con 0x7fb0bc057cc0 2024-09-13T18:52:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.061+0000 7fb0c953f700 1 -- 172.21.15.7:0/2340886364 >> v1:172.21.15.171:6789/0 conn(0x7fb0bc005e10 legacy=0x7fb0bc0575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.061+0000 7fb0c953f700 1 -- 172.21.15.7:0/2340886364 >> v1:172.21.15.7:6789/0 conn(0x7fb0bc057cc0 legacy=0x7fb0bc0061f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.061+0000 7fb0c37fe700 1 -- 172.21.15.7:0/2340886364 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3024745793 0 0) 0x7fb0ac0035d0 con 0x7fb0bc057cc0 2024-09-13T18:52:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.061+0000 7fb0c953f700 1 -- 172.21.15.7:0/2340886364 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc058190 con 0x7fb0bc03d590 2024-09-13T18:52:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.061+0000 7fb0c953f700 1 --2- 172.21.15.7:0/2340886364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0bc03d590 0x7fb0bc0ed770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb0c404f480 tx=0x7fb0c4074840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23d4404878405682 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.061+0000 7fb0c37fe700 1 -- 172.21.15.7:0/2340886364 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8001670 con 0x7fb0bc03d590 2024-09-13T18:52:48.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.061+0000 7fb0c37fe700 1 -- 172.21.15.7:0/2340886364 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c4073ca0 con 0x7fb0bc03d590 2024-09-13T18:52:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.065+0000 7fb0c37fe700 1 -- 172.21.15.7:0/2340886364 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c40730f0 con 0x7fb0bc03d590 2024-09-13T18:52:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.065+0000 7fb0c9d40700 1 -- 172.21.15.7:0/2340886364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0bc03d590 msgr2=0x7fb0bc0ed770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.065+0000 7fb0c9d40700 1 --2- 172.21.15.7:0/2340886364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0bc03d590 0x7fb0bc0ed770 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb0c404f480 tx=0x7fb0c4074840 comp rx=0 tx=0).stop 2024-09-13T18:52:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.065+0000 7fb0c9d40700 1 -- 172.21.15.7:0/2340886364 shutdown_connections 2024-09-13T18:52:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.065+0000 7fb0c9d40700 1 --2- 172.21.15.7:0/2340886364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0bc03d590 0x7fb0bc0ed770 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.065+0000 7fb0c9d40700 1 -- 172.21.15.7:0/2340886364 >> 172.21.15.7:0/2340886364 conn(0x7fb0bc03a550 msgr2=0x7fb0bc03a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.065+0000 7fb0c9d40700 1 -- 172.21.15.7:0/2340886364 shutdown_connections 2024-09-13T18:52:48.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.065+0000 7fb0c9d40700 1 -- 172.21.15.7:0/2340886364 wait complete. 2024-09-13T18:52:48.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.065+0000 7fb0c9d40700 1 Processor -- start 2024-09-13T18:52:48.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.065+0000 7fb0c9d40700 1 -- start start 2024-09-13T18:52:48.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c9d40700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc005e10 0x7fb0bc0d6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:48.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c9d40700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb0bc03d590 0x7fb0bc04c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:48.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c9d40700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0bc057cc0 0x7fb0bc04cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:48.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c9d40700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc0d72e0 con 0x7fb0bc005e10 2024-09-13T18:52:48.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c9d40700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc050920 con 0x7fb0bc057cc0 2024-09-13T18:52:48.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c9d40700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc0e52f0 con 0x7fb0bc03d590 2024-09-13T18:52:48.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c8d3e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc005e10 0x7fb0bc0d6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:48.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c8d3e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc005e10 0x7fb0bc0d6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46078/0 (socket says 172.21.15.7:46078) 2024-09-13T18:52:48.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c8d3e700 1 -- 172.21.15.7:0/1677118560 learned_addr learned my addr 172.21.15.7:0/1677118560 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:48.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c953f700 1 --2- 172.21.15.7:0/1677118560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0bc057cc0 0x7fb0bc04cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:48.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c3fff700 1 --2- 172.21.15.7:0/1677118560 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb0bc03d590 0x7fb0bc04c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:48.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c8d3e700 1 -- 172.21.15.7:0/1677118560 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb0bc03d590 msgr2=0x7fb0bc04c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:48.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c8d3e700 1 --2- 172.21.15.7:0/1677118560 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb0bc03d590 0x7fb0bc04c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c8d3e700 1 -- 172.21.15.7:0/1677118560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0bc057cc0 msgr2=0x7fb0bc04cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:48.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c8d3e700 1 --2- 172.21.15.7:0/1677118560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0bc057cc0 0x7fb0bc04cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c8d3e700 1 -- 172.21.15.7:0/1677118560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c404f5c0 con 0x7fb0bc005e10 2024-09-13T18:52:48.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c3fff700 1 --2- 172.21.15.7:0/1677118560 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb0bc03d590 0x7fb0bc04c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:48.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c8d3e700 1 --2- 172.21.15.7:0/1677118560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc005e10 0x7fb0bc0d6da0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400c500 tx=0x7fb0b400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:48.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c17fa700 1 -- 172.21.15.7:0/1677118560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40052d0 con 0x7fb0bc005e10 2024-09-13T18:52:48.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c17fa700 1 -- 172.21.15.7:0/1677118560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b4005430 con 0x7fb0bc005e10 2024-09-13T18:52:48.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c9d40700 1 -- 172.21.15.7:0/1677118560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0bc0502d0 con 0x7fb0bc005e10 2024-09-13T18:52:48.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c17fa700 1 -- 172.21.15.7:0/1677118560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40055a0 con 0x7fb0bc005e10 2024-09-13T18:52:48.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.069+0000 7fb0c9d40700 1 -- 172.21.15.7:0/1677118560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0bc0e45a0 con 0x7fb0bc005e10 2024-09-13T18:52:48.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.073+0000 7fb0c9d40700 1 -- 172.21.15.7:0/1677118560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0bc0e55d0 con 0x7fb0bc005e10 2024-09-13T18:52:48.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.073+0000 7fb0c17fa700 1 -- 172.21.15.7:0/1677118560 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb0b40038e0 con 0x7fb0bc005e10 2024-09-13T18:52:48.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.073+0000 7fb0c17fa700 1 --2- 172.21.15.7:0/1677118560 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb0b0066730 0x7fb0b0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:48.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.073+0000 7fb0c17fa700 1 -- 172.21.15.7:0/1677118560 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0b4083670 con 0x7fb0bc005e10 2024-09-13T18:52:48.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.073+0000 7fb0c3fff700 1 --2- 172.21.15.7:0/1677118560 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb0b0066730 0x7fb0b0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:48.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.073+0000 7fb0c3fff700 1 --2- 172.21.15.7:0/1677118560 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb0b0066730 0x7fb0b0068bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc17bdf0 tx=0x7fb0b8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.077+0000 7fb0c17fa700 1 -- 172.21.15.7:0/1677118560 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb0b4050940 con 0x7fb0bc005e10 2024-09-13T18:52:48.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.245+0000 7fb0c9d40700 1 -- 172.21.15.7:0/1677118560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0bc0381a0 con 0x7fb0bc005e10 2024-09-13T18:52:48.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.245+0000 7fb0c17fa700 1 -- 172.21.15.7:0/1677118560 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb0b4053f60 con 0x7fb0bc005e10 2024-09-13T18:52:48.252 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:52:48.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.249+0000 7fb0c9d40700 1 -- 172.21.15.7:0/1677118560 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb0b0066730 msgr2=0x7fb0b0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:48.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.249+0000 7fb0c9d40700 1 --2- 172.21.15.7:0/1677118560 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb0b0066730 0x7fb0b0068bf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc17bdf0 tx=0x7fb0b8009040 comp rx=0 tx=0).stop 2024-09-13T18:52:48.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.249+0000 7fb0c9d40700 1 -- 172.21.15.7:0/1677118560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc005e10 msgr2=0x7fb0bc0d6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:48.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.249+0000 7fb0c9d40700 1 --2- 172.21.15.7:0/1677118560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc005e10 0x7fb0bc0d6da0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400c500 tx=0x7fb0b400c810 comp rx=0 tx=0).stop 2024-09-13T18:52:48.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.249+0000 7fb0c9d40700 1 -- 172.21.15.7:0/1677118560 shutdown_connections 2024-09-13T18:52:48.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.249+0000 7fb0c9d40700 1 --2- 172.21.15.7:0/1677118560 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb0b0066730 0x7fb0b0068bf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.249+0000 7fb0c9d40700 1 --2- 172.21.15.7:0/1677118560 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0bc057cc0 0x7fb0bc04cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.249+0000 7fb0c9d40700 1 --2- 172.21.15.7:0/1677118560 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb0bc03d590 0x7fb0bc04c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.249+0000 7fb0c9d40700 1 --2- 172.21.15.7:0/1677118560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc005e10 0x7fb0bc0d6da0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.249+0000 7fb0c9d40700 1 -- 172.21.15.7:0/1677118560 >> 172.21.15.7:0/1677118560 conn(0x7fb0bc03a550 msgr2=0x7fb0bc0383c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:48.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.249+0000 7fb0c9d40700 1 -- 172.21.15.7:0/1677118560 shutdown_connections 2024-09-13T18:52:48.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.249+0000 7fb0c9d40700 1 -- 172.21.15.7:0/1677118560 wait complete. 2024-09-13T18:52:48.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj' 2024-09-13T18:52:48.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.377+0000 7f7ea994e700 1 Processor -- start 2024-09-13T18:52:48.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.377+0000 7f7ea994e700 1 -- start start 2024-09-13T18:52:48.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.377+0000 7f7ea994e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ea415cae0 0x7f7ea40cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.377+0000 7f7ea994e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f7ea415cec0 0x7f7ea40d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.377+0000 7f7ea994e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ea406ac00 0x7f7ea406b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.377+0000 7f7ea994e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea40d1750 con 0x7f7ea415cae0 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.377+0000 7f7ea994e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea406b590 con 0x7f7ea415cec0 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.377+0000 7f7ea994e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40d21e0 con 0x7f7ea406ac00 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.377+0000 7f7ea894c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ea415cae0 0x7f7ea40cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.377+0000 7f7ea894c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7ea415cae0 0x7f7ea40cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46084/0 (socket says 172.21.15.7:46084) 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.377+0000 7f7ea894c700 1 -- 172.21.15.7:0/3258108751 learned_addr learned my addr 172.21.15.7:0/3258108751 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea3fff700 1 --2- 172.21.15.7:0/3258108751 >> v2:172.21.15.171:3300/0 conn(0x7f7ea415cec0 0x7f7ea40d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea914d700 1 --2- 172.21.15.7:0/3258108751 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ea406ac00 0x7f7ea406b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:48.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea894c700 1 -- 172.21.15.7:0/3258108751 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ea406ac00 msgr2=0x7f7ea406b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea894c700 1 --2- 172.21.15.7:0/3258108751 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ea406ac00 0x7f7ea406b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea894c700 1 -- 172.21.15.7:0/3258108751 >> v2:172.21.15.171:3300/0 conn(0x7f7ea415cec0 msgr2=0x7f7ea40d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea894c700 1 --2- 172.21.15.7:0/3258108751 >> v2:172.21.15.171:3300/0 conn(0x7f7ea415cec0 0x7f7ea40d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea894c700 1 -- 172.21.15.7:0/3258108751 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea40d1940 con 0x7f7ea415cae0 2024-09-13T18:52:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea894c700 1 --2- 172.21.15.7:0/3258108751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea415cae0 0x7f7ea40cf030 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009910 tx=0x7f7e98009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b0cba18950d3cf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea37fe700 1 -- 172.21.15.7:0/3258108751 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98011830 con 0x7f7ea415cae0 2024-09-13T18:52:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea37fe700 1 -- 172.21.15.7:0/3258108751 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e98011990 con 0x7f7ea415cae0 2024-09-13T18:52:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea37fe700 1 -- 172.21.15.7:0/3258108751 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98011d40 con 0x7f7ea415cae0 2024-09-13T18:52:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea994e700 1 -- 172.21.15.7:0/3258108751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea415cae0 msgr2=0x7f7ea40cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea994e700 1 --2- 172.21.15.7:0/3258108751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea415cae0 0x7f7ea40cf030 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009910 tx=0x7f7e98009c20 comp rx=0 tx=0).stop 2024-09-13T18:52:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea994e700 1 -- 172.21.15.7:0/3258108751 shutdown_connections 2024-09-13T18:52:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea994e700 1 --2- 172.21.15.7:0/3258108751 >> v2:172.21.15.171:3300/0 conn(0x7f7ea415cec0 0x7f7ea40d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea994e700 1 --2- 172.21.15.7:0/3258108751 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ea406ac00 0x7f7ea406b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea994e700 1 --2- 172.21.15.7:0/3258108751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea415cae0 0x7f7ea40cf030 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea994e700 1 -- 172.21.15.7:0/3258108751 >> 172.21.15.7:0/3258108751 conn(0x7f7ea40d2c80 msgr2=0x7f7ea40d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:48.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea994e700 1 -- 172.21.15.7:0/3258108751 shutdown_connections 2024-09-13T18:52:48.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.381+0000 7f7ea994e700 1 -- 172.21.15.7:0/3258108751 wait complete. 2024-09-13T18:52:48.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea994e700 1 Processor -- start 2024-09-13T18:52:48.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea994e700 1 -- start start 2024-09-13T18:52:48.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea994e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ea406ac00 0x7f7ea40a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:48.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea994e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ea415cae0 0x7f7ea41f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:48.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea994e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea415cec0 0x7f7ea4158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:48.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea994e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea41f26f0 con 0x7f7ea415cec0 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea994e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40aa3c0 con 0x7f7ea415cae0 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea994e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40b3de0 con 0x7f7ea406ac00 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea914d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea415cec0 0x7f7ea4158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea914d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea415cec0 0x7f7ea4158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46086/0 (socket says 172.21.15.7:46086) 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea914d700 1 -- 172.21.15.7:0/1059967965 learned_addr learned my addr 172.21.15.7:0/1059967965 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea3fff700 1 --2- 172.21.15.7:0/1059967965 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ea415cae0 0x7f7ea41f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea894c700 1 --2- 172.21.15.7:0/1059967965 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ea406ac00 0x7f7ea40a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea914d700 1 -- 172.21.15.7:0/1059967965 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ea406ac00 msgr2=0x7f7ea40a95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea914d700 1 --2- 172.21.15.7:0/1059967965 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ea406ac00 0x7f7ea40a95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea914d700 1 -- 172.21.15.7:0/1059967965 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ea415cae0 msgr2=0x7f7ea41f21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea914d700 1 --2- 172.21.15.7:0/1059967965 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ea415cae0 0x7f7ea41f21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:48.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea914d700 1 -- 172.21.15.7:0/1059967965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e98009570 con 0x7f7ea415cec0 2024-09-13T18:52:48.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea914d700 1 --2- 172.21.15.7:0/1059967965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea415cec0 0x7f7ea4158f40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00eed0 tx=0x7f7e9c00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:48.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea17fa700 1 -- 172.21.15.7:0/1059967965 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c010040 con 0x7f7ea415cec0 2024-09-13T18:52:48.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea17fa700 1 -- 172.21.15.7:0/1059967965 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9c00baa0 con 0x7f7ea415cec0 2024-09-13T18:52:48.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea994e700 1 -- 172.21.15.7:0/1059967965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea40b40c0 con 0x7f7ea415cec0 2024-09-13T18:52:48.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea17fa700 1 -- 172.21.15.7:0/1059967965 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00be50 con 0x7f7ea415cec0 2024-09-13T18:52:48.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.385+0000 7f7ea994e700 1 -- 172.21.15.7:0/1059967965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea4159480 con 0x7f7ea415cec0 2024-09-13T18:52:48.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.389+0000 7f7ea994e700 1 -- 172.21.15.7:0/1059967965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea40b45e0 con 0x7f7ea415cec0 2024-09-13T18:52:48.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.389+0000 7f7ea17fa700 1 -- 172.21.15.7:0/1059967965 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7e9c014400 con 0x7f7ea415cec0 2024-09-13T18:52:48.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.389+0000 7f7ea17fa700 1 --2- 172.21.15.7:0/1059967965 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7e90066730 0x7f7e90068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:48.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.389+0000 7f7ea17fa700 1 -- 172.21.15.7:0/1059967965 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e9c084450 con 0x7f7ea415cec0 2024-09-13T18:52:48.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.389+0000 7f7ea894c700 1 --2- 172.21.15.7:0/1059967965 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7e90066730 0x7f7e90068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:48.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.389+0000 7f7ea894c700 1 --2- 172.21.15.7:0/1059967965 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7e90066730 0x7f7e90068bf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7e98000c00 tx=0x7f7e98008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:48.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.397+0000 7f7ea17fa700 1 -- 172.21.15.7:0/1059967965 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7e9c0517d0 con 0x7f7ea415cec0 2024-09-13T18:52:48.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:48.637+0000 7f7ea994e700 1 -- 172.21.15.7:0/1059967965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj"]} v 0) v1 -- 0x7f7ea406b050 con 0x7f7ea415cec0 2024-09-13T18:52:49.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.333+0000 7f7ea17fa700 1 -- 172.21.15.7:0/1059967965 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj"]}]=0 v46) v1 ==== 150+0+0 (secure 0 0 0) 0x7f7e9c054df0 con 0x7f7ea415cec0 2024-09-13T18:52:49.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.337+0000 7f7ea994e700 1 -- 172.21.15.7:0/1059967965 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7e90066730 msgr2=0x7f7e90068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:49.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.337+0000 7f7ea994e700 1 --2- 172.21.15.7:0/1059967965 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7e90066730 0x7f7e90068bf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7e98000c00 tx=0x7f7e98008030 comp rx=0 tx=0).stop 2024-09-13T18:52:49.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.337+0000 7f7ea994e700 1 -- 172.21.15.7:0/1059967965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea415cec0 msgr2=0x7f7ea4158f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:49.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.337+0000 7f7ea994e700 1 --2- 172.21.15.7:0/1059967965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea415cec0 0x7f7ea4158f40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00eed0 tx=0x7f7e9c00ef00 comp rx=0 tx=0).stop 2024-09-13T18:52:49.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.337+0000 7f7ea994e700 1 -- 172.21.15.7:0/1059967965 shutdown_connections 2024-09-13T18:52:49.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.337+0000 7f7ea994e700 1 --2- 172.21.15.7:0/1059967965 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7e90066730 0x7f7e90068bf0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:49.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.337+0000 7f7ea994e700 1 --2- 172.21.15.7:0/1059967965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ea415cec0 0x7f7ea4158f40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:49.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.337+0000 7f7ea994e700 1 --2- 172.21.15.7:0/1059967965 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ea415cae0 0x7f7ea41f21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:49.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.337+0000 7f7ea994e700 1 --2- 172.21.15.7:0/1059967965 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ea406ac00 0x7f7ea40a95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:49.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.337+0000 7f7ea994e700 1 -- 172.21.15.7:0/1059967965 >> 172.21.15.7:0/1059967965 conn(0x7f7ea40d2c80 msgr2=0x7f7ea40d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:49.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.337+0000 7f7ea994e700 1 -- 172.21.15.7:0/1059967965 shutdown_connections 2024-09-13T18:52:49.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.337+0000 7f7ea994e700 1 -- 172.21.15.7:0/1059967965 wait complete. 2024-09-13T18:52:49.362 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T18:52:49.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.481+0000 7f6b972fa700 1 Processor -- start 2024-09-13T18:52:49.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.481+0000 7f6b972fa700 1 -- start start 2024-09-13T18:52:49.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b972fa700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6b9006c800 0x7f6b9006a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:49.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b972fa700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f6b9006e420 0x7f6b9006e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:49.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b972fa700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b9015a0c0 con 0x7f6b9006c420 2024-09-13T18:52:49.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b972fa700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b9006edf0 con 0x7f6b9006e420 2024-09-13T18:52:49.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b972fa700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9015a3a0 con 0x7f6b9006c800 2024-09-13T18:52:49.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b94895700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6b9006c800 0x7f6b9006a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:49.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b95897700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f6b9006e420 0x7f6b9006e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:49.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b95897700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f6b9006e420 0x7f6b9006e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:56638/0 (socket says 172.21.15.7:56638) 2024-09-13T18:52:49.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b95897700 1 -- 172.21.15.7:0/4119720175 learned_addr learned my addr 172.21.15.7:0/4119720175 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:49.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b94895700 1 -- 172.21.15.7:0/4119720175 >> v2:172.21.15.171:3300/0 conn(0x7f6b9006e420 msgr2=0x7f6b9006e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:49.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b94895700 1 --2- 172.21.15.7:0/4119720175 >> v2:172.21.15.171:3300/0 conn(0x7f6b9006e420 0x7f6b9006e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:49.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b94895700 1 -- 172.21.15.7:0/4119720175 >> v1:172.21.15.171:6789/0 conn(0x7f6b9006c420 legacy=0x7f6b900cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:49.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b94895700 1 -- 172.21.15.7:0/4119720175 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9015a680 con 0x7f6b9006c800 2024-09-13T18:52:49.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b95897700 1 --2- 172.21.15.7:0/4119720175 >> v2:172.21.15.171:3300/0 conn(0x7f6b9006e420 0x7f6b9006e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b94895700 1 --2- 172.21.15.7:0/4119720175 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6b9006c800 0x7f6b9006a310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009850 tx=0x7f6b80009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57f06178ed5e78c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b87fff700 1 -- 172.21.15.7:0/4119720175 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b80011830 con 0x7f6b9006c800 2024-09-13T18:52:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b87fff700 1 -- 172.21.15.7:0/4119720175 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b80011990 con 0x7f6b9006c800 2024-09-13T18:52:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b87fff700 1 -- 172.21.15.7:0/4119720175 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b80011d40 con 0x7f6b9006c800 2024-09-13T18:52:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b972fa700 1 -- 172.21.15.7:0/4119720175 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6b9006c800 msgr2=0x7f6b9006a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b972fa700 1 --2- 172.21.15.7:0/4119720175 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6b9006c800 0x7f6b9006a310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009850 tx=0x7f6b80009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b972fa700 1 -- 172.21.15.7:0/4119720175 shutdown_connections 2024-09-13T18:52:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b972fa700 1 --2- 172.21.15.7:0/4119720175 >> v2:172.21.15.171:3300/0 conn(0x7f6b9006e420 0x7f6b9006e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b972fa700 1 --2- 172.21.15.7:0/4119720175 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6b9006c800 0x7f6b9006a310 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b972fa700 1 -- 172.21.15.7:0/4119720175 >> 172.21.15.7:0/4119720175 conn(0x7f6b9006d900 msgr2=0x7f6b9006dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b972fa700 1 -- 172.21.15.7:0/4119720175 shutdown_connections 2024-09-13T18:52:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.485+0000 7f6b972fa700 1 -- 172.21.15.7:0/4119720175 wait complete. 2024-09-13T18:52:49.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.489+0000 7f6b972fa700 1 Processor -- start 2024-09-13T18:52:49.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.489+0000 7f6b972fa700 1 -- start start 2024-09-13T18:52:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.489+0000 7f6b972fa700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b9006c420 0x7f6b9014c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.489+0000 7f6b972fa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b9006c800 0x7f6b901ee910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.489+0000 7f6b972fa700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6b9006e420 0x7f6b9014d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.489+0000 7f6b972fa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9014cb10 con 0x7f6b9006c800 2024-09-13T18:52:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.489+0000 7f6b972fa700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b901eee50 con 0x7f6b9006c420 2024-09-13T18:52:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.489+0000 7f6b972fa700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b901f1cc0 con 0x7f6b9006e420 2024-09-13T18:52:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.489+0000 7f6b94895700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b9006c800 0x7f6b901ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.489+0000 7f6b94895700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b9006c800 0x7f6b901ee910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46100/0 (socket says 172.21.15.7:46100) 2024-09-13T18:52:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.489+0000 7f6b94895700 1 -- 172.21.15.7:0/251233706 learned_addr learned my addr 172.21.15.7:0/251233706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.493+0000 7f6b95096700 1 --2- 172.21.15.7:0/251233706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b9006c420 0x7f6b9014c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.493+0000 7f6b95897700 1 --2- 172.21.15.7:0/251233706 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6b9006e420 0x7f6b9014d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.493+0000 7f6b94895700 1 -- 172.21.15.7:0/251233706 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6b9006e420 msgr2=0x7f6b9014d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.493+0000 7f6b94895700 1 --2- 172.21.15.7:0/251233706 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6b9006e420 0x7f6b9014d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.493+0000 7f6b94895700 1 -- 172.21.15.7:0/251233706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b9006c420 msgr2=0x7f6b9014c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.493+0000 7f6b94895700 1 --2- 172.21.15.7:0/251233706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b9006c420 0x7f6b9014c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.493+0000 7f6b94895700 1 -- 172.21.15.7:0/251233706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b88009b30 con 0x7f6b9006c800 2024-09-13T18:52:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.493+0000 7f6b95096700 1 --2- 172.21.15.7:0/251233706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b9006c420 0x7f6b9014c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.493+0000 7f6b94895700 1 --2- 172.21.15.7:0/251233706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b9006c800 0x7f6b901ee910 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6b9015a4e0 tx=0x7f6b80004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.493+0000 7f6b85ffb700 1 -- 172.21.15.7:0/251233706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b80004530 con 0x7f6b9006c800 2024-09-13T18:52:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.493+0000 7f6b85ffb700 1 -- 172.21.15.7:0/251233706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8001f070 con 0x7f6b9006c800 2024-09-13T18:52:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.493+0000 7f6b972fa700 1 -- 172.21.15.7:0/251233706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b800094b0 con 0x7f6b9006c800 2024-09-13T18:52:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.493+0000 7f6b85ffb700 1 -- 172.21.15.7:0/251233706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b80003900 con 0x7f6b9006c800 2024-09-13T18:52:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.493+0000 7f6b972fa700 1 -- 172.21.15.7:0/251233706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b9014e510 con 0x7f6b9006c800 2024-09-13T18:52:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.493+0000 7f6b972fa700 1 -- 172.21.15.7:0/251233706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b900cba00 con 0x7f6b9006c800 2024-09-13T18:52:49.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.493+0000 7f6b85ffb700 1 -- 172.21.15.7:0/251233706 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6b80021030 con 0x7f6b9006c800 2024-09-13T18:52:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.493+0000 7f6b85ffb700 1 --2- 172.21.15.7:0/251233706 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6b74066730 0x7f6b74068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.493+0000 7f6b85ffb700 1 -- 172.21.15.7:0/251233706 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b80085500 con 0x7f6b9006c800 2024-09-13T18:52:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.497+0000 7f6b95096700 1 --2- 172.21.15.7:0/251233706 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6b74066730 0x7f6b74068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.497+0000 7f6b95096700 1 --2- 172.21.15.7:0/251233706 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6b74066730 0x7f6b74068bf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6b90154700 tx=0x7f6b8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:49.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.505+0000 7f6b85ffb700 1 -- 172.21.15.7:0/251233706 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6b900cba00 con 0x7f6b9006c800 2024-09-13T18:52:49.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.677+0000 7f6b972fa700 1 -- 172.21.15.7:0/251233706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6b900cba00 con 0x7f6b9006c800 2024-09-13T18:52:49.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.685+0000 7f6b85ffb700 1 -- 172.21.15.7:0/251233706 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6b900cba00 con 0x7f6b9006c800 2024-09-13T18:52:49.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.689+0000 7f6b972fa700 1 -- 172.21.15.7:0/251233706 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6b74066730 msgr2=0x7f6b74068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:49.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.689+0000 7f6b972fa700 1 --2- 172.21.15.7:0/251233706 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6b74066730 0x7f6b74068bf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6b90154700 tx=0x7f6b8c009040 comp rx=0 tx=0).stop 2024-09-13T18:52:49.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.689+0000 7f6b972fa700 1 -- 172.21.15.7:0/251233706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b9006c800 msgr2=0x7f6b901ee910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:49.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.689+0000 7f6b972fa700 1 --2- 172.21.15.7:0/251233706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b9006c800 0x7f6b901ee910 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6b9015a4e0 tx=0x7f6b80004770 comp rx=0 tx=0).stop 2024-09-13T18:52:49.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.689+0000 7f6b972fa700 1 -- 172.21.15.7:0/251233706 shutdown_connections 2024-09-13T18:52:49.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.689+0000 7f6b972fa700 1 --2- 172.21.15.7:0/251233706 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6b74066730 0x7f6b74068bf0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:49.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.689+0000 7f6b972fa700 1 --2- 172.21.15.7:0/251233706 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6b9006e420 0x7f6b9014d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:49.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.689+0000 7f6b972fa700 1 --2- 172.21.15.7:0/251233706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b9006c800 0x7f6b901ee910 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:49.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.689+0000 7f6b972fa700 1 --2- 172.21.15.7:0/251233706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6b9006c420 0x7f6b9014c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:49.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.689+0000 7f6b972fa700 1 -- 172.21.15.7:0/251233706 >> 172.21.15.7:0/251233706 conn(0x7f6b9006d900 msgr2=0x7f6b900af590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:49.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.689+0000 7f6b972fa700 1 -- 172.21.15.7:0/251233706 shutdown_connections 2024-09-13T18:52:49.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.689+0000 7f6b972fa700 1 -- 172.21.15.7:0/251233706 wait complete. 2024-09-13T18:52:49.710 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T18:52:49.710 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T18:52:49.724 DEBUG:teuthology.orchestra.run.smithi171:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T18:52:49.744 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T18:52:49.744 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:52:49.744 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:52:49.751 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:52:49.751 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:52:49.800 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:49.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.909+0000 7f7acd192700 1 Processor -- start 2024-09-13T18:52:49.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.909+0000 7f7acd192700 1 -- start start 2024-09-13T18:52:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.913+0000 7f7acd192700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ac80d0a10 0x7f7ac80d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.913+0000 7f7acd192700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac80b5050 con 0x7f7ac80cb150 2024-09-13T18:52:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.913+0000 7f7acd192700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac80ce3f0 con 0x7f7ac80b57e0 2024-09-13T18:52:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.913+0000 7f7acd192700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac815b3e0 con 0x7f7ac80d0a10 2024-09-13T18:52:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.913+0000 7f7ac77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7ac80cb150 0x7f7ac80cb590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57874/0 (socket says 172.21.15.7:57874) 2024-09-13T18:52:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.913+0000 7f7ac77fe700 1 -- 172.21.15.7:0/21949235 learned_addr learned my addr 172.21.15.7:0/21949235 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.913+0000 7f7ac7fff700 1 --2- 172.21.15.7:0/21949235 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ac80d0a10 0x7f7ac80d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:49.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.913+0000 7f7ac6ffd700 1 -- 172.21.15.7:0/21949235 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765458925 0 0) 0x7f7ac80ce3f0 con 0x7f7ac80b57e0 2024-09-13T18:52:49.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.913+0000 7f7ac6ffd700 1 -- 172.21.15.7:0/21949235 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ab00035d0 con 0x7f7ac80b57e0 2024-09-13T18:52:49.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.913+0000 7f7ac6ffd700 1 -- 172.21.15.7:0/21949235 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235754052 0 0) 0x7f7ac80b5050 con 0x7f7ac80cb150 2024-09-13T18:52:49.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.913+0000 7f7ac6ffd700 1 -- 172.21.15.7:0/21949235 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ac80ce3f0 con 0x7f7ac80cb150 2024-09-13T18:52:49.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.913+0000 7f7ac7fff700 1 -- 172.21.15.7:0/21949235 >> v1:172.21.15.171:6789/0 conn(0x7f7ac80b57e0 legacy=0x7f7ac80b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:49.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.913+0000 7f7ac7fff700 1 -- 172.21.15.7:0/21949235 >> v1:172.21.15.7:6789/0 conn(0x7f7ac80cb150 legacy=0x7f7ac80cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:49.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.913+0000 7f7ac7fff700 1 -- 172.21.15.7:0/21949235 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac80d1330 con 0x7f7ac80d0a10 2024-09-13T18:52:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.913+0000 7f7ac7fff700 1 --2- 172.21.15.7:0/21949235 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ac80d0a10 0x7f7ac80d0df0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7abc0098b0 tx=0x7f7abc009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c8acd8a674f011e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.913+0000 7f7ac6ffd700 1 -- 172.21.15.7:0/21949235 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc011830 con 0x7f7ac80d0a10 2024-09-13T18:52:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.913+0000 7f7ac6ffd700 1 -- 172.21.15.7:0/21949235 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7abc011990 con 0x7f7ac80d0a10 2024-09-13T18:52:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.913+0000 7f7ac6ffd700 1 -- 172.21.15.7:0/21949235 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc011d00 con 0x7f7ac80d0a10 2024-09-13T18:52:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7acd192700 1 -- 172.21.15.7:0/21949235 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ac80d0a10 msgr2=0x7f7ac80d0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7acd192700 1 --2- 172.21.15.7:0/21949235 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ac80d0a10 0x7f7ac80d0df0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7abc0098b0 tx=0x7f7abc009bc0 comp rx=0 tx=0).stop 2024-09-13T18:52:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7acd192700 1 -- 172.21.15.7:0/21949235 shutdown_connections 2024-09-13T18:52:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7acd192700 1 --2- 172.21.15.7:0/21949235 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ac80d0a10 0x7f7ac80d0df0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7acd192700 1 -- 172.21.15.7:0/21949235 >> 172.21.15.7:0/21949235 conn(0x7f7ac8071ee0 msgr2=0x7f7ac80722c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7acd192700 1 -- 172.21.15.7:0/21949235 shutdown_connections 2024-09-13T18:52:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7acd192700 1 -- 172.21.15.7:0/21949235 wait complete. 2024-09-13T18:52:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7acd192700 1 Processor -- start 2024-09-13T18:52:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7acd192700 1 -- start start 2024-09-13T18:52:49.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7acd192700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ac80b57e0 0x7f7ac8140420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:49.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7acd192700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ac80cb150 0x7f7ac81f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:49.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7acd192700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ac80d0a10 0x7f7ac80ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:49.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7acd192700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac814c810 con 0x7f7ac80cb150 2024-09-13T18:52:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7acd192700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac814c220 con 0x7f7ac80d0a10 2024-09-13T18:52:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7acd192700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac81f4c30 con 0x7f7ac80b57e0 2024-09-13T18:52:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7ac77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ac80cb150 0x7f7ac81f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7ac77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ac80cb150 0x7f7ac81f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46114/0 (socket says 172.21.15.7:46114) 2024-09-13T18:52:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7ac77fe700 1 -- 172.21.15.7:0/3603803863 learned_addr learned my addr 172.21.15.7:0/3603803863 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7ac7fff700 1 --2- 172.21.15.7:0/3603803863 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ac80b57e0 0x7f7ac8140420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.917+0000 7f7acc991700 1 --2- 172.21.15.7:0/3603803863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ac80d0a10 0x7f7ac80ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.921+0000 7f7ac77fe700 1 -- 172.21.15.7:0/3603803863 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ac80b57e0 msgr2=0x7f7ac8140420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.921+0000 7f7ac77fe700 1 --2- 172.21.15.7:0/3603803863 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ac80b57e0 0x7f7ac8140420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.921+0000 7f7ac77fe700 1 -- 172.21.15.7:0/3603803863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ac80d0a10 msgr2=0x7f7ac80ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.921+0000 7f7ac77fe700 1 --2- 172.21.15.7:0/3603803863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ac80d0a10 0x7f7ac80ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:49.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.921+0000 7f7ac77fe700 1 -- 172.21.15.7:0/3603803863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7abc0094b0 con 0x7f7ac80cb150 2024-09-13T18:52:49.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.921+0000 7f7acc991700 1 --2- 172.21.15.7:0/3603803863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ac80d0a10 0x7f7ac80ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:49.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.921+0000 7f7ac77fe700 1 --2- 172.21.15.7:0/3603803863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ac80cb150 0x7f7ac81f41e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800c520 tx=0x7f7ab800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:49.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.921+0000 7f7ac4ff9700 1 -- 172.21.15.7:0/3603803863 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8005d90 con 0x7f7ac80cb150 2024-09-13T18:52:49.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.921+0000 7f7ac4ff9700 1 -- 172.21.15.7:0/3603803863 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab80070d0 con 0x7f7ac80cb150 2024-09-13T18:52:49.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.921+0000 7f7ac4ff9700 1 -- 172.21.15.7:0/3603803863 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8007440 con 0x7f7ac80cb150 2024-09-13T18:52:49.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.921+0000 7f7acd192700 1 -- 172.21.15.7:0/3603803863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac81f4f10 con 0x7f7ac80cb150 2024-09-13T18:52:49.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.921+0000 7f7acd192700 1 -- 172.21.15.7:0/3603803863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac814a940 con 0x7f7ac80cb150 2024-09-13T18:52:49.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.921+0000 7f7ac4ff9700 1 -- 172.21.15.7:0/3603803863 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7ab80075a0 con 0x7f7ac80cb150 2024-09-13T18:52:49.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.921+0000 7f7acd192700 1 -- 172.21.15.7:0/3603803863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac80d0df0 con 0x7f7ac80cb150 2024-09-13T18:52:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.925+0000 7f7ac4ff9700 1 --2- 172.21.15.7:0/3603803863 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7ab4066730 0x7f7ab4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.925+0000 7f7ac4ff9700 1 -- 172.21.15.7:0/3603803863 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ab8083540 con 0x7f7ac80cb150 2024-09-13T18:52:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.925+0000 7f7ac7fff700 1 --2- 172.21.15.7:0/3603803863 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7ab4066730 0x7f7ab4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.925+0000 7f7ac7fff700 1 --2- 172.21.15.7:0/3603803863 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7ab4066730 0x7f7ab4068bf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7abc0090b0 tx=0x7f7abc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:49.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:49.933+0000 7f7ac4ff9700 1 -- 172.21.15.7:0/3603803863 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7ab80508e0 con 0x7f7ac80cb150 2024-09-13T18:52:50.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.105+0000 7f7acd192700 1 -- 172.21.15.7:0/3603803863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ac80cde00 con 0x7f7ac80cb150 2024-09-13T18:52:50.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.109+0000 7f7ac4ff9700 1 -- 172.21.15.7:0/3603803863 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f7ab8053f00 con 0x7f7ac80cb150 2024-09-13T18:52:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:50.114 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:50.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.109+0000 7f7acd192700 1 -- 172.21.15.7:0/3603803863 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7ab4066730 msgr2=0x7f7ab4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.109+0000 7f7acd192700 1 --2- 172.21.15.7:0/3603803863 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7ab4066730 0x7f7ab4068bf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7abc0090b0 tx=0x7f7abc008030 comp rx=0 tx=0).stop 2024-09-13T18:52:50.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.109+0000 7f7acd192700 1 -- 172.21.15.7:0/3603803863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ac80cb150 msgr2=0x7f7ac81f41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.109+0000 7f7acd192700 1 --2- 172.21.15.7:0/3603803863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ac80cb150 0x7f7ac81f41e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800c520 tx=0x7f7ab800c830 comp rx=0 tx=0).stop 2024-09-13T18:52:50.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.109+0000 7f7acd192700 1 -- 172.21.15.7:0/3603803863 shutdown_connections 2024-09-13T18:52:50.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.109+0000 7f7acd192700 1 --2- 172.21.15.7:0/3603803863 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7ab4066730 0x7f7ab4068bf0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.109+0000 7f7acd192700 1 --2- 172.21.15.7:0/3603803863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7ac80d0a10 0x7f7ac80ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.109+0000 7f7acd192700 1 --2- 172.21.15.7:0/3603803863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ac80cb150 0x7f7ac81f41e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.109+0000 7f7acd192700 1 --2- 172.21.15.7:0/3603803863 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7ac80b57e0 0x7f7ac8140420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.109+0000 7f7acd192700 1 -- 172.21.15.7:0/3603803863 >> 172.21.15.7:0/3603803863 conn(0x7f7ac8071ee0 msgr2=0x7f7ac80cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:50.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.109+0000 7f7acd192700 1 -- 172.21.15.7:0/3603803863 shutdown_connections 2024-09-13T18:52:50.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.109+0000 7f7acd192700 1 -- 172.21.15.7:0/3603803863 wait complete. 2024-09-13T18:52:50.134 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:52:15.554683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:52:16.896879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:52:50.134 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:52:50.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb4559a5700 1 Processor -- start 2024-09-13T18:52:50.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb4559a5700 1 -- start start 2024-09-13T18:52:50.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb4559a5700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb45006a520 0x7fb45006a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb4559a5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4500b8430 con 0x7fb45006b900 2024-09-13T18:52:50.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb4559a5700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45006ae90 con 0x7fb45015cda0 2024-09-13T18:52:50.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb4559a5700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4500b64f0 con 0x7fb45006a520 2024-09-13T18:52:50.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb44ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb45006b900 0x7fb4500b5de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57888/0 (socket says 172.21.15.7:57888) 2024-09-13T18:52:50.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb44ffff700 1 -- 172.21.15.7:0/1679480969 learned_addr learned my addr 172.21.15.7:0/1679480969 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:50.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb4551a4700 1 --2- 172.21.15.7:0/1679480969 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb45006a520 0x7fb45006a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb44f7fe700 1 -- 172.21.15.7:0/1679480969 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378311399 0 0) 0x7fb4500b8430 con 0x7fb45006b900 2024-09-13T18:52:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb44f7fe700 1 -- 172.21.15.7:0/1679480969 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4400035d0 con 0x7fb45006b900 2024-09-13T18:52:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb44f7fe700 1 -- 172.21.15.7:0/1679480969 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113209303 0 0) 0x7fb45006ae90 con 0x7fb45015cda0 2024-09-13T18:52:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb44f7fe700 1 -- 172.21.15.7:0/1679480969 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4500b8430 con 0x7fb45015cda0 2024-09-13T18:52:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb4551a4700 1 -- 172.21.15.7:0/1679480969 >> v1:172.21.15.171:6789/0 conn(0x7fb45015cda0 legacy=0x7fb4500ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb4551a4700 1 -- 172.21.15.7:0/1679480969 >> v1:172.21.15.7:6789/0 conn(0x7fb45006b900 legacy=0x7fb4500b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb4551a4700 1 -- 172.21.15.7:0/1679480969 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45006bd60 con 0x7fb45006a520 2024-09-13T18:52:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb4551a4700 1 --2- 172.21.15.7:0/1679480969 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb45006a520 0x7fb45006a950 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb448009dc0 tx=0x7fb44800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c606cdb6b94184c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb44f7fe700 1 -- 172.21.15.7:0/1679480969 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44800e070 con 0x7fb45006a520 2024-09-13T18:52:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb44f7fe700 1 -- 172.21.15.7:0/1679480969 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44800bb20 con 0x7fb45006a520 2024-09-13T18:52:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.253+0000 7fb44f7fe700 1 -- 172.21.15.7:0/1679480969 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44800be90 con 0x7fb45006a520 2024-09-13T18:52:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.257+0000 7fb4559a5700 1 -- 172.21.15.7:0/1679480969 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb45006a520 msgr2=0x7fb45006a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.257+0000 7fb4559a5700 1 --2- 172.21.15.7:0/1679480969 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb45006a520 0x7fb45006a950 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb448009dc0 tx=0x7fb44800b330 comp rx=0 tx=0).stop 2024-09-13T18:52:50.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.257+0000 7fb4559a5700 1 -- 172.21.15.7:0/1679480969 shutdown_connections 2024-09-13T18:52:50.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.257+0000 7fb4559a5700 1 --2- 172.21.15.7:0/1679480969 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb45006a520 0x7fb45006a950 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.257+0000 7fb4559a5700 1 -- 172.21.15.7:0/1679480969 >> 172.21.15.7:0/1679480969 conn(0x7fb4500d2cb0 msgr2=0x7fb4500d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:50.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.257+0000 7fb4559a5700 1 -- 172.21.15.7:0/1679480969 shutdown_connections 2024-09-13T18:52:50.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.257+0000 7fb4559a5700 1 -- 172.21.15.7:0/1679480969 wait complete. 2024-09-13T18:52:50.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.257+0000 7fb4559a5700 1 Processor -- start 2024-09-13T18:52:50.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.257+0000 7fb4559a5700 1 -- start start 2024-09-13T18:52:50.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4559a5700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb45006a520 0x7fb4501f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4559a5700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb45006b900 0x7fb4501f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4559a5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb45015cda0 0x7fb4501f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4559a5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4500c71b0 con 0x7fb45015cda0 2024-09-13T18:52:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4559a5700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4501f3380 con 0x7fb45006b900 2024-09-13T18:52:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4559a5700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4500c6880 con 0x7fb45006a520 2024-09-13T18:52:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4551a4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb45015cda0 0x7fb4501f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4551a4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb45015cda0 0x7fb4501f2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46122/0 (socket says 172.21.15.7:46122) 2024-09-13T18:52:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4551a4700 1 -- 172.21.15.7:0/789397339 learned_addr learned my addr 172.21.15.7:0/789397339 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb44ffff700 1 --2- 172.21.15.7:0/789397339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb45006b900 0x7fb4501f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4549a3700 1 --2- 172.21.15.7:0/789397339 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb45006a520 0x7fb4501f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4551a4700 1 -- 172.21.15.7:0/789397339 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb45006a520 msgr2=0x7fb4501f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4551a4700 1 --2- 172.21.15.7:0/789397339 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb45006a520 0x7fb4501f5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4551a4700 1 -- 172.21.15.7:0/789397339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb45006b900 msgr2=0x7fb4501f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4551a4700 1 --2- 172.21.15.7:0/789397339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb45006b900 0x7fb4501f3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4551a4700 1 -- 172.21.15.7:0/789397339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb448009a70 con 0x7fb45015cda0 2024-09-13T18:52:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4551a4700 1 --2- 172.21.15.7:0/789397339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb45015cda0 0x7fb4501f2e40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb448013000 tx=0x7fb448004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb44d7fa700 1 -- 172.21.15.7:0/789397339 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4480098f0 con 0x7fb45015cda0 2024-09-13T18:52:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb44d7fa700 1 -- 172.21.15.7:0/789397339 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44801f070 con 0x7fb45015cda0 2024-09-13T18:52:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb44d7fa700 1 -- 172.21.15.7:0/789397339 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448003c80 con 0x7fb45015cda0 2024-09-13T18:52:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4559a5700 1 -- 172.21.15.7:0/789397339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4500c6b60 con 0x7fb45015cda0 2024-09-13T18:52:50.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4559a5700 1 -- 172.21.15.7:0/789397339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4500c9170 con 0x7fb45015cda0 2024-09-13T18:52:50.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.261+0000 7fb4559a5700 1 -- 172.21.15.7:0/789397339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4500c7360 con 0x7fb45015cda0 2024-09-13T18:52:50.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.265+0000 7fb44d7fa700 1 -- 172.21.15.7:0/789397339 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb448004030 con 0x7fb45015cda0 2024-09-13T18:52:50.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.265+0000 7fb44d7fa700 1 --2- 172.21.15.7:0/789397339 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb434066730 0x7fb434068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.265+0000 7fb44d7fa700 1 -- 172.21.15.7:0/789397339 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4480857d0 con 0x7fb45015cda0 2024-09-13T18:52:50.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.265+0000 7fb4549a3700 1 --2- 172.21.15.7:0/789397339 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb434066730 0x7fb434068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.265+0000 7fb4549a3700 1 --2- 172.21.15.7:0/789397339 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb434066730 0x7fb434068bf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb444000f60 tx=0x7fb444009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:50.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.269+0000 7fb44d7fa700 1 -- 172.21.15.7:0/789397339 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb4500c7360 con 0x7fb45015cda0 2024-09-13T18:52:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.441+0000 7fb4559a5700 1 -- 172.21.15.7:0/789397339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4500c7360 con 0x7fb45015cda0 2024-09-13T18:52:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.445+0000 7fb44d7fa700 1 -- 172.21.15.7:0/789397339 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb448021020 con 0x7fb45015cda0 2024-09-13T18:52:50.450 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:52:50.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.445+0000 7fb4559a5700 1 -- 172.21.15.7:0/789397339 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb434066730 msgr2=0x7fb434068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.445+0000 7fb4559a5700 1 --2- 172.21.15.7:0/789397339 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb434066730 0x7fb434068bf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb444000f60 tx=0x7fb444009040 comp rx=0 tx=0).stop 2024-09-13T18:52:50.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.445+0000 7fb4559a5700 1 -- 172.21.15.7:0/789397339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb45015cda0 msgr2=0x7fb4501f2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.445+0000 7fb4559a5700 1 --2- 172.21.15.7:0/789397339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb45015cda0 0x7fb4501f2e40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb448013000 tx=0x7fb448004af0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.445+0000 7fb4559a5700 1 -- 172.21.15.7:0/789397339 shutdown_connections 2024-09-13T18:52:50.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.445+0000 7fb4559a5700 1 --2- 172.21.15.7:0/789397339 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb434066730 0x7fb434068bf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.445+0000 7fb4559a5700 1 --2- 172.21.15.7:0/789397339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb45015cda0 0x7fb4501f2e40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.445+0000 7fb4559a5700 1 --2- 172.21.15.7:0/789397339 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb45006b900 0x7fb4501f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.445+0000 7fb4559a5700 1 --2- 172.21.15.7:0/789397339 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb45006a520 0x7fb4501f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.445+0000 7fb4559a5700 1 -- 172.21.15.7:0/789397339 >> 172.21.15.7:0/789397339 conn(0x7fb4500d2cb0 msgr2=0x7fb4500b5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:50.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.445+0000 7fb4559a5700 1 -- 172.21.15.7:0/789397339 shutdown_connections 2024-09-13T18:52:50.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.445+0000 7fb4559a5700 1 -- 172.21.15.7:0/789397339 wait complete. 2024-09-13T18:52:50.465 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:52:50.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.581+0000 7f5429e60700 1 Processor -- start 2024-09-13T18:52:50.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.581+0000 7f5429e60700 1 -- start start 2024-09-13T18:52:50.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.581+0000 7f5429e60700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5424144340 0x7f542406d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.581+0000 7f5429e60700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5424144720 con 0x7f542406bdb0 2024-09-13T18:52:50.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.581+0000 7f5429e60700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542406dd40 con 0x7f54240cdc90 2024-09-13T18:52:50.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.581+0000 7f5429e60700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542406da60 con 0x7f5424144340 2024-09-13T18:52:50.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.581+0000 7f5428e5e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5424144340 0x7f542406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.581+0000 7f541bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f542406bdb0 0x7f54240cd580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57894/0 (socket says 172.21.15.7:57894) 2024-09-13T18:52:50.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.581+0000 7f541bfff700 1 -- 172.21.15.7:0/3003758136 learned_addr learned my addr 172.21.15.7:0/3003758136 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:50.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.581+0000 7f5428e5e700 1 -- 172.21.15.7:0/3003758136 >> v1:172.21.15.171:6789/0 conn(0x7f54240cdc90 legacy=0x7f542406fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.581+0000 7f541b7fe700 1 -- 172.21.15.7:0/3003758136 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3999296489 0 0) 0x7f542406dd40 con 0x7f54240cdc90 2024-09-13T18:52:50.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.581+0000 7f5428e5e700 1 -- 172.21.15.7:0/3003758136 >> v1:172.21.15.7:6789/0 conn(0x7f542406bdb0 legacy=0x7f54240cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.581+0000 7f5428e5e700 1 -- 172.21.15.7:0/3003758136 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542415e660 con 0x7f5424144340 2024-09-13T18:52:50.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.585+0000 7f5428e5e700 1 --2- 172.21.15.7:0/3003758136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5424144340 0x7f542406d480 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5420009850 tx=0x7f5420009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9592bb6e52561e69 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:50.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.585+0000 7f541b7fe700 1 -- 172.21.15.7:0/3003758136 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5420004230 con 0x7f5424144340 2024-09-13T18:52:50.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.585+0000 7f541b7fe700 1 -- 172.21.15.7:0/3003758136 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5420004390 con 0x7f5424144340 2024-09-13T18:52:50.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.585+0000 7f541b7fe700 1 -- 172.21.15.7:0/3003758136 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54200036c0 con 0x7f5424144340 2024-09-13T18:52:50.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.585+0000 7f5429e60700 1 -- 172.21.15.7:0/3003758136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5424144340 msgr2=0x7f542406d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.585+0000 7f5429e60700 1 --2- 172.21.15.7:0/3003758136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5424144340 0x7f542406d480 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5420009850 tx=0x7f5420009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:50.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.585+0000 7f5429e60700 1 -- 172.21.15.7:0/3003758136 shutdown_connections 2024-09-13T18:52:50.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.585+0000 7f5429e60700 1 --2- 172.21.15.7:0/3003758136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5424144340 0x7f542406d480 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.585+0000 7f5429e60700 1 -- 172.21.15.7:0/3003758136 >> 172.21.15.7:0/3003758136 conn(0x7f54240b0e10 msgr2=0x7f54240b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:50.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.585+0000 7f5429e60700 1 -- 172.21.15.7:0/3003758136 shutdown_connections 2024-09-13T18:52:50.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.585+0000 7f5429e60700 1 -- 172.21.15.7:0/3003758136 wait complete. 2024-09-13T18:52:50.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.589+0000 7f5429e60700 1 Processor -- start 2024-09-13T18:52:50.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.589+0000 7f5429e60700 1 -- start start 2024-09-13T18:52:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.589+0000 7f5429e60700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f542406bdb0 0x7f54240c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.589+0000 7f5429e60700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54240cdc90 0x7f54240c38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.589+0000 7f5429e60700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5424144340 0x7f54241f1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.589+0000 7f5429e60700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54240c3e10 con 0x7f5424144340 2024-09-13T18:52:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.589+0000 7f5429e60700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54240c4040 con 0x7f54240cdc90 2024-09-13T18:52:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.589+0000 7f5429e60700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54240ab640 con 0x7f542406bdb0 2024-09-13T18:52:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.589+0000 7f542965f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5424144340 0x7f54241f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.589+0000 7f541bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54240cdc90 0x7f54240c38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.589+0000 7f5428e5e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f542406bdb0 0x7f54240c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.589+0000 7f542965f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5424144340 0x7f54241f1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46146/0 (socket says 172.21.15.7:46146) 2024-09-13T18:52:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.589+0000 7f542965f700 1 -- 172.21.15.7:0/2369511677 learned_addr learned my addr 172.21.15.7:0/2369511677 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.593+0000 7f542965f700 1 -- 172.21.15.7:0/2369511677 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f542406bdb0 msgr2=0x7f54240c4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.593+0000 7f542965f700 1 --2- 172.21.15.7:0/2369511677 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f542406bdb0 0x7f54240c4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.593+0000 7f542965f700 1 -- 172.21.15.7:0/2369511677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54240cdc90 msgr2=0x7f54240c38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.593+0000 7f542965f700 1 --2- 172.21.15.7:0/2369511677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54240cdc90 0x7f54240c38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.593+0000 7f542965f700 1 -- 172.21.15.7:0/2369511677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54200094b0 con 0x7f5424144340 2024-09-13T18:52:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.593+0000 7f5428e5e700 1 --2- 172.21.15.7:0/2369511677 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f542406bdb0 0x7f54240c4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.593+0000 7f541bfff700 1 --2- 172.21.15.7:0/2369511677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54240cdc90 0x7f54240c38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.593+0000 7f542965f700 1 --2- 172.21.15.7:0/2369511677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5424144340 0x7f54241f1580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f541c00cc40 tx=0x7f541c00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.593+0000 7f54197fa700 1 -- 172.21.15.7:0/2369511677 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c003e50 con 0x7f5424144340 2024-09-13T18:52:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.593+0000 7f5429e60700 1 -- 172.21.15.7:0/2369511677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54240ab920 con 0x7f5424144340 2024-09-13T18:52:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.593+0000 7f5429e60700 1 -- 172.21.15.7:0/2369511677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54240abee0 con 0x7f5424144340 2024-09-13T18:52:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.593+0000 7f54197fa700 1 -- 172.21.15.7:0/2369511677 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c00a6b0 con 0x7f5424144340 2024-09-13T18:52:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.593+0000 7f54197fa700 1 -- 172.21.15.7:0/2369511677 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c00aa60 con 0x7f5424144340 2024-09-13T18:52:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.593+0000 7f5429e60700 1 -- 172.21.15.7:0/2369511677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5404005310 con 0x7f5424144340 2024-09-13T18:52:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.597+0000 7f54197fa700 1 -- 172.21.15.7:0/2369511677 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f541c00abc0 con 0x7f5424144340 2024-09-13T18:52:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.597+0000 7f54197fa700 1 --2- 172.21.15.7:0/2369511677 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5410066730 0x7f5410068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.597+0000 7f54197fa700 1 -- 172.21.15.7:0/2369511677 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f541c083f60 con 0x7f5424144340 2024-09-13T18:52:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.597+0000 7f5428e5e700 1 --2- 172.21.15.7:0/2369511677 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5410066730 0x7f5410068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.597+0000 7f5428e5e700 1 --2- 172.21.15.7:0/2369511677 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5410066730 0x7f5410068bf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5420009850 tx=0x7f5420010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:50.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.601+0000 7f54197fa700 1 -- 172.21.15.7:0/2369511677 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f541c051340 con 0x7f5424144340 2024-09-13T18:52:50.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.765+0000 7f5429e60700 1 -- 172.21.15.7:0/2369511677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5404005b40 con 0x7f5424144340 2024-09-13T18:52:50.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.769+0000 7f54197fa700 1 -- 172.21.15.7:0/2369511677 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f541c001cc0 con 0x7f5424144340 2024-09-13T18:52:50.774 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:52:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.769+0000 7f5429e60700 1 -- 172.21.15.7:0/2369511677 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5410066730 msgr2=0x7f5410068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.769+0000 7f5429e60700 1 --2- 172.21.15.7:0/2369511677 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5410066730 0x7f5410068bf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5420009850 tx=0x7f5420010000 comp rx=0 tx=0).stop 2024-09-13T18:52:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.769+0000 7f5429e60700 1 -- 172.21.15.7:0/2369511677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5424144340 msgr2=0x7f54241f1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.769+0000 7f5429e60700 1 --2- 172.21.15.7:0/2369511677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5424144340 0x7f54241f1580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f541c00cc40 tx=0x7f541c00cf50 comp rx=0 tx=0).stop 2024-09-13T18:52:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.769+0000 7f5429e60700 1 -- 172.21.15.7:0/2369511677 shutdown_connections 2024-09-13T18:52:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.769+0000 7f5429e60700 1 --2- 172.21.15.7:0/2369511677 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5410066730 0x7f5410068bf0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.769+0000 7f5429e60700 1 --2- 172.21.15.7:0/2369511677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5424144340 0x7f54241f1580 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.769+0000 7f5429e60700 1 --2- 172.21.15.7:0/2369511677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54240cdc90 0x7f54240c38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.769+0000 7f5429e60700 1 --2- 172.21.15.7:0/2369511677 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f542406bdb0 0x7f54240c4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.769+0000 7f5429e60700 1 -- 172.21.15.7:0/2369511677 >> 172.21.15.7:0/2369511677 conn(0x7f54240b0e10 msgr2=0x7f54240b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.769+0000 7f5429e60700 1 -- 172.21.15.7:0/2369511677 shutdown_connections 2024-09-13T18:52:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.769+0000 7f5429e60700 1 -- 172.21.15.7:0/2369511677 wait complete. 2024-09-13T18:52:50.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:52:50.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.909+0000 7f659af1f700 1 Processor -- start 2024-09-13T18:52:50.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.909+0000 7f659af1f700 1 -- start start 2024-09-13T18:52:50.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.909+0000 7f659af1f700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f659406b2b0 0x7f65940cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.909+0000 7f659af1f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f659415ad10 con 0x7f659415a930 2024-09-13T18:52:50.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.909+0000 7f659af1f700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65940718e0 con 0x7f65940cd420 2024-09-13T18:52:50.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.909+0000 7f659af1f700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65940cd0a0 con 0x7f659406b2b0 2024-09-13T18:52:50.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.909+0000 7f6599f1d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f659415a930 0x7f6594071d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57908/0 (socket says 172.21.15.7:57908) 2024-09-13T18:52:50.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.909+0000 7f6599f1d700 1 -- 172.21.15.7:0/1312641940 learned_addr learned my addr 172.21.15.7:0/1312641940 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:50.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.909+0000 7f659a71e700 1 --2- 172.21.15.7:0/1312641940 >> v2:172.21.15.171:3300/0 conn(0x7f659406b2b0 0x7f65940cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.909+0000 7f6598f1b700 1 -- 172.21.15.7:0/1312641940 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240331171 0 0) 0x7f659415ad10 con 0x7f659415a930 2024-09-13T18:52:50.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.909+0000 7f6598f1b700 1 -- 172.21.15.7:0/1312641940 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65800035d0 con 0x7f659415a930 2024-09-13T18:52:50.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f6598f1b700 1 -- 172.21.15.7:0/1312641940 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226761691 0 0) 0x7f65940718e0 con 0x7f65940cd420 2024-09-13T18:52:50.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f6598f1b700 1 -- 172.21.15.7:0/1312641940 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f659415ad10 con 0x7f65940cd420 2024-09-13T18:52:50.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659a71e700 1 -- 172.21.15.7:0/1312641940 >> v1:172.21.15.171:6789/0 conn(0x7f65940cd420 legacy=0x7f659406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659a71e700 1 -- 172.21.15.7:0/1312641940 >> v1:172.21.15.7:6789/0 conn(0x7f659415a930 legacy=0x7f6594071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f6598f1b700 1 -- 172.21.15.7:0/1312641940 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 125766420 0 0) 0x7f65800035d0 con 0x7f659415a930 2024-09-13T18:52:50.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659a71e700 1 -- 172.21.15.7:0/1312641940 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6594070cd0 con 0x7f659406b2b0 2024-09-13T18:52:50.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659a71e700 1 --2- 172.21.15.7:0/1312641940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f659406b2b0 0x7f65940cd890 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f658c009d90 tx=0x7f658c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c19600d3b7cbceb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:50.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f6598f1b700 1 -- 172.21.15.7:0/1312641940 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658c00e070 con 0x7f659406b2b0 2024-09-13T18:52:50.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f6598f1b700 1 -- 172.21.15.7:0/1312641940 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658c003a40 con 0x7f659406b2b0 2024-09-13T18:52:50.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f6598f1b700 1 -- 172.21.15.7:0/1312641940 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590001670 con 0x7f659406b2b0 2024-09-13T18:52:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659af1f700 1 -- 172.21.15.7:0/1312641940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f659406b2b0 msgr2=0x7f65940cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659af1f700 1 --2- 172.21.15.7:0/1312641940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f659406b2b0 0x7f65940cd890 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f658c009d90 tx=0x7f658c00b330 comp rx=0 tx=0).stop 2024-09-13T18:52:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659af1f700 1 -- 172.21.15.7:0/1312641940 shutdown_connections 2024-09-13T18:52:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659af1f700 1 --2- 172.21.15.7:0/1312641940 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f659406b2b0 0x7f65940cd890 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659af1f700 1 -- 172.21.15.7:0/1312641940 >> 172.21.15.7:0/1312641940 conn(0x7f65940b1810 msgr2=0x7f65940b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659af1f700 1 -- 172.21.15.7:0/1312641940 shutdown_connections 2024-09-13T18:52:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659af1f700 1 -- 172.21.15.7:0/1312641940 wait complete. 2024-09-13T18:52:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659af1f700 1 Processor -- start 2024-09-13T18:52:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659af1f700 1 -- start start 2024-09-13T18:52:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659af1f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f659406b2b0 0x7f6594153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659af1f700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65940cd420 0x7f65940cf260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659af1f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f659415a930 0x7f65940cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659af1f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65941f24c0 con 0x7f659406b2b0 2024-09-13T18:52:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659af1f700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65941f6770 con 0x7f659415a930 2024-09-13T18:52:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f659af1f700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65941f26d0 con 0x7f65940cd420 2024-09-13T18:52:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.913+0000 7f6599f1d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f659406b2b0 0x7f6594153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.917+0000 7f6599f1d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f659406b2b0 0x7f6594153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46160/0 (socket says 172.21.15.7:46160) 2024-09-13T18:52:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.917+0000 7f6599f1d700 1 -- 172.21.15.7:0/1527239309 learned_addr learned my addr 172.21.15.7:0/1527239309 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.917+0000 7f659a71e700 1 --2- 172.21.15.7:0/1527239309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f659415a930 0x7f65940cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.917+0000 7f659971c700 1 --2- 172.21.15.7:0/1527239309 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65940cd420 0x7f65940cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.917+0000 7f6599f1d700 1 -- 172.21.15.7:0/1527239309 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65940cd420 msgr2=0x7f65940cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.917+0000 7f6599f1d700 1 --2- 172.21.15.7:0/1527239309 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65940cd420 0x7f65940cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.917+0000 7f6599f1d700 1 -- 172.21.15.7:0/1527239309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f659415a930 msgr2=0x7f65940cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.917+0000 7f6599f1d700 1 --2- 172.21.15.7:0/1527239309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f659415a930 0x7f65940cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.917+0000 7f6599f1d700 1 -- 172.21.15.7:0/1527239309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658c009a70 con 0x7f659406b2b0 2024-09-13T18:52:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.917+0000 7f6599f1d700 1 --2- 172.21.15.7:0/1527239309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f659406b2b0 0x7f6594153b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f659000c750 tx=0x7f659000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.917+0000 7f658a7fc700 1 -- 172.21.15.7:0/1527239309 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590004ca0 con 0x7f659406b2b0 2024-09-13T18:52:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.917+0000 7f658a7fc700 1 -- 172.21.15.7:0/1527239309 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6590004e00 con 0x7f659406b2b0 2024-09-13T18:52:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.917+0000 7f658a7fc700 1 -- 172.21.15.7:0/1527239309 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65900072d0 con 0x7f659406b2b0 2024-09-13T18:52:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.917+0000 7f659af1f700 1 -- 172.21.15.7:0/1527239309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65941f29b0 con 0x7f659406b2b0 2024-09-13T18:52:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.917+0000 7f659af1f700 1 -- 172.21.15.7:0/1527239309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65941f31b0 con 0x7f659406b2b0 2024-09-13T18:52:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.917+0000 7f659af1f700 1 -- 172.21.15.7:0/1527239309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65941f2ed0 con 0x7f659406b2b0 2024-09-13T18:52:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.921+0000 7f658a7fc700 1 -- 172.21.15.7:0/1527239309 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6590007430 con 0x7f659406b2b0 2024-09-13T18:52:50.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.921+0000 7f658a7fc700 1 --2- 172.21.15.7:0/1527239309 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6578066730 0x7f6578068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:50.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.921+0000 7f658a7fc700 1 -- 172.21.15.7:0/1527239309 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6590083f10 con 0x7f659406b2b0 2024-09-13T18:52:50.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.921+0000 7f659971c700 1 --2- 172.21.15.7:0/1527239309 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6578066730 0x7f6578068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:50.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.921+0000 7f659971c700 1 --2- 172.21.15.7:0/1527239309 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6578066730 0x7f6578068bf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f659415cd10 tx=0x7f6584009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:50.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:50.925+0000 7f658a7fc700 1 -- 172.21.15.7:0/1527239309 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f65900511e0 con 0x7f659406b2b0 2024-09-13T18:52:51.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.093+0000 7f659af1f700 1 -- 172.21.15.7:0/1527239309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65940cd890 con 0x7f659406b2b0 2024-09-13T18:52:51.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.093+0000 7f658a7fc700 1 -- 172.21.15.7:0/1527239309 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6590054800 con 0x7f659406b2b0 2024-09-13T18:52:51.098 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:52:51.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.093+0000 7f659af1f700 1 -- 172.21.15.7:0/1527239309 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6578066730 msgr2=0x7f6578068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.093+0000 7f659af1f700 1 --2- 172.21.15.7:0/1527239309 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6578066730 0x7f6578068bf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f659415cd10 tx=0x7f6584009040 comp rx=0 tx=0).stop 2024-09-13T18:52:51.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.093+0000 7f659af1f700 1 -- 172.21.15.7:0/1527239309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f659406b2b0 msgr2=0x7f6594153b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.093+0000 7f659af1f700 1 --2- 172.21.15.7:0/1527239309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f659406b2b0 0x7f6594153b00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f659000c750 tx=0x7f659000ca60 comp rx=0 tx=0).stop 2024-09-13T18:52:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.093+0000 7f659af1f700 1 -- 172.21.15.7:0/1527239309 shutdown_connections 2024-09-13T18:52:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.093+0000 7f659af1f700 1 --2- 172.21.15.7:0/1527239309 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6578066730 0x7f6578068bf0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.093+0000 7f659af1f700 1 --2- 172.21.15.7:0/1527239309 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f659415a930 0x7f65940cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.093+0000 7f659af1f700 1 --2- 172.21.15.7:0/1527239309 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65940cd420 0x7f65940cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.093+0000 7f659af1f700 1 --2- 172.21.15.7:0/1527239309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f659406b2b0 0x7f6594153b00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.093+0000 7f659af1f700 1 -- 172.21.15.7:0/1527239309 >> 172.21.15.7:0/1527239309 conn(0x7f65940b1810 msgr2=0x7f6594071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.093+0000 7f659af1f700 1 -- 172.21.15.7:0/1527239309 shutdown_connections 2024-09-13T18:52:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.093+0000 7f659af1f700 1 -- 172.21.15.7:0/1527239309 wait complete. 2024-09-13T18:52:51.114 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:52:51.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.221+0000 7f982da5f700 1 Processor -- start 2024-09-13T18:52:51.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.221+0000 7f982da5f700 1 -- start start 2024-09-13T18:52:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f982da5f700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f982806d8c0 0x7f9828164a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f982da5f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9828162570 con 0x7f9828070bd0 2024-09-13T18:52:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f982da5f700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98281653f0 con 0x7f9828164f90 2024-09-13T18:52:51.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f982da5f700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9828162e60 con 0x7f982806d8c0 2024-09-13T18:52:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f982d25e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9828070bd0 0x7f9828163280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38732/0 (socket says 172.21.15.7:38732) 2024-09-13T18:52:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f982d25e700 1 -- 172.21.15.7:0/1003063536 learned_addr learned my addr 172.21.15.7:0/1003063536 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:51.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f982ca5d700 1 --2- 172.21.15.7:0/1003063536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f982806d8c0 0x7f9828164a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:51.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f98277fe700 1 -- 172.21.15.7:0/1003063536 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344557008 0 0) 0x7f9828162570 con 0x7f9828070bd0 2024-09-13T18:52:51.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f98277fe700 1 -- 172.21.15.7:0/1003063536 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98100035d0 con 0x7f9828070bd0 2024-09-13T18:52:51.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f98277fe700 1 -- 172.21.15.7:0/1003063536 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1066301759 0 0) 0x7f98281653f0 con 0x7f9828164f90 2024-09-13T18:52:51.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f98277fe700 1 -- 172.21.15.7:0/1003063536 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9828162570 con 0x7f9828164f90 2024-09-13T18:52:51.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f982ca5d700 1 -- 172.21.15.7:0/1003063536 >> v1:172.21.15.171:6789/0 conn(0x7f9828164f90 legacy=0x7f98280704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f982ca5d700 1 -- 172.21.15.7:0/1003063536 >> v1:172.21.15.7:6789/0 conn(0x7f9828070bd0 legacy=0x7f9828163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f982ca5d700 1 -- 172.21.15.7:0/1003063536 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9828071240 con 0x7f982806d8c0 2024-09-13T18:52:51.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f982ca5d700 1 --2- 172.21.15.7:0/1003063536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f982806d8c0 0x7f9828164a50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f981c009880 tx=0x7f981c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fd5a8b5891a561b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:51.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f98277fe700 1 -- 172.21.15.7:0/1003063536 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c011830 con 0x7f982806d8c0 2024-09-13T18:52:51.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f98277fe700 1 -- 172.21.15.7:0/1003063536 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981c011990 con 0x7f982806d8c0 2024-09-13T18:52:51.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f98277fe700 1 -- 172.21.15.7:0/1003063536 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c011d00 con 0x7f982806d8c0 2024-09-13T18:52:51.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f982da5f700 1 -- 172.21.15.7:0/1003063536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f982806d8c0 msgr2=0x7f9828164a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f982da5f700 1 --2- 172.21.15.7:0/1003063536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f982806d8c0 0x7f9828164a50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f981c009880 tx=0x7f981c009b90 comp rx=0 tx=0).stop 2024-09-13T18:52:51.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f982da5f700 1 -- 172.21.15.7:0/1003063536 shutdown_connections 2024-09-13T18:52:51.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.225+0000 7f982da5f700 1 --2- 172.21.15.7:0/1003063536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f982806d8c0 0x7f9828164a50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.229+0000 7f982da5f700 1 -- 172.21.15.7:0/1003063536 >> 172.21.15.7:0/1003063536 conn(0x7f98280b0e10 msgr2=0x7f98280b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:51.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.229+0000 7f982da5f700 1 -- 172.21.15.7:0/1003063536 shutdown_connections 2024-09-13T18:52:51.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.229+0000 7f982da5f700 1 -- 172.21.15.7:0/1003063536 wait complete. 2024-09-13T18:52:51.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.229+0000 7f982da5f700 1 Processor -- start 2024-09-13T18:52:51.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.229+0000 7f982da5f700 1 -- start start 2024-09-13T18:52:51.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.229+0000 7f982da5f700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f982806d8c0 0x7f982813f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:51.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.229+0000 7f982da5f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9828070bd0 0x7f982813fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.229+0000 7f982da5f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9828164f90 0x7f9828140120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.229+0000 7f982da5f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98281601b0 con 0x7f9828164f90 2024-09-13T18:52:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.229+0000 7f982da5f700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982815fb30 con 0x7f9828070bd0 2024-09-13T18:52:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.229+0000 7f982da5f700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98281588f0 con 0x7f982806d8c0 2024-09-13T18:52:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.229+0000 7f982d25e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9828164f90 0x7f9828140120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.229+0000 7f982d25e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9828164f90 0x7f9828140120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38946/0 (socket says 172.21.15.7:38946) 2024-09-13T18:52:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.229+0000 7f982d25e700 1 -- 172.21.15.7:0/3748037775 learned_addr learned my addr 172.21.15.7:0/3748037775 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.229+0000 7f982ca5d700 1 --2- 172.21.15.7:0/3748037775 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f982806d8c0 0x7f982813f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.233+0000 7f9827fff700 1 --2- 172.21.15.7:0/3748037775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9828070bd0 0x7f982813fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.233+0000 7f982d25e700 1 -- 172.21.15.7:0/3748037775 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f982806d8c0 msgr2=0x7f982813f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.233+0000 7f982d25e700 1 --2- 172.21.15.7:0/3748037775 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f982806d8c0 0x7f982813f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.233+0000 7f982d25e700 1 -- 172.21.15.7:0/3748037775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9828070bd0 msgr2=0x7f982813fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.233+0000 7f982d25e700 1 --2- 172.21.15.7:0/3748037775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9828070bd0 0x7f982813fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.233+0000 7f982d25e700 1 -- 172.21.15.7:0/3748037775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c0094b0 con 0x7f9828164f90 2024-09-13T18:52:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.233+0000 7f9827fff700 1 --2- 172.21.15.7:0/3748037775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9828070bd0 0x7f982813fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.233+0000 7f982d25e700 1 --2- 172.21.15.7:0/3748037775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9828164f90 0x7f9828140120 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f982000cba0 tx=0x7f982000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:51.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.233+0000 7f98257fa700 1 -- 172.21.15.7:0/3748037775 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982000a7b0 con 0x7f9828164f90 2024-09-13T18:52:51.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.233+0000 7f98257fa700 1 -- 172.21.15.7:0/3748037775 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982000a910 con 0x7f9828164f90 2024-09-13T18:52:51.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.233+0000 7f982da5f700 1 -- 172.21.15.7:0/3748037775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f982815fdd0 con 0x7f9828164f90 2024-09-13T18:52:51.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.233+0000 7f98257fa700 1 -- 172.21.15.7:0/3748037775 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982000ac80 con 0x7f9828164f90 2024-09-13T18:52:51.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.233+0000 7f982da5f700 1 -- 172.21.15.7:0/3748037775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9828158580 con 0x7f9828164f90 2024-09-13T18:52:51.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.233+0000 7f982da5f700 1 -- 172.21.15.7:0/3748037775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98281602f0 con 0x7f9828164f90 2024-09-13T18:52:51.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.237+0000 7f98257fa700 1 -- 172.21.15.7:0/3748037775 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f982001b3b0 con 0x7f9828164f90 2024-09-13T18:52:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.237+0000 7f98257fa700 1 --2- 172.21.15.7:0/3748037775 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9814066730 0x7f9814068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.237+0000 7f98257fa700 1 -- 172.21.15.7:0/3748037775 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98200839e0 con 0x7f9828164f90 2024-09-13T18:52:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.237+0000 7f982ca5d700 1 --2- 172.21.15.7:0/3748037775 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9814066730 0x7f9814068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:51.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.237+0000 7f982ca5d700 1 --2- 172.21.15.7:0/3748037775 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9814066730 0x7f9814068bf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f981c0090b0 tx=0x7f981c010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:51.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.241+0000 7f98257fa700 1 -- 172.21.15.7:0/3748037775 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f98281602f0 con 0x7f9828164f90 2024-09-13T18:52:51.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.425+0000 7f982da5f700 1 -- 172.21.15.7:0/3748037775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f98281602f0 con 0x7f9828164f90 2024-09-13T18:52:51.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.429+0000 7f98257fa700 1 -- 172.21.15.7:0/3748037775 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f98281602f0 con 0x7f9828164f90 2024-09-13T18:52:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.429+0000 7f982da5f700 1 -- 172.21.15.7:0/3748037775 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9814066730 msgr2=0x7f9814068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.429+0000 7f982da5f700 1 --2- 172.21.15.7:0/3748037775 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9814066730 0x7f9814068bf0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f981c0090b0 tx=0x7f981c010000 comp rx=0 tx=0).stop 2024-09-13T18:52:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.429+0000 7f982da5f700 1 -- 172.21.15.7:0/3748037775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9828164f90 msgr2=0x7f9828140120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.429+0000 7f982da5f700 1 --2- 172.21.15.7:0/3748037775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9828164f90 0x7f9828140120 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f982000cba0 tx=0x7f982000cbd0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.429+0000 7f982da5f700 1 -- 172.21.15.7:0/3748037775 shutdown_connections 2024-09-13T18:52:51.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.429+0000 7f982da5f700 1 --2- 172.21.15.7:0/3748037775 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9814066730 0x7f9814068bf0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.429+0000 7f982da5f700 1 --2- 172.21.15.7:0/3748037775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9828164f90 0x7f9828140120 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.429+0000 7f982da5f700 1 --2- 172.21.15.7:0/3748037775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9828070bd0 0x7f982813fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.429+0000 7f982da5f700 1 --2- 172.21.15.7:0/3748037775 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f982806d8c0 0x7f982813f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.429+0000 7f982da5f700 1 -- 172.21.15.7:0/3748037775 >> 172.21.15.7:0/3748037775 conn(0x7f98280b0e10 msgr2=0x7f98280b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:51.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.429+0000 7f982da5f700 1 -- 172.21.15.7:0/3748037775 shutdown_connections 2024-09-13T18:52:51.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.429+0000 7f982da5f700 1 -- 172.21.15.7:0/3748037775 wait complete. 2024-09-13T18:52:51.438 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-13T18:52:51.449 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:52:51.449 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:52:51.457 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:51.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff990e5a700 1 Processor -- start 2024-09-13T18:52:51.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff990e5a700 1 -- start start 2024-09-13T18:52:51.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff990e5a700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c165590 0x7ff98c1619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:51.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff990e5a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98c071f30 con 0x7ff98c0b3ae0 2024-09-13T18:52:51.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff990e5a700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98c1622e0 con 0x7ff98c160eb0 2024-09-13T18:52:51.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff990e5a700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c162000 con 0x7ff98c165590 2024-09-13T18:52:51.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff98affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff98c0b3ae0 0x7ff98c161290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38738/0 (socket says 172.21.15.7:38738) 2024-09-13T18:52:51.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff98affd700 1 -- 172.21.15.7:0/1897885408 learned_addr learned my addr 172.21.15.7:0/1897885408 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:51.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff98bfff700 1 --2- 172.21.15.7:0/1897885408 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c165590 0x7ff98c1619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:51.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff98a7fc700 1 -- 172.21.15.7:0/1897885408 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481356045 0 0) 0x7ff98c071f30 con 0x7ff98c0b3ae0 2024-09-13T18:52:51.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff98a7fc700 1 -- 172.21.15.7:0/1897885408 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9780035d0 con 0x7ff98c0b3ae0 2024-09-13T18:52:51.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff98a7fc700 1 -- 172.21.15.7:0/1897885408 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704724183 0 0) 0x7ff98c1622e0 con 0x7ff98c160eb0 2024-09-13T18:52:51.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff98a7fc700 1 -- 172.21.15.7:0/1897885408 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff98c071f30 con 0x7ff98c160eb0 2024-09-13T18:52:51.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff98bfff700 1 -- 172.21.15.7:0/1897885408 >> v1:172.21.15.171:6789/0 conn(0x7ff98c160eb0 legacy=0x7ff98c0b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff98bfff700 1 -- 172.21.15.7:0/1897885408 >> v1:172.21.15.7:6789/0 conn(0x7ff98c0b3ae0 legacy=0x7ff98c161290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff98bfff700 1 -- 172.21.15.7:0/1897885408 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c164ab0 con 0x7ff98c165590 2024-09-13T18:52:51.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff98bfff700 1 --2- 172.21.15.7:0/1897885408 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c165590 0x7ff98c1619a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff97c009880 tx=0x7ff97c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef155252854dcab9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff98a7fc700 1 -- 172.21.15.7:0/1897885408 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c011830 con 0x7ff98c165590 2024-09-13T18:52:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff98a7fc700 1 -- 172.21.15.7:0/1897885408 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97c011990 con 0x7ff98c165590 2024-09-13T18:52:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff98a7fc700 1 -- 172.21.15.7:0/1897885408 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97c011d20 con 0x7ff98c165590 2024-09-13T18:52:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff990e5a700 1 -- 172.21.15.7:0/1897885408 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c165590 msgr2=0x7ff98c1619a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.609+0000 7ff990e5a700 1 --2- 172.21.15.7:0/1897885408 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c165590 0x7ff98c1619a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff97c009880 tx=0x7ff97c009b90 comp rx=0 tx=0).stop 2024-09-13T18:52:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff990e5a700 1 -- 172.21.15.7:0/1897885408 shutdown_connections 2024-09-13T18:52:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff990e5a700 1 --2- 172.21.15.7:0/1897885408 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c165590 0x7ff98c1619a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff990e5a700 1 -- 172.21.15.7:0/1897885408 >> 172.21.15.7:0/1897885408 conn(0x7ff98c0b0f00 msgr2=0x7ff98c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:51.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff990e5a700 1 -- 172.21.15.7:0/1897885408 shutdown_connections 2024-09-13T18:52:51.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff990e5a700 1 -- 172.21.15.7:0/1897885408 wait complete. 2024-09-13T18:52:51.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff990e5a700 1 Processor -- start 2024-09-13T18:52:51.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff990e5a700 1 -- start start 2024-09-13T18:52:51.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff990e5a700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c0b3ae0 0x7ff98c15e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:51.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff990e5a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff98c160eb0 0x7ff98c157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:51.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff990e5a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff98c165590 0x7ff98c15b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:51.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff990e5a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c1578d0 con 0x7ff98c160eb0 2024-09-13T18:52:51.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff990e5a700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c1572a0 con 0x7ff98c165590 2024-09-13T18:52:51.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff990e5a700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c15e8b0 con 0x7ff98c0b3ae0 2024-09-13T18:52:51.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff98affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff98c160eb0 0x7ff98c157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:51.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff98affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff98c160eb0 0x7ff98c157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38950/0 (socket says 172.21.15.7:38950) 2024-09-13T18:52:51.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff98affd700 1 -- 172.21.15.7:0/3325215695 learned_addr learned my addr 172.21.15.7:0/3325215695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:51.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff98b7fe700 1 --2- 172.21.15.7:0/3325215695 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c0b3ae0 0x7ff98c15e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:51.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff98bfff700 1 --2- 172.21.15.7:0/3325215695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff98c165590 0x7ff98c15b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:51.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff98affd700 1 -- 172.21.15.7:0/3325215695 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c0b3ae0 msgr2=0x7ff98c15e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff98affd700 1 --2- 172.21.15.7:0/3325215695 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c0b3ae0 0x7ff98c15e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff98affd700 1 -- 172.21.15.7:0/3325215695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff98c165590 msgr2=0x7ff98c15b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff98affd700 1 --2- 172.21.15.7:0/3325215695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff98c165590 0x7ff98c15b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff98affd700 1 -- 172.21.15.7:0/3325215695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff97c0094b0 con 0x7ff98c160eb0 2024-09-13T18:52:51.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff98affd700 1 --2- 172.21.15.7:0/3325215695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff98c160eb0 0x7ff98c157cf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff98400cbc0 tx=0x7ff98400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:51.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff983fff700 1 -- 172.21.15.7:0/3325215695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98400a770 con 0x7ff98c160eb0 2024-09-13T18:52:51.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff983fff700 1 -- 172.21.15.7:0/3325215695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98400a8f0 con 0x7ff98c160eb0 2024-09-13T18:52:51.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff990e5a700 1 -- 172.21.15.7:0/3325215695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98c1602a0 con 0x7ff98c160eb0 2024-09-13T18:52:51.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff983fff700 1 -- 172.21.15.7:0/3325215695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98400aca0 con 0x7ff98c160eb0 2024-09-13T18:52:51.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.613+0000 7ff990e5a700 1 -- 172.21.15.7:0/3325215695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98c1574f0 con 0x7ff98c160eb0 2024-09-13T18:52:51.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.617+0000 7ff990e5a700 1 -- 172.21.15.7:0/3325215695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff98c157a10 con 0x7ff98c160eb0 2024-09-13T18:52:51.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.617+0000 7ff983fff700 1 -- 172.21.15.7:0/3325215695 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff98401b3b0 con 0x7ff98c160eb0 2024-09-13T18:52:51.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.617+0000 7ff983fff700 1 --2- 172.21.15.7:0/3325215695 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff96c066730 0x7ff96c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:51.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.617+0000 7ff983fff700 1 -- 172.21.15.7:0/3325215695 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff984083b70 con 0x7ff98c160eb0 2024-09-13T18:52:51.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.617+0000 7ff98b7fe700 1 --2- 172.21.15.7:0/3325215695 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff96c066730 0x7ff96c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:51.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.617+0000 7ff98b7fe700 1 --2- 172.21.15.7:0/3325215695 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff96c066730 0x7ff96c068bf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff9740011e0 tx=0x7ff974009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:51.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.625+0000 7ff983fff700 1 -- 172.21.15.7:0/3325215695 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff98c157a10 con 0x7ff98c160eb0 2024-09-13T18:52:51.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.801+0000 7ff990e5a700 1 -- 172.21.15.7:0/3325215695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff98c157a10 con 0x7ff98c160eb0 2024-09-13T18:52:51.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.805+0000 7ff983fff700 1 -- 172.21.15.7:0/3325215695 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7ff98c157a10 con 0x7ff98c160eb0 2024-09-13T18:52:51.810 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:51.811 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.805+0000 7ff990e5a700 1 -- 172.21.15.7:0/3325215695 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff96c066730 msgr2=0x7ff96c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.805+0000 7ff990e5a700 1 --2- 172.21.15.7:0/3325215695 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff96c066730 0x7ff96c068bf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff9740011e0 tx=0x7ff974009040 comp rx=0 tx=0).stop 2024-09-13T18:52:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.805+0000 7ff990e5a700 1 -- 172.21.15.7:0/3325215695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff98c160eb0 msgr2=0x7ff98c157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.805+0000 7ff990e5a700 1 --2- 172.21.15.7:0/3325215695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff98c160eb0 0x7ff98c157cf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff98400cbc0 tx=0x7ff98400cbf0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.805+0000 7ff990e5a700 1 -- 172.21.15.7:0/3325215695 shutdown_connections 2024-09-13T18:52:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.805+0000 7ff990e5a700 1 --2- 172.21.15.7:0/3325215695 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff96c066730 0x7ff96c068bf0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.805+0000 7ff990e5a700 1 --2- 172.21.15.7:0/3325215695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff98c165590 0x7ff98c15b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.805+0000 7ff990e5a700 1 --2- 172.21.15.7:0/3325215695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff98c160eb0 0x7ff98c157cf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.805+0000 7ff990e5a700 1 --2- 172.21.15.7:0/3325215695 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c0b3ae0 0x7ff98c15e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.805+0000 7ff990e5a700 1 -- 172.21.15.7:0/3325215695 >> 172.21.15.7:0/3325215695 conn(0x7ff98c0b0f00 msgr2=0x7ff98c161290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:51.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.805+0000 7ff990e5a700 1 -- 172.21.15.7:0/3325215695 shutdown_connections 2024-09-13T18:52:51.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.805+0000 7ff990e5a700 1 -- 172.21.15.7:0/3325215695 wait complete. 2024-09-13T18:52:51.828 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:52:15.554683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:52:16.896879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:52:51.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:52:51.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2e5aa2700 1 Processor -- start 2024-09-13T18:52:51.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2e5aa2700 1 -- start start 2024-09-13T18:52:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2e5aa2700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e015cda0 0x7fa2e00d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2e5aa2700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa2e00b5e00 0x7fa2e00b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2e5aa2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e00d0880 con 0x7fa2e00b6780 2024-09-13T18:52:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2e5aa2700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e00cde50 con 0x7fa2e00b5e00 2024-09-13T18:52:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2e5aa2700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e00d16e0 con 0x7fa2e015cda0 2024-09-13T18:52:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2dffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa2e00b6780 0x7fa2e006a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38748/0 (socket says 172.21.15.7:38748) 2024-09-13T18:52:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2dffff700 1 -- 172.21.15.7:0/3779170072 learned_addr learned my addr 172.21.15.7:0/3779170072 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2df7fe700 1 --2- 172.21.15.7:0/3779170072 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e015cda0 0x7fa2e00d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2deffd700 1 --2- 172.21.15.7:0/3779170072 >> v2:172.21.15.171:3300/0 conn(0x7fa2e00b5e00 0x7fa2e00b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:51.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2de7fc700 1 -- 172.21.15.7:0/3779170072 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865586291 0 0) 0x7fa2e00d0880 con 0x7fa2e00b6780 2024-09-13T18:52:51.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2de7fc700 1 -- 172.21.15.7:0/3779170072 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2cc0035d0 con 0x7fa2e00b6780 2024-09-13T18:52:51.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2df7fe700 1 -- 172.21.15.7:0/3779170072 >> v2:172.21.15.171:3300/0 conn(0x7fa2e00b5e00 msgr2=0x7fa2e00b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:51.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2df7fe700 1 --2- 172.21.15.7:0/3779170072 >> v2:172.21.15.171:3300/0 conn(0x7fa2e00b5e00 0x7fa2e00b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2df7fe700 1 -- 172.21.15.7:0/3779170072 >> v1:172.21.15.7:6789/0 conn(0x7fa2e00b6780 legacy=0x7fa2e006a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2df7fe700 1 -- 172.21.15.7:0/3779170072 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e00d0ad0 con 0x7fa2e015cda0 2024-09-13T18:52:51.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2df7fe700 1 --2- 172.21.15.7:0/3779170072 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e015cda0 0x7fa2e00d1b70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009880 tx=0x7fa2d0009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c8c7b99e3dc49ec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:51.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2de7fc700 1 -- 172.21.15.7:0/3779170072 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0011830 con 0x7fa2e015cda0 2024-09-13T18:52:51.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2de7fc700 1 -- 172.21.15.7:0/3779170072 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d0011990 con 0x7fa2e015cda0 2024-09-13T18:52:51.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.945+0000 7fa2de7fc700 1 -- 172.21.15.7:0/3779170072 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0011d00 con 0x7fa2e015cda0 2024-09-13T18:52:51.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.949+0000 7fa2e5aa2700 1 -- 172.21.15.7:0/3779170072 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e015cda0 msgr2=0x7fa2e00d1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.949+0000 7fa2e5aa2700 1 --2- 172.21.15.7:0/3779170072 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e015cda0 0x7fa2e00d1b70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009880 tx=0x7fa2d0009b90 comp rx=0 tx=0).stop 2024-09-13T18:52:51.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.949+0000 7fa2e5aa2700 1 -- 172.21.15.7:0/3779170072 shutdown_connections 2024-09-13T18:52:51.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.949+0000 7fa2e5aa2700 1 --2- 172.21.15.7:0/3779170072 >> v2:172.21.15.171:3300/0 conn(0x7fa2e00b5e00 0x7fa2e00b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.949+0000 7fa2e5aa2700 1 --2- 172.21.15.7:0/3779170072 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e015cda0 0x7fa2e00d1b70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.949+0000 7fa2e5aa2700 1 -- 172.21.15.7:0/3779170072 >> 172.21.15.7:0/3779170072 conn(0x7fa2e00d2cb0 msgr2=0x7fa2e00d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:51.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.949+0000 7fa2e5aa2700 1 -- 172.21.15.7:0/3779170072 shutdown_connections 2024-09-13T18:52:51.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.949+0000 7fa2e5aa2700 1 -- 172.21.15.7:0/3779170072 wait complete. 2024-09-13T18:52:51.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.949+0000 7fa2e5aa2700 1 Processor -- start 2024-09-13T18:52:51.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.949+0000 7fa2e5aa2700 1 -- start start 2024-09-13T18:52:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2e5aa2700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2e00b5e00 0x7fa2e00c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2e5aa2700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e00b6780 0x7fa2e015e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2e5aa2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e015cda0 0x7fa2e01f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2e5aa2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e01f2230 con 0x7fa2e015cda0 2024-09-13T18:52:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2e5aa2700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e015ec50 con 0x7fa2e00b5e00 2024-09-13T18:52:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2e5aa2700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e015ef30 con 0x7fa2e00b6780 2024-09-13T18:52:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2df7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2e00b5e00 0x7fa2e00c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2deffd700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e00b6780 0x7fa2e015e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2dffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e015cda0 0x7fa2e01f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2df7fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2e00b5e00 0x7fa2e00c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:41524/0 (socket says 172.21.15.7:41524) 2024-09-13T18:52:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2df7fe700 1 -- 172.21.15.7:0/3105074211 learned_addr learned my addr 172.21.15.7:0/3105074211 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2dffff700 1 -- 172.21.15.7:0/3105074211 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e00b6780 msgr2=0x7fa2e015e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2dffff700 1 --2- 172.21.15.7:0/3105074211 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e00b6780 0x7fa2e015e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2dffff700 1 -- 172.21.15.7:0/3105074211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2e00b5e00 msgr2=0x7fa2e00c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2dffff700 1 --2- 172.21.15.7:0/3105074211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2e00b5e00 0x7fa2e00c93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2dffff700 1 -- 172.21.15.7:0/3105074211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d00094b0 con 0x7fa2e015cda0 2024-09-13T18:52:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2df7fe700 1 --2- 172.21.15.7:0/3105074211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2e00b5e00 0x7fa2e00c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2deffd700 1 --2- 172.21.15.7:0/3105074211 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e00b6780 0x7fa2e015e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2dffff700 1 --2- 172.21.15.7:0/3105074211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e015cda0 0x7fa2e01f5940 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa2e008bf80 tx=0x7fa2d800ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2d7fff700 1 -- 172.21.15.7:0/3105074211 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8007480 con 0x7fa2e015cda0 2024-09-13T18:52:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2d7fff700 1 -- 172.21.15.7:0/3105074211 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d80075e0 con 0x7fa2e015cda0 2024-09-13T18:52:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2e5aa2700 1 -- 172.21.15.7:0/3105074211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e015f210 con 0x7fa2e015cda0 2024-09-13T18:52:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2d7fff700 1 -- 172.21.15.7:0/3105074211 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8007950 con 0x7fa2e015cda0 2024-09-13T18:52:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.953+0000 7fa2e5aa2700 1 -- 172.21.15.7:0/3105074211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e00c9910 con 0x7fa2e015cda0 2024-09-13T18:52:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.957+0000 7fa2e5aa2700 1 -- 172.21.15.7:0/3105074211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e015fa10 con 0x7fa2e015cda0 2024-09-13T18:52:51.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.957+0000 7fa2d7fff700 1 -- 172.21.15.7:0/3105074211 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa2d801b3b0 con 0x7fa2e015cda0 2024-09-13T18:52:51.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.957+0000 7fa2d7fff700 1 --2- 172.21.15.7:0/3105074211 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa2c0066730 0x7fa2c0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:51.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.957+0000 7fa2d7fff700 1 -- 172.21.15.7:0/3105074211 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2d80839b0 con 0x7fa2e015cda0 2024-09-13T18:52:51.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.957+0000 7fa2df7fe700 1 --2- 172.21.15.7:0/3105074211 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa2c0066730 0x7fa2c0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:51.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.957+0000 7fa2df7fe700 1 --2- 172.21.15.7:0/3105074211 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa2c0066730 0x7fa2c0068bf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa2e015fd60 tx=0x7fa2d0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:51.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:51.965+0000 7fa2d7fff700 1 -- 172.21.15.7:0/3105074211 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa2e015fa10 con 0x7fa2e015cda0 2024-09-13T18:52:52.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.133+0000 7fa2e5aa2700 1 -- 172.21.15.7:0/3105074211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2e015fa10 con 0x7fa2e015cda0 2024-09-13T18:52:52.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.133+0000 7fa2d7fff700 1 -- 172.21.15.7:0/3105074211 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2e015fa10 con 0x7fa2e015cda0 2024-09-13T18:52:52.140 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:52:52.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.137+0000 7fa2e5aa2700 1 -- 172.21.15.7:0/3105074211 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa2c0066730 msgr2=0x7fa2c0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:52.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.137+0000 7fa2e5aa2700 1 --2- 172.21.15.7:0/3105074211 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa2c0066730 0x7fa2c0068bf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa2e015fd60 tx=0x7fa2d0008030 comp rx=0 tx=0).stop 2024-09-13T18:52:52.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.137+0000 7fa2e5aa2700 1 -- 172.21.15.7:0/3105074211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e015cda0 msgr2=0x7fa2e01f5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:52.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.137+0000 7fa2e5aa2700 1 --2- 172.21.15.7:0/3105074211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e015cda0 0x7fa2e01f5940 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa2e008bf80 tx=0x7fa2d800ca50 comp rx=0 tx=0).stop 2024-09-13T18:52:52.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.137+0000 7fa2e5aa2700 1 -- 172.21.15.7:0/3105074211 shutdown_connections 2024-09-13T18:52:52.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.137+0000 7fa2e5aa2700 1 --2- 172.21.15.7:0/3105074211 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa2c0066730 0x7fa2c0068bf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.137+0000 7fa2e5aa2700 1 --2- 172.21.15.7:0/3105074211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e015cda0 0x7fa2e01f5940 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.137+0000 7fa2e5aa2700 1 --2- 172.21.15.7:0/3105074211 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e00b6780 0x7fa2e015e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.137+0000 7fa2e5aa2700 1 --2- 172.21.15.7:0/3105074211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2e00b5e00 0x7fa2e00c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.137+0000 7fa2e5aa2700 1 -- 172.21.15.7:0/3105074211 >> 172.21.15.7:0/3105074211 conn(0x7fa2e00d2cb0 msgr2=0x7fa2e00c9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:52.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.137+0000 7fa2e5aa2700 1 -- 172.21.15.7:0/3105074211 shutdown_connections 2024-09-13T18:52:52.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.137+0000 7fa2e5aa2700 1 -- 172.21.15.7:0/3105074211 wait complete. 2024-09-13T18:52:52.155 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:52:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.269+0000 7fbf5dced700 1 Processor -- start 2024-09-13T18:52:52.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.269+0000 7fbf5dced700 1 -- start start 2024-09-13T18:52:52.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.269+0000 7fbf5dced700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf580cf690 0x7fbf580ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:52.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.269+0000 7fbf5dced700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fbf580d08a0 0x7fbf5815b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:52.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.269+0000 7fbf5dced700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf580d1910 con 0x7fbf580b5c70 2024-09-13T18:52:52.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.269+0000 7fbf5dced700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf58151e90 con 0x7fbf580cf690 2024-09-13T18:52:52.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.269+0000 7fbf5dced700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf580d0d90 con 0x7fbf580d08a0 2024-09-13T18:52:52.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.269+0000 7fbf577fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf580cf690 0x7fbf580ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:52.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf577fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf580cf690 0x7fbf580ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38970/0 (socket says 172.21.15.7:38970) 2024-09-13T18:52:52.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf577fe700 1 -- 172.21.15.7:0/2555067239 learned_addr learned my addr 172.21.15.7:0/2555067239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:52.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf56ffd700 1 --2- 172.21.15.7:0/2555067239 >> v2:172.21.15.171:3300/0 conn(0x7fbf580d08a0 0x7fbf5815b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf567fc700 1 -- 172.21.15.7:0/2555067239 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586145056 0 0) 0x7fbf580d1910 con 0x7fbf580b5c70 2024-09-13T18:52:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf577fe700 1 -- 172.21.15.7:0/2555067239 >> v2:172.21.15.171:3300/0 conn(0x7fbf580d08a0 msgr2=0x7fbf5815b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf577fe700 1 --2- 172.21.15.7:0/2555067239 >> v2:172.21.15.171:3300/0 conn(0x7fbf580d08a0 0x7fbf5815b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf577fe700 1 -- 172.21.15.7:0/2555067239 >> v1:172.21.15.171:6789/0 conn(0x7fbf580b5c70 legacy=0x7fbf580cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf577fe700 1 -- 172.21.15.7:0/2555067239 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf580d1070 con 0x7fbf580cf690 2024-09-13T18:52:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf577fe700 1 --2- 172.21.15.7:0/2555067239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf580cf690 0x7fbf580ce7d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbf5815af30 tx=0x7fbf48009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccac99b97c91be8e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf567fc700 1 -- 172.21.15.7:0/2555067239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48011830 con 0x7fbf580cf690 2024-09-13T18:52:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf567fc700 1 -- 172.21.15.7:0/2555067239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf48011990 con 0x7fbf580cf690 2024-09-13T18:52:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf567fc700 1 -- 172.21.15.7:0/2555067239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf48011d20 con 0x7fbf580cf690 2024-09-13T18:52:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf5dced700 1 -- 172.21.15.7:0/2555067239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf580cf690 msgr2=0x7fbf580ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf5dced700 1 --2- 172.21.15.7:0/2555067239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf580cf690 0x7fbf580ce7d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbf5815af30 tx=0x7fbf48009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf5dced700 1 -- 172.21.15.7:0/2555067239 shutdown_connections 2024-09-13T18:52:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf5dced700 1 --2- 172.21.15.7:0/2555067239 >> v2:172.21.15.171:3300/0 conn(0x7fbf580d08a0 0x7fbf5815b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf5dced700 1 --2- 172.21.15.7:0/2555067239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf580cf690 0x7fbf580ce7d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf5dced700 1 -- 172.21.15.7:0/2555067239 >> 172.21.15.7:0/2555067239 conn(0x7fbf58071da0 msgr2=0x7fbf58072180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf5dced700 1 -- 172.21.15.7:0/2555067239 shutdown_connections 2024-09-13T18:52:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.273+0000 7fbf5dced700 1 -- 172.21.15.7:0/2555067239 wait complete. 2024-09-13T18:52:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf5dced700 1 Processor -- start 2024-09-13T18:52:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf5dced700 1 -- start start 2024-09-13T18:52:52.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf5dced700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf580b5c70 0x7fbf58143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:52.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf5dced700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf580cf690 0x7fbf581535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:52.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf5dced700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbf580d08a0 0x7fbf58154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:52.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf5dced700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf58143ea0 con 0x7fbf580cf690 2024-09-13T18:52:52.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf5dced700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf581565e0 con 0x7fbf580b5c70 2024-09-13T18:52:52.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf5dced700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf581559d0 con 0x7fbf580d08a0 2024-09-13T18:52:52.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf56ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf580cf690 0x7fbf581535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:52.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf56ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf580cf690 0x7fbf581535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38982/0 (socket says 172.21.15.7:38982) 2024-09-13T18:52:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf56ffd700 1 -- 172.21.15.7:0/1762948863 learned_addr learned my addr 172.21.15.7:0/1762948863 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf577fe700 1 --2- 172.21.15.7:0/1762948863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf580b5c70 0x7fbf58143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf57fff700 1 --2- 172.21.15.7:0/1762948863 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbf580d08a0 0x7fbf58154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf56ffd700 1 -- 172.21.15.7:0/1762948863 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbf580d08a0 msgr2=0x7fbf58154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf56ffd700 1 --2- 172.21.15.7:0/1762948863 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbf580d08a0 0x7fbf58154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf56ffd700 1 -- 172.21.15.7:0/1762948863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf580b5c70 msgr2=0x7fbf58143960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf56ffd700 1 --2- 172.21.15.7:0/1762948863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf580b5c70 0x7fbf58143960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf56ffd700 1 -- 172.21.15.7:0/1762948863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf48009570 con 0x7fbf580cf690 2024-09-13T18:52:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf577fe700 1 --2- 172.21.15.7:0/1762948863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf580b5c70 0x7fbf58143960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf57fff700 1 --2- 172.21.15.7:0/1762948863 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbf580d08a0 0x7fbf58154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf56ffd700 1 --2- 172.21.15.7:0/1762948863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf580cf690 0x7fbf581535f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000e9d0 tx=0x7fbf4000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf4ffff700 1 -- 172.21.15.7:0/1762948863 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf40009720 con 0x7fbf580cf690 2024-09-13T18:52:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf4ffff700 1 -- 172.21.15.7:0/1762948863 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4000c850 con 0x7fbf580cf690 2024-09-13T18:52:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf5dced700 1 -- 172.21.15.7:0/1762948863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf58155cb0 con 0x7fbf580cf690 2024-09-13T18:52:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf4ffff700 1 -- 172.21.15.7:0/1762948863 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4000cbc0 con 0x7fbf580cf690 2024-09-13T18:52:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.277+0000 7fbf5dced700 1 -- 172.21.15.7:0/1762948863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf58156270 con 0x7fbf580cf690 2024-09-13T18:52:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.281+0000 7fbf5dced700 1 -- 172.21.15.7:0/1762948863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf581552f0 con 0x7fbf580cf690 2024-09-13T18:52:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.281+0000 7fbf4ffff700 1 -- 172.21.15.7:0/1762948863 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbf4000cd20 con 0x7fbf580cf690 2024-09-13T18:52:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.281+0000 7fbf4ffff700 1 --2- 172.21.15.7:0/1762948863 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbf38066730 0x7fbf38068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.281+0000 7fbf4ffff700 1 -- 172.21.15.7:0/1762948863 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf40084780 con 0x7fbf580cf690 2024-09-13T18:52:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.281+0000 7fbf577fe700 1 --2- 172.21.15.7:0/1762948863 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbf38066730 0x7fbf38068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:52.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.285+0000 7fbf577fe700 1 --2- 172.21.15.7:0/1762948863 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbf38066730 0x7fbf38068bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbf48000c00 tx=0x7fbf48008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.289+0000 7fbf4ffff700 1 -- 172.21.15.7:0/1762948863 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbf40051940 con 0x7fbf580cf690 2024-09-13T18:52:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.465+0000 7fbf5dced700 1 -- 172.21.15.7:0/1762948863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbf58073820 con 0x7fbf580cf690 2024-09-13T18:52:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.465+0000 7fbf4ffff700 1 -- 172.21.15.7:0/1762948863 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbf40054f60 con 0x7fbf580cf690 2024-09-13T18:52:52.472 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:52:52.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.469+0000 7fbf5dced700 1 -- 172.21.15.7:0/1762948863 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbf38066730 msgr2=0x7fbf38068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:52.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.469+0000 7fbf5dced700 1 --2- 172.21.15.7:0/1762948863 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbf38066730 0x7fbf38068bf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbf48000c00 tx=0x7fbf48008030 comp rx=0 tx=0).stop 2024-09-13T18:52:52.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.469+0000 7fbf5dced700 1 -- 172.21.15.7:0/1762948863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf580cf690 msgr2=0x7fbf581535f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:52.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.469+0000 7fbf5dced700 1 --2- 172.21.15.7:0/1762948863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf580cf690 0x7fbf581535f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000e9d0 tx=0x7fbf4000ed90 comp rx=0 tx=0).stop 2024-09-13T18:52:52.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.469+0000 7fbf5dced700 1 -- 172.21.15.7:0/1762948863 shutdown_connections 2024-09-13T18:52:52.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.469+0000 7fbf5dced700 1 --2- 172.21.15.7:0/1762948863 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbf38066730 0x7fbf38068bf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.469+0000 7fbf5dced700 1 --2- 172.21.15.7:0/1762948863 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbf580d08a0 0x7fbf58154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.469+0000 7fbf5dced700 1 --2- 172.21.15.7:0/1762948863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf580cf690 0x7fbf581535f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.469+0000 7fbf5dced700 1 --2- 172.21.15.7:0/1762948863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf580b5c70 0x7fbf58143960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.469+0000 7fbf5dced700 1 -- 172.21.15.7:0/1762948863 >> 172.21.15.7:0/1762948863 conn(0x7fbf58071da0 msgr2=0x7fbf5806e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:52.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.469+0000 7fbf5dced700 1 -- 172.21.15.7:0/1762948863 shutdown_connections 2024-09-13T18:52:52.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.469+0000 7fbf5dced700 1 -- 172.21.15.7:0/1762948863 wait complete. 2024-09-13T18:52:52.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:52:52.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.601+0000 7f1c6371a700 1 Processor -- start 2024-09-13T18:52:52.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.601+0000 7f1c6371a700 1 -- start start 2024-09-13T18:52:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c6371a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1c5c06b530 0x7f1c5c0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c6371a700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f1c5c15cfb0 0x7f1c5c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:52.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c6371a700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c5c0d12f0 con 0x7f1c5c15cbd0 2024-09-13T18:52:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c6371a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5c0b6590 con 0x7f1c5c06b530 2024-09-13T18:52:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c6371a700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5c06ba80 con 0x7f1c5c15cfb0 2024-09-13T18:52:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c60cb5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1c5c06b530 0x7f1c5c0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c60cb5700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1c5c06b530 0x7f1c5c0ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38994/0 (socket says 172.21.15.7:38994) 2024-09-13T18:52:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c60cb5700 1 -- 172.21.15.7:0/4132140323 learned_addr learned my addr 172.21.15.7:0/4132140323 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:52.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c61cb7700 1 --2- 172.21.15.7:0/4132140323 >> v2:172.21.15.171:3300/0 conn(0x7f1c5c15cfb0 0x7f1c5c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c60cb5700 1 -- 172.21.15.7:0/4132140323 >> v2:172.21.15.171:3300/0 conn(0x7f1c5c15cfb0 msgr2=0x7f1c5c0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c60cb5700 1 --2- 172.21.15.7:0/4132140323 >> v2:172.21.15.171:3300/0 conn(0x7f1c5c15cfb0 0x7f1c5c0ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c60cb5700 1 -- 172.21.15.7:0/4132140323 >> v1:172.21.15.171:6789/0 conn(0x7f1c5c15cbd0 legacy=0x7f1c5c0b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c60cb5700 1 -- 172.21.15.7:0/4132140323 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c0d09c0 con 0x7f1c5c06b530 2024-09-13T18:52:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c60cb5700 1 --2- 172.21.15.7:0/4132140323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c06b530 0x7f1c5c0ce020 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c06cde0 tx=0x7f1c58009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30b9a62b15ed369 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c53fff700 1 -- 172.21.15.7:0/4132140323 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58011830 con 0x7f1c5c06b530 2024-09-13T18:52:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c53fff700 1 -- 172.21.15.7:0/4132140323 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c58011990 con 0x7f1c5c06b530 2024-09-13T18:52:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c53fff700 1 -- 172.21.15.7:0/4132140323 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58011d20 con 0x7f1c5c06b530 2024-09-13T18:52:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c6371a700 1 -- 172.21.15.7:0/4132140323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c06b530 msgr2=0x7f1c5c0ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c6371a700 1 --2- 172.21.15.7:0/4132140323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c06b530 0x7f1c5c0ce020 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c06cde0 tx=0x7f1c58009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c6371a700 1 -- 172.21.15.7:0/4132140323 shutdown_connections 2024-09-13T18:52:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c6371a700 1 --2- 172.21.15.7:0/4132140323 >> v2:172.21.15.171:3300/0 conn(0x7f1c5c15cfb0 0x7f1c5c0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c6371a700 1 --2- 172.21.15.7:0/4132140323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c06b530 0x7f1c5c0ce020 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c6371a700 1 -- 172.21.15.7:0/4132140323 >> 172.21.15.7:0/4132140323 conn(0x7f1c5c0d2f30 msgr2=0x7f1c5c0d3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c6371a700 1 -- 172.21.15.7:0/4132140323 shutdown_connections 2024-09-13T18:52:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.605+0000 7f1c6371a700 1 -- 172.21.15.7:0/4132140323 wait complete. 2024-09-13T18:52:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c6371a700 1 Processor -- start 2024-09-13T18:52:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c6371a700 1 -- start start 2024-09-13T18:52:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c6371a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c06b530 0x7f1c5c159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c6371a700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1c5c15cbd0 0x7f1c5c15d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c6371a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c5c15cfb0 0x7f1c5c1f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c6371a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c15ff70 con 0x7f1c5c06b530 2024-09-13T18:52:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c6371a700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c15f2f0 con 0x7f1c5c15cfb0 2024-09-13T18:52:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c6371a700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c15f5d0 con 0x7f1c5c15cbd0 2024-09-13T18:52:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c614b6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c06b530 0x7f1c5c159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c614b6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c06b530 0x7f1c5c159560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38996/0 (socket says 172.21.15.7:38996) 2024-09-13T18:52:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c614b6700 1 -- 172.21.15.7:0/3921877273 learned_addr learned my addr 172.21.15.7:0/3921877273 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c61cb7700 1 --2- 172.21.15.7:0/3921877273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c5c15cfb0 0x7f1c5c1f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c60cb5700 1 --2- 172.21.15.7:0/3921877273 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1c5c15cbd0 0x7f1c5c15d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c614b6700 1 -- 172.21.15.7:0/3921877273 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1c5c15cbd0 msgr2=0x7f1c5c15d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c614b6700 1 --2- 172.21.15.7:0/3921877273 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1c5c15cbd0 0x7f1c5c15d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c614b6700 1 -- 172.21.15.7:0/3921877273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c5c15cfb0 msgr2=0x7f1c5c1f5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c614b6700 1 --2- 172.21.15.7:0/3921877273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c5c15cfb0 0x7f1c5c1f5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c614b6700 1 -- 172.21.15.7:0/3921877273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c58009570 con 0x7f1c5c06b530 2024-09-13T18:52:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.609+0000 7f1c61cb7700 1 --2- 172.21.15.7:0/3921877273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c5c15cfb0 0x7f1c5c1f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.613+0000 7f1c614b6700 1 --2- 172.21.15.7:0/3921877273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c06b530 0x7f1c5c159560 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00c500 tx=0x7f1c4c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.613+0000 7f1c51ffb700 1 -- 172.21.15.7:0/3921877273 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c0052d0 con 0x7f1c5c06b530 2024-09-13T18:52:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.613+0000 7f1c51ffb700 1 -- 172.21.15.7:0/3921877273 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4c005430 con 0x7f1c5c06b530 2024-09-13T18:52:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.613+0000 7f1c6371a700 1 -- 172.21.15.7:0/3921877273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c15f8b0 con 0x7f1c5c06b530 2024-09-13T18:52:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.613+0000 7f1c51ffb700 1 -- 172.21.15.7:0/3921877273 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c0055a0 con 0x7f1c5c06b530 2024-09-13T18:52:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.613+0000 7f1c6371a700 1 -- 172.21.15.7:0/3921877273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c160390 con 0x7f1c5c06b530 2024-09-13T18:52:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.613+0000 7f1c6371a700 1 -- 172.21.15.7:0/3921877273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5c160580 con 0x7f1c5c06b530 2024-09-13T18:52:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.613+0000 7f1c51ffb700 1 -- 172.21.15.7:0/3921877273 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1c4c0038e0 con 0x7f1c5c06b530 2024-09-13T18:52:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.613+0000 7f1c51ffb700 1 --2- 172.21.15.7:0/3921877273 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1c40066730 0x7f1c40068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.617+0000 7f1c51ffb700 1 -- 172.21.15.7:0/3921877273 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c4c083470 con 0x7f1c5c06b530 2024-09-13T18:52:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.617+0000 7f1c60cb5700 1 --2- 172.21.15.7:0/3921877273 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1c40066730 0x7f1c40068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.617+0000 7f1c60cb5700 1 --2- 172.21.15.7:0/3921877273 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1c40066730 0x7f1c40068bf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1c58000c00 tx=0x7f1c58008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:52.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.625+0000 7f1c51ffb700 1 -- 172.21.15.7:0/3921877273 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1c4c050870 con 0x7f1c5c06b530 2024-09-13T18:52:52.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.789+0000 7f1c6371a700 1 -- 172.21.15.7:0/3921877273 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1c5c0b5e80 con 0x7f1c5c06b530 2024-09-13T18:52:52.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.789+0000 7f1c51ffb700 1 -- 172.21.15.7:0/3921877273 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1c4c053e90 con 0x7f1c5c06b530 2024-09-13T18:52:52.796 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:52:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.793+0000 7f1c6371a700 1 -- 172.21.15.7:0/3921877273 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1c40066730 msgr2=0x7f1c40068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.793+0000 7f1c6371a700 1 --2- 172.21.15.7:0/3921877273 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1c40066730 0x7f1c40068bf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1c58000c00 tx=0x7f1c58008030 comp rx=0 tx=0).stop 2024-09-13T18:52:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.793+0000 7f1c6371a700 1 -- 172.21.15.7:0/3921877273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c06b530 msgr2=0x7f1c5c159560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.793+0000 7f1c6371a700 1 --2- 172.21.15.7:0/3921877273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c06b530 0x7f1c5c159560 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00c500 tx=0x7f1c4c00c810 comp rx=0 tx=0).stop 2024-09-13T18:52:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.793+0000 7f1c6371a700 1 -- 172.21.15.7:0/3921877273 shutdown_connections 2024-09-13T18:52:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.793+0000 7f1c6371a700 1 --2- 172.21.15.7:0/3921877273 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1c40066730 0x7f1c40068bf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.793+0000 7f1c6371a700 1 --2- 172.21.15.7:0/3921877273 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1c5c15cfb0 0x7f1c5c1f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.793+0000 7f1c6371a700 1 --2- 172.21.15.7:0/3921877273 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1c5c15cbd0 0x7f1c5c15d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.793+0000 7f1c6371a700 1 --2- 172.21.15.7:0/3921877273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c5c06b530 0x7f1c5c159560 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.793+0000 7f1c6371a700 1 -- 172.21.15.7:0/3921877273 >> 172.21.15.7:0/3921877273 conn(0x7f1c5c0d2f30 msgr2=0x7f1c5c0b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.793+0000 7f1c6371a700 1 -- 172.21.15.7:0/3921877273 shutdown_connections 2024-09-13T18:52:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.793+0000 7f1c6371a700 1 -- 172.21.15.7:0/3921877273 wait complete. 2024-09-13T18:52:52.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:52:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60f59e700 1 Processor -- start 2024-09-13T18:52:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60f59e700 1 -- start start 2024-09-13T18:52:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60f59e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7ff61006d8c0 0x7ff610164a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60f59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff610070bd0 0x7ff610163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60f59e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff610162570 con 0x7ff610164f90 2024-09-13T18:52:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60f59e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff610162e60 con 0x7ff61006d8c0 2024-09-13T18:52:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60f59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff610071060 con 0x7ff610070bd0 2024-09-13T18:52:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60dd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff610164f90 0x7ff6100704c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38762/0 (socket says 172.21.15.7:38762) 2024-09-13T18:52:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60dd9b700 1 -- 172.21.15.7:0/1018477573 learned_addr learned my addr 172.21.15.7:0/1018477573 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60ed9d700 1 --2- 172.21.15.7:0/1018477573 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff610070bd0 0x7ff610163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60e59c700 1 --2- 172.21.15.7:0/1018477573 >> v2:172.21.15.171:3300/0 conn(0x7ff61006d8c0 0x7ff610164a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60d59a700 1 -- 172.21.15.7:0/1018477573 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612873087 0 0) 0x7ff610162570 con 0x7ff610164f90 2024-09-13T18:52:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60d59a700 1 -- 172.21.15.7:0/1018477573 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5fc0035d0 con 0x7ff610164f90 2024-09-13T18:52:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60ed9d700 1 -- 172.21.15.7:0/1018477573 >> v2:172.21.15.171:3300/0 conn(0x7ff61006d8c0 msgr2=0x7ff610164a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60ed9d700 1 --2- 172.21.15.7:0/1018477573 >> v2:172.21.15.171:3300/0 conn(0x7ff61006d8c0 0x7ff610164a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60ed9d700 1 -- 172.21.15.7:0/1018477573 >> v1:172.21.15.7:6789/0 conn(0x7ff610164f90 legacy=0x7ff6100704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60ed9d700 1 -- 172.21.15.7:0/1018477573 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff610071240 con 0x7ff610070bd0 2024-09-13T18:52:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60e59c700 1 --2- 172.21.15.7:0/1018477573 >> v2:172.21.15.171:3300/0 conn(0x7ff61006d8c0 0x7ff610164a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60ed9d700 1 --2- 172.21.15.7:0/1018477573 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff610070bd0 0x7ff610163280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff608009850 tx=0x7ff608009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aafd112b6cf2f5a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60d59a700 1 -- 172.21.15.7:0/1018477573 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff608011830 con 0x7ff610070bd0 2024-09-13T18:52:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60d59a700 1 -- 172.21.15.7:0/1018477573 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff608011990 con 0x7ff610070bd0 2024-09-13T18:52:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60d59a700 1 -- 172.21.15.7:0/1018477573 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff608011d00 con 0x7ff610070bd0 2024-09-13T18:52:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60f59e700 1 -- 172.21.15.7:0/1018477573 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff610070bd0 msgr2=0x7ff610163280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.921+0000 7ff60f59e700 1 --2- 172.21.15.7:0/1018477573 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff610070bd0 0x7ff610163280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff608009850 tx=0x7ff608009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60f59e700 1 -- 172.21.15.7:0/1018477573 shutdown_connections 2024-09-13T18:52:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60f59e700 1 --2- 172.21.15.7:0/1018477573 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff610070bd0 0x7ff610163280 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60f59e700 1 --2- 172.21.15.7:0/1018477573 >> v2:172.21.15.171:3300/0 conn(0x7ff61006d8c0 0x7ff610164a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60f59e700 1 -- 172.21.15.7:0/1018477573 >> 172.21.15.7:0/1018477573 conn(0x7ff6100b0e10 msgr2=0x7ff6100b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60f59e700 1 -- 172.21.15.7:0/1018477573 shutdown_connections 2024-09-13T18:52:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60f59e700 1 -- 172.21.15.7:0/1018477573 wait complete. 2024-09-13T18:52:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60f59e700 1 Processor -- start 2024-09-13T18:52:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60f59e700 1 -- start start 2024-09-13T18:52:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60f59e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff61006d8c0 0x7ff61014dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff610070bd0 0x7ff61015bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60f59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff610164f90 0x7ff61015e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60f59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6101507b0 con 0x7ff610070bd0 2024-09-13T18:52:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60f59e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61015dee0 con 0x7ff61006d8c0 2024-09-13T18:52:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60f59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61014f420 con 0x7ff610164f90 2024-09-13T18:52:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff610070bd0 0x7ff61015bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff610070bd0 0x7ff61015bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39004/0 (socket says 172.21.15.7:39004) 2024-09-13T18:52:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60dd9b700 1 -- 172.21.15.7:0/346708115 learned_addr learned my addr 172.21.15.7:0/346708115 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60ed9d700 1 --2- 172.21.15.7:0/346708115 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff610164f90 0x7ff61015e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.925+0000 7ff60e59c700 1 --2- 172.21.15.7:0/346708115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff61006d8c0 0x7ff61014dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.929+0000 7ff60ed9d700 1 -- 172.21.15.7:0/346708115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff61006d8c0 msgr2=0x7ff61014dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.929+0000 7ff60ed9d700 1 --2- 172.21.15.7:0/346708115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff61006d8c0 0x7ff61014dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.929+0000 7ff60ed9d700 1 -- 172.21.15.7:0/346708115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff610070bd0 msgr2=0x7ff61015bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.929+0000 7ff60ed9d700 1 --2- 172.21.15.7:0/346708115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff610070bd0 0x7ff61015bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.929+0000 7ff60ed9d700 1 -- 172.21.15.7:0/346708115 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f80094b0 con 0x7ff610164f90 2024-09-13T18:52:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.929+0000 7ff60dd9b700 1 --2- 172.21.15.7:0/346708115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff610070bd0 0x7ff61015bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.929+0000 7ff60ed9d700 1 --2- 172.21.15.7:0/346708115 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff610164f90 0x7ff61015e300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff608016000 tx=0x7ff608004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.929+0000 7ff606ffd700 1 -- 172.21.15.7:0/346708115 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff608004450 con 0x7ff610164f90 2024-09-13T18:52:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.929+0000 7ff606ffd700 1 -- 172.21.15.7:0/346708115 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60801f070 con 0x7ff610164f90 2024-09-13T18:52:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.929+0000 7ff60f59e700 1 -- 172.21.15.7:0/346708115 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6080094b0 con 0x7ff610164f90 2024-09-13T18:52:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.929+0000 7ff606ffd700 1 -- 172.21.15.7:0/346708115 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6080038e0 con 0x7ff610164f90 2024-09-13T18:52:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.929+0000 7ff60f59e700 1 -- 172.21.15.7:0/346708115 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6101504d0 con 0x7ff610164f90 2024-09-13T18:52:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.929+0000 7ff60f59e700 1 -- 172.21.15.7:0/346708115 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61015e020 con 0x7ff610164f90 2024-09-13T18:52:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.929+0000 7ff606ffd700 1 -- 172.21.15.7:0/346708115 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff608003a40 con 0x7ff610164f90 2024-09-13T18:52:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.933+0000 7ff606ffd700 1 --2- 172.21.15.7:0/346708115 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff5f0066730 0x7ff5f0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.933+0000 7ff606ffd700 1 -- 172.21.15.7:0/346708115 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff608021070 con 0x7ff610164f90 2024-09-13T18:52:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.933+0000 7ff60e59c700 1 --2- 172.21.15.7:0/346708115 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff5f0066730 0x7ff5f0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:52.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.933+0000 7ff60e59c700 1 --2- 172.21.15.7:0/346708115 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff5f0066730 0x7ff5f0068bf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff610164660 tx=0x7ff5f8009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:52.941+0000 7ff606ffd700 1 -- 172.21.15.7:0/346708115 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff608053090 con 0x7ff610164f90 2024-09-13T18:52:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.129+0000 7ff60f59e700 1 -- 172.21.15.7:0/346708115 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff61015e020 con 0x7ff610164f90 2024-09-13T18:52:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.129+0000 7ff606ffd700 1 -- 172.21.15.7:0/346708115 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff61015e020 con 0x7ff610164f90 2024-09-13T18:52:53.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.133+0000 7ff60f59e700 1 -- 172.21.15.7:0/346708115 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff5f0066730 msgr2=0x7ff5f0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.133+0000 7ff60f59e700 1 --2- 172.21.15.7:0/346708115 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff5f0066730 0x7ff5f0068bf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff610164660 tx=0x7ff5f8009240 comp rx=0 tx=0).stop 2024-09-13T18:52:53.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.133+0000 7ff60f59e700 1 -- 172.21.15.7:0/346708115 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff610164f90 msgr2=0x7ff61015e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.133+0000 7ff60f59e700 1 --2- 172.21.15.7:0/346708115 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff610164f90 0x7ff61015e300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff608016000 tx=0x7ff608004770 comp rx=0 tx=0).stop 2024-09-13T18:52:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.133+0000 7ff60f59e700 1 -- 172.21.15.7:0/346708115 shutdown_connections 2024-09-13T18:52:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.133+0000 7ff60f59e700 1 --2- 172.21.15.7:0/346708115 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff5f0066730 0x7ff5f0068bf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.133+0000 7ff60f59e700 1 --2- 172.21.15.7:0/346708115 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff610164f90 0x7ff61015e300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.133+0000 7ff60f59e700 1 --2- 172.21.15.7:0/346708115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff610070bd0 0x7ff61015bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.133+0000 7ff60f59e700 1 --2- 172.21.15.7:0/346708115 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff61006d8c0 0x7ff61014dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.133+0000 7ff60f59e700 1 -- 172.21.15.7:0/346708115 >> 172.21.15.7:0/346708115 conn(0x7ff6100b0e10 msgr2=0x7ff6100aee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.133+0000 7ff60f59e700 1 -- 172.21.15.7:0/346708115 shutdown_connections 2024-09-13T18:52:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.133+0000 7ff60f59e700 1 -- 172.21.15.7:0/346708115 wait complete. 2024-09-13T18:52:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-13T18:52:53.155 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2024-09-13T18:52:53.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:53.155 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T18:52:53.177 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:52:53.177 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-13T18:52:53.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:53.177 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T18:52:53.216 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:52:53.216 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:52:53.224 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:52:53.224 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:52:53.231 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:53.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.345+0000 7f1f8a559700 1 Processor -- start 2024-09-13T18:52:53.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.345+0000 7f1f8a559700 1 -- start start 2024-09-13T18:52:53.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f8a559700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f84159720 0x7f1f8406cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f8a559700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f1f8406aa90 0x7f1f840b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f8a559700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8406af00 con 0x7f1f840b3910 2024-09-13T18:52:53.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f8a559700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f840b3da0 con 0x7f1f84159720 2024-09-13T18:52:53.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f8a559700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f8406d690 con 0x7f1f8406aa90 2024-09-13T18:52:53.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f89557700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f84159720 0x7f1f8406cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f89557700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f84159720 0x7f1f8406cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39016/0 (socket says 172.21.15.7:39016) 2024-09-13T18:52:53.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f89557700 1 -- 172.21.15.7:0/1207221339 learned_addr learned my addr 172.21.15.7:0/1207221339 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:53.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f88d56700 1 --2- 172.21.15.7:0/1207221339 >> v2:172.21.15.171:3300/0 conn(0x7f1f8406aa90 0x7f1f840b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f89557700 1 -- 172.21.15.7:0/1207221339 >> v2:172.21.15.171:3300/0 conn(0x7f1f8406aa90 msgr2=0x7f1f840b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:53.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f89557700 1 --2- 172.21.15.7:0/1207221339 >> v2:172.21.15.171:3300/0 conn(0x7f1f8406aa90 0x7f1f840b33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f89557700 1 -- 172.21.15.7:0/1207221339 >> v1:172.21.15.171:6789/0 conn(0x7f1f840b3910 legacy=0x7f1f84070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f89557700 1 -- 172.21.15.7:0/1207221339 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f840cbfe0 con 0x7f1f84159720 2024-09-13T18:52:53.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f89557700 1 --2- 172.21.15.7:0/1207221339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f84159720 0x7f1f8406cf90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1f8406b160 tx=0x7f1f80009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa37983a46fbc335 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:53.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f77fff700 1 -- 172.21.15.7:0/1207221339 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80011830 con 0x7f1f84159720 2024-09-13T18:52:53.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f77fff700 1 -- 172.21.15.7:0/1207221339 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f80011990 con 0x7f1f84159720 2024-09-13T18:52:53.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f77fff700 1 -- 172.21.15.7:0/1207221339 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80011d20 con 0x7f1f84159720 2024-09-13T18:52:53.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f8a559700 1 -- 172.21.15.7:0/1207221339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f84159720 msgr2=0x7f1f8406cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f8a559700 1 --2- 172.21.15.7:0/1207221339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f84159720 0x7f1f8406cf90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1f8406b160 tx=0x7f1f80009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f8a559700 1 -- 172.21.15.7:0/1207221339 shutdown_connections 2024-09-13T18:52:53.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f8a559700 1 --2- 172.21.15.7:0/1207221339 >> v2:172.21.15.171:3300/0 conn(0x7f1f8406aa90 0x7f1f840b33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f8a559700 1 --2- 172.21.15.7:0/1207221339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f84159720 0x7f1f8406cf90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f8a559700 1 -- 172.21.15.7:0/1207221339 >> 172.21.15.7:0/1207221339 conn(0x7f1f840b0f00 msgr2=0x7f1f840b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:53.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.349+0000 7f1f8a559700 1 -- 172.21.15.7:0/1207221339 shutdown_connections 2024-09-13T18:52:53.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.353+0000 7f1f8a559700 1 -- 172.21.15.7:0/1207221339 wait complete. 2024-09-13T18:52:53.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.353+0000 7f1f8a559700 1 Processor -- start 2024-09-13T18:52:53.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.353+0000 7f1f8a559700 1 -- start start 2024-09-13T18:52:53.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.353+0000 7f1f8a559700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1f8406aa90 0x7f1f8415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.353+0000 7f1f8a559700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1f840b3910 0x7f1f841454b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.353+0000 7f1f8a559700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f84159720 0x7f1f841459f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.353+0000 7f1f8a559700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8415cd90 con 0x7f1f84159720 2024-09-13T18:52:53.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.353+0000 7f1f8a559700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f84159db0 con 0x7f1f8406aa90 2024-09-13T18:52:53.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.353+0000 7f1f8a559700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8415a090 con 0x7f1f840b3910 2024-09-13T18:52:53.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.353+0000 7f1f89d58700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f84159720 0x7f1f841459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.353+0000 7f1f88d56700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1f840b3910 0x7f1f841454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.353+0000 7f1f89d58700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f84159720 0x7f1f841459f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39022/0 (socket says 172.21.15.7:39022) 2024-09-13T18:52:53.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.357+0000 7f1f89d58700 1 -- 172.21.15.7:0/637642345 learned_addr learned my addr 172.21.15.7:0/637642345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:53.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.357+0000 7f1f89557700 1 --2- 172.21.15.7:0/637642345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1f8406aa90 0x7f1f8415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.357+0000 7f1f89d58700 1 -- 172.21.15.7:0/637642345 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1f840b3910 msgr2=0x7f1f841454b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.357+0000 7f1f89d58700 1 --2- 172.21.15.7:0/637642345 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1f840b3910 0x7f1f841454b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.357+0000 7f1f89d58700 1 -- 172.21.15.7:0/637642345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1f8406aa90 msgr2=0x7f1f8415ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.357+0000 7f1f89d58700 1 --2- 172.21.15.7:0/637642345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1f8406aa90 0x7f1f8415ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.357+0000 7f1f89d58700 1 -- 172.21.15.7:0/637642345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f80009570 con 0x7f1f84159720 2024-09-13T18:52:53.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.357+0000 7f1f89d58700 1 --2- 172.21.15.7:0/637642345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f84159720 0x7f1f841459f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00cc50 tx=0x7f1f7c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:53.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.357+0000 7f1f75ffb700 1 -- 172.21.15.7:0/637642345 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c003c20 con 0x7f1f84159720 2024-09-13T18:52:53.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.357+0000 7f1f8a559700 1 -- 172.21.15.7:0/637642345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8415a370 con 0x7f1f84159720 2024-09-13T18:52:53.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.357+0000 7f1f75ffb700 1 -- 172.21.15.7:0/637642345 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c003d80 con 0x7f1f84159720 2024-09-13T18:52:53.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.357+0000 7f1f8a559700 1 -- 172.21.15.7:0/637642345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8415a930 con 0x7f1f84159720 2024-09-13T18:52:53.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.357+0000 7f1f75ffb700 1 -- 172.21.15.7:0/637642345 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c00aa70 con 0x7f1f84159720 2024-09-13T18:52:53.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.357+0000 7f1f75ffb700 1 -- 172.21.15.7:0/637642345 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1f7c00abd0 con 0x7f1f84159720 2024-09-13T18:52:53.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.357+0000 7f1f75ffb700 1 --2- 172.21.15.7:0/637642345 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1f68066730 0x7f1f68068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.357+0000 7f1f75ffb700 1 -- 172.21.15.7:0/637642345 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f7c084170 con 0x7f1f84159720 2024-09-13T18:52:53.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.357+0000 7f1f8a559700 1 -- 172.21.15.7:0/637642345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8415ab50 con 0x7f1f84159720 2024-09-13T18:52:53.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.361+0000 7f1f89557700 1 --2- 172.21.15.7:0/637642345 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1f68066730 0x7f1f68068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.361+0000 7f1f89557700 1 --2- 172.21.15.7:0/637642345 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1f68066730 0x7f1f68068bf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1f80000c00 tx=0x7f1f80008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:53.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.365+0000 7f1f75ffb700 1 -- 172.21.15.7:0/637642345 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1f7c0514f0 con 0x7f1f84159720 2024-09-13T18:52:53.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.549+0000 7f1f8a559700 1 -- 172.21.15.7:0/637642345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f840b33d0 con 0x7f1f84159720 2024-09-13T18:52:53.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.549+0000 7f1f75ffb700 1 -- 172.21.15.7:0/637642345 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f1f7c054b10 con 0x7f1f84159720 2024-09-13T18:52:53.557 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:53.558 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.553+0000 7f1f8a559700 1 -- 172.21.15.7:0/637642345 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1f68066730 msgr2=0x7f1f68068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.553+0000 7f1f8a559700 1 --2- 172.21.15.7:0/637642345 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1f68066730 0x7f1f68068bf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1f80000c00 tx=0x7f1f80008030 comp rx=0 tx=0).stop 2024-09-13T18:52:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.553+0000 7f1f8a559700 1 -- 172.21.15.7:0/637642345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f84159720 msgr2=0x7f1f841459f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.553+0000 7f1f8a559700 1 --2- 172.21.15.7:0/637642345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f84159720 0x7f1f841459f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00cc50 tx=0x7f1f7c00cf60 comp rx=0 tx=0).stop 2024-09-13T18:52:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.553+0000 7f1f8a559700 1 -- 172.21.15.7:0/637642345 shutdown_connections 2024-09-13T18:52:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.553+0000 7f1f8a559700 1 --2- 172.21.15.7:0/637642345 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1f68066730 0x7f1f68068bf0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.553+0000 7f1f8a559700 1 --2- 172.21.15.7:0/637642345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f84159720 0x7f1f841459f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.553+0000 7f1f8a559700 1 --2- 172.21.15.7:0/637642345 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1f840b3910 0x7f1f841454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.553+0000 7f1f8a559700 1 --2- 172.21.15.7:0/637642345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1f8406aa90 0x7f1f8415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.553+0000 7f1f8a559700 1 -- 172.21.15.7:0/637642345 >> 172.21.15.7:0/637642345 conn(0x7f1f840b0f00 msgr2=0x7f1f840b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.553+0000 7f1f8a559700 1 -- 172.21.15.7:0/637642345 shutdown_connections 2024-09-13T18:52:53.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.553+0000 7f1f8a559700 1 -- 172.21.15.7:0/637642345 wait complete. 2024-09-13T18:52:53.573 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:52:15.554683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:52:16.896879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:52:53.574 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:52:53.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.693+0000 7fb7b8a17700 1 Processor -- start 2024-09-13T18:52:53.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.693+0000 7fb7b8a17700 1 -- start start 2024-09-13T18:52:53.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.693+0000 7fb7b8a17700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7b415cda0 0x7fb7b406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.693+0000 7fb7b8a17700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fb7b406a520 0x7fb7b406a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.693+0000 7fb7b8a17700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b40b8430 con 0x7fb7b40ceb70 2024-09-13T18:52:53.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.693+0000 7fb7b8a17700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b406be40 con 0x7fb7b415cda0 2024-09-13T18:52:53.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.693+0000 7fb7b8a17700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b40b64f0 con 0x7fb7b406a520 2024-09-13T18:52:53.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.693+0000 7fb7b2d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7b415cda0 0x7fb7b406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.693+0000 7fb7b2d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7b415cda0 0x7fb7b406b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39032/0 (socket says 172.21.15.7:39032) 2024-09-13T18:52:53.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.693+0000 7fb7b2d9d700 1 -- 172.21.15.7:0/845314424 learned_addr learned my addr 172.21.15.7:0/845314424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:53.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.693+0000 7fb7b359e700 1 --2- 172.21.15.7:0/845314424 >> v2:172.21.15.171:3300/0 conn(0x7fb7b406a520 0x7fb7b406a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.697+0000 7fb7b2d9d700 1 -- 172.21.15.7:0/845314424 >> v2:172.21.15.171:3300/0 conn(0x7fb7b406a520 msgr2=0x7fb7b406a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:53.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.697+0000 7fb7b2d9d700 1 --2- 172.21.15.7:0/845314424 >> v2:172.21.15.171:3300/0 conn(0x7fb7b406a520 0x7fb7b406a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.697+0000 7fb7b2d9d700 1 -- 172.21.15.7:0/845314424 >> v1:172.21.15.171:6789/0 conn(0x7fb7b40ceb70 legacy=0x7fb7b40b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.697+0000 7fb7b2d9d700 1 -- 172.21.15.7:0/845314424 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b406ae70 con 0x7fb7b415cda0 2024-09-13T18:52:53.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.697+0000 7fb7b2d9d700 1 --2- 172.21.15.7:0/845314424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b415cda0 0x7fb7b406b900 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40d2c60 tx=0x7fb7a8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7403d54b7f6b0ac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:53.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.697+0000 7fb7b1d9b700 1 -- 172.21.15.7:0/845314424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a800b8a0 con 0x7fb7b415cda0 2024-09-13T18:52:53.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.697+0000 7fb7b1d9b700 1 -- 172.21.15.7:0/845314424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a800ba00 con 0x7fb7b415cda0 2024-09-13T18:52:53.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.697+0000 7fb7b1d9b700 1 -- 172.21.15.7:0/845314424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a800bd90 con 0x7fb7b415cda0 2024-09-13T18:52:53.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.697+0000 7fb7b8a17700 1 -- 172.21.15.7:0/845314424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b415cda0 msgr2=0x7fb7b406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.697+0000 7fb7b8a17700 1 --2- 172.21.15.7:0/845314424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b415cda0 0x7fb7b406b900 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40d2c60 tx=0x7fb7a8009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.697+0000 7fb7b8a17700 1 -- 172.21.15.7:0/845314424 shutdown_connections 2024-09-13T18:52:53.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.697+0000 7fb7b8a17700 1 --2- 172.21.15.7:0/845314424 >> v2:172.21.15.171:3300/0 conn(0x7fb7b406a520 0x7fb7b406a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.697+0000 7fb7b8a17700 1 --2- 172.21.15.7:0/845314424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b415cda0 0x7fb7b406b900 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.697+0000 7fb7b8a17700 1 -- 172.21.15.7:0/845314424 >> 172.21.15.7:0/845314424 conn(0x7fb7b40d2cb0 msgr2=0x7fb7b40d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:53.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.697+0000 7fb7b8a17700 1 -- 172.21.15.7:0/845314424 shutdown_connections 2024-09-13T18:52:53.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.697+0000 7fb7b8a17700 1 -- 172.21.15.7:0/845314424 wait complete. 2024-09-13T18:52:53.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.697+0000 7fb7b8a17700 1 Processor -- start 2024-09-13T18:52:53.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.697+0000 7fb7b8a17700 1 -- start start 2024-09-13T18:52:53.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b8a17700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b406a520 0x7fb7b4154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b8a17700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7b40ceb70 0x7fb7b40b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b8a17700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb7b415cda0 0x7fb7b40c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b8a17700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40c6580 con 0x7fb7b406a520 2024-09-13T18:52:53.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b8a17700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40b3f80 con 0x7fb7b40ceb70 2024-09-13T18:52:53.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b8a17700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40c8bc0 con 0x7fb7b415cda0 2024-09-13T18:52:53.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b406a520 0x7fb7b4154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b406a520 0x7fb7b4154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39034/0 (socket says 172.21.15.7:39034) 2024-09-13T18:52:53.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b2d9d700 1 -- 172.21.15.7:0/2558551284 learned_addr learned my addr 172.21.15.7:0/2558551284 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:53.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b359e700 1 --2- 172.21.15.7:0/2558551284 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb7b415cda0 0x7fb7b40c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b259c700 1 --2- 172.21.15.7:0/2558551284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7b40ceb70 0x7fb7b40b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b2d9d700 1 -- 172.21.15.7:0/2558551284 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb7b415cda0 msgr2=0x7fb7b40c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b2d9d700 1 --2- 172.21.15.7:0/2558551284 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb7b415cda0 0x7fb7b40c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b2d9d700 1 -- 172.21.15.7:0/2558551284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7b40ceb70 msgr2=0x7fb7b40b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b2d9d700 1 --2- 172.21.15.7:0/2558551284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7b40ceb70 0x7fb7b40b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b2d9d700 1 -- 172.21.15.7:0/2558551284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a8009570 con 0x7fb7b406a520 2024-09-13T18:52:53.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b259c700 1 --2- 172.21.15.7:0/2558551284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7b40ceb70 0x7fb7b40b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:53.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b359e700 1 --2- 172.21.15.7:0/2558551284 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb7b415cda0 0x7fb7b40c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:52:53.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b2d9d700 1 --2- 172.21.15.7:0/2558551284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b406a520 0x7fb7b4154ce0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8000c00 tx=0x7fb7a8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:53.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb79b7fe700 1 -- 172.21.15.7:0/2558551284 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8003ad0 con 0x7fb7b406a520 2024-09-13T18:52:53.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb79b7fe700 1 -- 172.21.15.7:0/2558551284 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a801e070 con 0x7fb7b406a520 2024-09-13T18:52:53.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b8a17700 1 -- 172.21.15.7:0/2558551284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b40c9990 con 0x7fb7b406a520 2024-09-13T18:52:53.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb79b7fe700 1 -- 172.21.15.7:0/2558551284 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8003e70 con 0x7fb7b406a520 2024-09-13T18:52:53.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.701+0000 7fb7b8a17700 1 -- 172.21.15.7:0/2558551284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b40c90f0 con 0x7fb7b406a520 2024-09-13T18:52:53.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.705+0000 7fb7b8a17700 1 -- 172.21.15.7:0/2558551284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b40d1120 con 0x7fb7b406a520 2024-09-13T18:52:53.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.705+0000 7fb79b7fe700 1 -- 172.21.15.7:0/2558551284 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb7a8025480 con 0x7fb7b406a520 2024-09-13T18:52:53.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.705+0000 7fb79b7fe700 1 --2- 172.21.15.7:0/2558551284 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb7a0066730 0x7fb7a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:53.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.705+0000 7fb79b7fe700 1 -- 172.21.15.7:0/2558551284 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7a8020070 con 0x7fb7b406a520 2024-09-13T18:52:53.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.705+0000 7fb7b259c700 1 --2- 172.21.15.7:0/2558551284 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb7a0066730 0x7fb7a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:53.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.705+0000 7fb7b259c700 1 --2- 172.21.15.7:0/2558551284 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb7a0066730 0x7fb7a0068bf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb7b41558d0 tx=0x7fb7a4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:53.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.713+0000 7fb79b7fe700 1 -- 172.21.15.7:0/2558551284 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb7a8052c20 con 0x7fb7b406a520 2024-09-13T18:52:53.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.881+0000 7fb7b8a17700 1 -- 172.21.15.7:0/2558551284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7b4144d60 con 0x7fb7b406a520 2024-09-13T18:52:53.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.885+0000 7fb79b7fe700 1 -- 172.21.15.7:0/2558551284 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7a8085020 con 0x7fb7b406a520 2024-09-13T18:52:53.889 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:52:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.889+0000 7fb7b8a17700 1 -- 172.21.15.7:0/2558551284 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb7a0066730 msgr2=0x7fb7a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.889+0000 7fb7b8a17700 1 --2- 172.21.15.7:0/2558551284 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb7a0066730 0x7fb7a0068bf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb7b41558d0 tx=0x7fb7a4009040 comp rx=0 tx=0).stop 2024-09-13T18:52:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.889+0000 7fb7b8a17700 1 -- 172.21.15.7:0/2558551284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b406a520 msgr2=0x7fb7b4154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.889+0000 7fb7b8a17700 1 --2- 172.21.15.7:0/2558551284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b406a520 0x7fb7b4154ce0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8000c00 tx=0x7fb7a8004770 comp rx=0 tx=0).stop 2024-09-13T18:52:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.889+0000 7fb7b8a17700 1 -- 172.21.15.7:0/2558551284 shutdown_connections 2024-09-13T18:52:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.889+0000 7fb7b8a17700 1 --2- 172.21.15.7:0/2558551284 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb7a0066730 0x7fb7a0068bf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.889+0000 7fb7b8a17700 1 --2- 172.21.15.7:0/2558551284 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb7b415cda0 0x7fb7b40c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.889+0000 7fb7b8a17700 1 --2- 172.21.15.7:0/2558551284 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7b40ceb70 0x7fb7b40b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.889+0000 7fb7b8a17700 1 --2- 172.21.15.7:0/2558551284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7b406a520 0x7fb7b4154ce0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.889+0000 7fb7b8a17700 1 -- 172.21.15.7:0/2558551284 >> 172.21.15.7:0/2558551284 conn(0x7fb7b40d2cb0 msgr2=0x7fb7b415bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.889+0000 7fb7b8a17700 1 -- 172.21.15.7:0/2558551284 shutdown_connections 2024-09-13T18:52:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:53.889+0000 7fb7b8a17700 1 -- 172.21.15.7:0/2558551284 wait complete. 2024-09-13T18:52:53.910 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:52:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa83d4a4700 1 Processor -- start 2024-09-13T18:52:54.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa83d4a4700 1 -- start start 2024-09-13T18:52:54.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa83d4a4700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa838158f50 0x7fa83806a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa83d4a4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa8380b4850 0x7fa83806d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa83d4a4700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa83806d780 0x7fa8380cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa83d4a4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8380ce770 con 0x7fa8380b4850 2024-09-13T18:52:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa83d4a4700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8380ce1b0 con 0x7fa838158f50 2024-09-13T18:52:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa83d4a4700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8380ce490 con 0x7fa83806d780 2024-09-13T18:52:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa8367fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa8380b4850 0x7fa83806d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa8367fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa8380b4850 0x7fa83806d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39046/0 (socket says 172.21.15.7:39046) 2024-09-13T18:52:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa8367fc700 1 -- 172.21.15.7:0/3584068664 learned_addr learned my addr 172.21.15.7:0/3584068664 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa8377fe700 1 --2- 172.21.15.7:0/3584068664 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa83806d780 0x7fa8380cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa836ffd700 1 --2- 172.21.15.7:0/3584068664 >> v2:172.21.15.171:3300/0 conn(0x7fa838158f50 0x7fa83806a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa8367fc700 1 -- 172.21.15.7:0/3584068664 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa83806d780 msgr2=0x7fa8380cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa8367fc700 1 --2- 172.21.15.7:0/3584068664 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa83806d780 0x7fa8380cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa8367fc700 1 -- 172.21.15.7:0/3584068664 >> v2:172.21.15.171:3300/0 conn(0x7fa838158f50 msgr2=0x7fa83806a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:54.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa8367fc700 1 --2- 172.21.15.7:0/3584068664 >> v2:172.21.15.171:3300/0 conn(0x7fa838158f50 0x7fa83806a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa8367fc700 1 -- 172.21.15.7:0/3584068664 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83806ca40 con 0x7fa8380b4850 2024-09-13T18:52:54.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.029+0000 7fa8367fc700 1 --2- 172.21.15.7:0/3584068664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8380b4850 0x7fa83806d240 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa838072110 tx=0x7fa828009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04e47a3c54d0791 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:54.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.033+0000 7fa835ffb700 1 -- 172.21.15.7:0/3584068664 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828011830 con 0x7fa8380b4850 2024-09-13T18:52:54.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.033+0000 7fa835ffb700 1 -- 172.21.15.7:0/3584068664 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa828011990 con 0x7fa8380b4850 2024-09-13T18:52:54.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.033+0000 7fa835ffb700 1 -- 172.21.15.7:0/3584068664 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa828011d40 con 0x7fa8380b4850 2024-09-13T18:52:54.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.033+0000 7fa83d4a4700 1 -- 172.21.15.7:0/3584068664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8380b4850 msgr2=0x7fa83806d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:54.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.033+0000 7fa83d4a4700 1 --2- 172.21.15.7:0/3584068664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8380b4850 0x7fa83806d240 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa838072110 tx=0x7fa828009c20 comp rx=0 tx=0).stop 2024-09-13T18:52:54.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.033+0000 7fa83d4a4700 1 -- 172.21.15.7:0/3584068664 shutdown_connections 2024-09-13T18:52:54.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.033+0000 7fa83d4a4700 1 --2- 172.21.15.7:0/3584068664 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa83806d780 0x7fa8380cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.033+0000 7fa83d4a4700 1 --2- 172.21.15.7:0/3584068664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8380b4850 0x7fa83806d240 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.033+0000 7fa83d4a4700 1 --2- 172.21.15.7:0/3584068664 >> v2:172.21.15.171:3300/0 conn(0x7fa838158f50 0x7fa83806a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.033+0000 7fa83d4a4700 1 -- 172.21.15.7:0/3584068664 >> 172.21.15.7:0/3584068664 conn(0x7fa8380af620 msgr2=0x7fa8380afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:54.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.033+0000 7fa83d4a4700 1 -- 172.21.15.7:0/3584068664 shutdown_connections 2024-09-13T18:52:54.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.033+0000 7fa83d4a4700 1 -- 172.21.15.7:0/3584068664 wait complete. 2024-09-13T18:52:54.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.033+0000 7fa83d4a4700 1 Processor -- start 2024-09-13T18:52:54.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.033+0000 7fa83d4a4700 1 -- start start 2024-09-13T18:52:54.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa83d4a4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa83806d780 0x7fa838148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa83d4a4700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa8380b4850 0x7fa838152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa83d4a4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa838158f50 0x7fa838153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa83d4a4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83815b7d0 con 0x7fa838158f50 2024-09-13T18:52:54.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa83d4a4700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa838154940 con 0x7fa83806d780 2024-09-13T18:52:54.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa83d4a4700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa838140390 con 0x7fa8380b4850 2024-09-13T18:52:54.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa8377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa838158f50 0x7fa838153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa8377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa838158f50 0x7fa838153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39050/0 (socket says 172.21.15.7:39050) 2024-09-13T18:52:54.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa8377fe700 1 -- 172.21.15.7:0/1768013837 learned_addr learned my addr 172.21.15.7:0/1768013837 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:54.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa836ffd700 1 --2- 172.21.15.7:0/1768013837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa83806d780 0x7fa838148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa8367fc700 1 --2- 172.21.15.7:0/1768013837 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa8380b4850 0x7fa838152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa8377fe700 1 -- 172.21.15.7:0/1768013837 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa8380b4850 msgr2=0x7fa838152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:54.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa8377fe700 1 --2- 172.21.15.7:0/1768013837 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa8380b4850 0x7fa838152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa8377fe700 1 -- 172.21.15.7:0/1768013837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa83806d780 msgr2=0x7fa838148520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:54.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa8377fe700 1 --2- 172.21.15.7:0/1768013837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa83806d780 0x7fa838148520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa8377fe700 1 -- 172.21.15.7:0/1768013837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa828009570 con 0x7fa838158f50 2024-09-13T18:52:54.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa836ffd700 1 --2- 172.21.15.7:0/1768013837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa83806d780 0x7fa838148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:54.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa8377fe700 1 --2- 172.21.15.7:0/1768013837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa838158f50 0x7fa838153500 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa83000efc0 tx=0x7fa83000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:54.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa81f7fe700 1 -- 172.21.15.7:0/1768013837 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83000cd40 con 0x7fa838158f50 2024-09-13T18:52:54.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa81f7fe700 1 -- 172.21.15.7:0/1768013837 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83000f040 con 0x7fa838158f50 2024-09-13T18:52:54.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa83d4a4700 1 -- 172.21.15.7:0/1768013837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa838154380 con 0x7fa838158f50 2024-09-13T18:52:54.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa81f7fe700 1 -- 172.21.15.7:0/1768013837 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8300146c0 con 0x7fa838158f50 2024-09-13T18:52:54.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa83d4a4700 1 -- 172.21.15.7:0/1768013837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa838140040 con 0x7fa838158f50 2024-09-13T18:52:54.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.037+0000 7fa83d4a4700 1 -- 172.21.15.7:0/1768013837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa83815b940 con 0x7fa838158f50 2024-09-13T18:52:54.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.041+0000 7fa81f7fe700 1 -- 172.21.15.7:0/1768013837 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa830014820 con 0x7fa838158f50 2024-09-13T18:52:54.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.041+0000 7fa81f7fe700 1 --2- 172.21.15.7:0/1768013837 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa824066730 0x7fa824068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.041+0000 7fa81f7fe700 1 -- 172.21.15.7:0/1768013837 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa830084a60 con 0x7fa838158f50 2024-09-13T18:52:54.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.041+0000 7fa836ffd700 1 --2- 172.21.15.7:0/1768013837 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa824066730 0x7fa824068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.041+0000 7fa836ffd700 1 --2- 172.21.15.7:0/1768013837 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa824066730 0x7fa824068bf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009b20 tx=0x7fa82c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:54.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.049+0000 7fa81f7fe700 1 -- 172.21.15.7:0/1768013837 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa83815b940 con 0x7fa838158f50 2024-09-13T18:52:54.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.217+0000 7fa83d4a4700 1 -- 172.21.15.7:0/1768013837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa83815b910 con 0x7fa838158f50 2024-09-13T18:52:54.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.221+0000 7fa81f7fe700 1 -- 172.21.15.7:0/1768013837 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa830055870 con 0x7fa838158f50 2024-09-13T18:52:54.226 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:52:54.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.221+0000 7fa83d4a4700 1 -- 172.21.15.7:0/1768013837 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa824066730 msgr2=0x7fa824068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.221+0000 7fa83d4a4700 1 --2- 172.21.15.7:0/1768013837 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa824066730 0x7fa824068bf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009b20 tx=0x7fa82c008040 comp rx=0 tx=0).stop 2024-09-13T18:52:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.221+0000 7fa83d4a4700 1 -- 172.21.15.7:0/1768013837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa838158f50 msgr2=0x7fa838153500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.221+0000 7fa83d4a4700 1 --2- 172.21.15.7:0/1768013837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa838158f50 0x7fa838153500 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa83000efc0 tx=0x7fa83000c3a0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.221+0000 7fa83d4a4700 1 -- 172.21.15.7:0/1768013837 shutdown_connections 2024-09-13T18:52:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.221+0000 7fa83d4a4700 1 --2- 172.21.15.7:0/1768013837 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa824066730 0x7fa824068bf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.221+0000 7fa83d4a4700 1 --2- 172.21.15.7:0/1768013837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa838158f50 0x7fa838153500 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.221+0000 7fa83d4a4700 1 --2- 172.21.15.7:0/1768013837 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa8380b4850 0x7fa838152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.221+0000 7fa83d4a4700 1 --2- 172.21.15.7:0/1768013837 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa83806d780 0x7fa838148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.221+0000 7fa83d4a4700 1 -- 172.21.15.7:0/1768013837 >> 172.21.15.7:0/1768013837 conn(0x7fa8380af620 msgr2=0x7fa83814a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.221+0000 7fa83d4a4700 1 -- 172.21.15.7:0/1768013837 shutdown_connections 2024-09-13T18:52:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.221+0000 7fa83d4a4700 1 -- 172.21.15.7:0/1768013837 wait complete. 2024-09-13T18:52:54.241 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:52:54.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.361+0000 7fc5faa6a700 1 Processor -- start 2024-09-13T18:52:54.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.361+0000 7fc5faa6a700 1 -- start start 2024-09-13T18:52:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.361+0000 7fc5faa6a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5ec057cc0 0x7fc5ec0061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.361+0000 7fc5faa6a700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fc5ec03d590 0x7fc5ec006730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.361+0000 7fc5faa6a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ec053d60 con 0x7fc5ec005e10 2024-09-13T18:52:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.361+0000 7fc5faa6a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ec03c0f0 con 0x7fc5ec057cc0 2024-09-13T18:52:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.361+0000 7fc5faa6a700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ec058190 con 0x7fc5ec03d590 2024-09-13T18:52:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5f9267700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5ec057cc0 0x7fc5ec0061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5f9a68700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc5ec005e10 0x7fc5ec0575b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38776/0 (socket says 172.21.15.7:38776) 2024-09-13T18:52:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5f9a68700 1 -- 172.21.15.7:0/923579366 learned_addr learned my addr 172.21.15.7:0/923579366 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5fa269700 1 --2- 172.21.15.7:0/923579366 >> v2:172.21.15.171:3300/0 conn(0x7fc5ec03d590 0x7fc5ec006730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5f9267700 1 -- 172.21.15.7:0/923579366 >> v2:172.21.15.171:3300/0 conn(0x7fc5ec03d590 msgr2=0x7fc5ec006730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:54.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5f9267700 1 --2- 172.21.15.7:0/923579366 >> v2:172.21.15.171:3300/0 conn(0x7fc5ec03d590 0x7fc5ec006730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5f9267700 1 -- 172.21.15.7:0/923579366 >> v1:172.21.15.7:6789/0 conn(0x7fc5ec005e10 legacy=0x7fc5ec0575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:54.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5f8a66700 1 -- 172.21.15.7:0/923579366 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237761099 0 0) 0x7fc5ec053d60 con 0x7fc5ec005e10 2024-09-13T18:52:54.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5f9267700 1 -- 172.21.15.7:0/923579366 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec03bb50 con 0x7fc5ec057cc0 2024-09-13T18:52:54.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5fa269700 1 --2- 172.21.15.7:0/923579366 >> v2:172.21.15.171:3300/0 conn(0x7fc5ec03d590 0x7fc5ec006730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:54.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5f9267700 1 --2- 172.21.15.7:0/923579366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec057cc0 0x7fc5ec0061f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec03b9f0 tx=0x7fc5f0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836c6ecbfa0cbd52 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:54.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5f8a66700 1 -- 172.21.15.7:0/923579366 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0011830 con 0x7fc5ec057cc0 2024-09-13T18:52:54.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5f8a66700 1 -- 172.21.15.7:0/923579366 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f0011990 con 0x7fc5ec057cc0 2024-09-13T18:52:54.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5f8a66700 1 -- 172.21.15.7:0/923579366 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0011d40 con 0x7fc5ec057cc0 2024-09-13T18:52:54.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5faa6a700 1 -- 172.21.15.7:0/923579366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec057cc0 msgr2=0x7fc5ec0061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:54.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5faa6a700 1 --2- 172.21.15.7:0/923579366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec057cc0 0x7fc5ec0061f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec03b9f0 tx=0x7fc5f0009b30 comp rx=0 tx=0).stop 2024-09-13T18:52:54.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5faa6a700 1 -- 172.21.15.7:0/923579366 shutdown_connections 2024-09-13T18:52:54.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5faa6a700 1 --2- 172.21.15.7:0/923579366 >> v2:172.21.15.171:3300/0 conn(0x7fc5ec03d590 0x7fc5ec006730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5faa6a700 1 --2- 172.21.15.7:0/923579366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec057cc0 0x7fc5ec0061f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5faa6a700 1 -- 172.21.15.7:0/923579366 >> 172.21.15.7:0/923579366 conn(0x7fc5ec03a550 msgr2=0x7fc5ec03a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:54.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5faa6a700 1 -- 172.21.15.7:0/923579366 shutdown_connections 2024-09-13T18:52:54.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5faa6a700 1 -- 172.21.15.7:0/923579366 wait complete. 2024-09-13T18:52:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5faa6a700 1 Processor -- start 2024-09-13T18:52:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.365+0000 7fc5faa6a700 1 -- start start 2024-09-13T18:52:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5faa6a700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5ec005e10 0x7fc5ec0dc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5faa6a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec03d590 0x7fc5ec04ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5faa6a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5ec057cc0 0x7fc5ec00e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5faa6a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec04eb40 con 0x7fc5ec03d590 2024-09-13T18:52:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5faa6a700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec0dfcb0 con 0x7fc5ec057cc0 2024-09-13T18:52:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5faa6a700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec17c680 con 0x7fc5ec005e10 2024-09-13T18:52:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5f9267700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec03d590 0x7fc5ec04ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5f9267700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec03d590 0x7fc5ec04ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39062/0 (socket says 172.21.15.7:39062) 2024-09-13T18:52:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5f9267700 1 -- 172.21.15.7:0/3952744315 learned_addr learned my addr 172.21.15.7:0/3952744315 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5fa269700 1 --2- 172.21.15.7:0/3952744315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5ec057cc0 0x7fc5ec00e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5f9a68700 1 --2- 172.21.15.7:0/3952744315 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5ec005e10 0x7fc5ec0dc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5f9267700 1 -- 172.21.15.7:0/3952744315 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5ec005e10 msgr2=0x7fc5ec0dc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:54.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5f9267700 1 --2- 172.21.15.7:0/3952744315 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5ec005e10 0x7fc5ec0dc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5f9267700 1 -- 172.21.15.7:0/3952744315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5ec057cc0 msgr2=0x7fc5ec00e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:54.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5f9267700 1 --2- 172.21.15.7:0/3952744315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5ec057cc0 0x7fc5ec00e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5f9267700 1 -- 172.21.15.7:0/3952744315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f00094b0 con 0x7fc5ec03d590 2024-09-13T18:52:54.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5f9267700 1 --2- 172.21.15.7:0/3952744315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec03d590 0x7fc5ec04ef60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0014040 tx=0x7fc5f0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:54.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5e67fc700 1 -- 172.21.15.7:0/3952744315 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0019a00 con 0x7fc5ec03d590 2024-09-13T18:52:54.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5faa6a700 1 -- 172.21.15.7:0/3952744315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ec17c960 con 0x7fc5ec03d590 2024-09-13T18:52:54.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5e67fc700 1 -- 172.21.15.7:0/3952744315 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f001e070 con 0x7fc5ec03d590 2024-09-13T18:52:54.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5e67fc700 1 -- 172.21.15.7:0/3952744315 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0019dc0 con 0x7fc5ec03d590 2024-09-13T18:52:54.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.369+0000 7fc5faa6a700 1 -- 172.21.15.7:0/3952744315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ec04e760 con 0x7fc5ec03d590 2024-09-13T18:52:54.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.373+0000 7fc5faa6a700 1 -- 172.21.15.7:0/3952744315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ec04ec80 con 0x7fc5ec03d590 2024-09-13T18:52:54.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.373+0000 7fc5e67fc700 1 -- 172.21.15.7:0/3952744315 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc5f002d450 con 0x7fc5ec03d590 2024-09-13T18:52:54.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.373+0000 7fc5e67fc700 1 --2- 172.21.15.7:0/3952744315 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc5d8066730 0x7fc5d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.373+0000 7fc5e67fc700 1 -- 172.21.15.7:0/3952744315 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5f0085270 con 0x7fc5ec03d590 2024-09-13T18:52:54.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.373+0000 7fc5f9a68700 1 --2- 172.21.15.7:0/3952744315 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc5d8066730 0x7fc5d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.373+0000 7fc5f9a68700 1 --2- 172.21.15.7:0/3952744315 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc5d8066730 0x7fc5d8068bf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8000f60 tx=0x7fc5e8007e00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:54.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.381+0000 7fc5e67fc700 1 -- 172.21.15.7:0/3952744315 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc5ec04ec80 con 0x7fc5ec03d590 2024-09-13T18:52:54.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.549+0000 7fc5faa6a700 1 -- 172.21.15.7:0/3952744315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc5ec04ec80 con 0x7fc5ec03d590 2024-09-13T18:52:54.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.549+0000 7fc5e67fc700 1 -- 172.21.15.7:0/3952744315 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc5ec04ec80 con 0x7fc5ec03d590 2024-09-13T18:52:54.556 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:52:54.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.553+0000 7fc5faa6a700 1 -- 172.21.15.7:0/3952744315 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc5d8066730 msgr2=0x7fc5d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:54.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.553+0000 7fc5faa6a700 1 --2- 172.21.15.7:0/3952744315 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc5d8066730 0x7fc5d8068bf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8000f60 tx=0x7fc5e8007e00 comp rx=0 tx=0).stop 2024-09-13T18:52:54.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.553+0000 7fc5faa6a700 1 -- 172.21.15.7:0/3952744315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec03d590 msgr2=0x7fc5ec04ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:54.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.553+0000 7fc5faa6a700 1 --2- 172.21.15.7:0/3952744315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec03d590 0x7fc5ec04ef60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0014040 tx=0x7fc5f0004770 comp rx=0 tx=0).stop 2024-09-13T18:52:54.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.553+0000 7fc5faa6a700 1 -- 172.21.15.7:0/3952744315 shutdown_connections 2024-09-13T18:52:54.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.553+0000 7fc5faa6a700 1 --2- 172.21.15.7:0/3952744315 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc5d8066730 0x7fc5d8068bf0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.553+0000 7fc5faa6a700 1 --2- 172.21.15.7:0/3952744315 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5ec057cc0 0x7fc5ec00e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.553+0000 7fc5faa6a700 1 --2- 172.21.15.7:0/3952744315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5ec03d590 0x7fc5ec04ef60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.553+0000 7fc5faa6a700 1 --2- 172.21.15.7:0/3952744315 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5ec005e10 0x7fc5ec0dc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.553+0000 7fc5faa6a700 1 -- 172.21.15.7:0/3952744315 >> 172.21.15.7:0/3952744315 conn(0x7fc5ec03a550 msgr2=0x7fc5ec0061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:54.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.553+0000 7fc5faa6a700 1 -- 172.21.15.7:0/3952744315 shutdown_connections 2024-09-13T18:52:54.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.553+0000 7fc5faa6a700 1 -- 172.21.15.7:0/3952744315 wait complete. 2024-09-13T18:52:54.572 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:52:54.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89befd9700 1 Processor -- start 2024-09-13T18:52:54.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89befd9700 1 -- start start 2024-09-13T18:52:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89befd9700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f89b81573e0 0x7f89b8070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89befd9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f89b80710e0 0x7f89b8159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89befd9700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b8071540 con 0x7f89b815a460 2024-09-13T18:52:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89befd9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b80cc1b0 con 0x7f89b80710e0 2024-09-13T18:52:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89befd9700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b815ac50 con 0x7f89b81573e0 2024-09-13T18:52:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89bd7d6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f89b80710e0 0x7f89b8159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89bd7d6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f89b80710e0 0x7f89b8159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39074/0 (socket says 172.21.15.7:39074) 2024-09-13T18:52:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89bd7d6700 1 -- 172.21.15.7:0/3146490391 learned_addr learned my addr 172.21.15.7:0/3146490391 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89bdfd7700 1 --2- 172.21.15.7:0/3146490391 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f89b81573e0 0x7f89b8070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89bd7d6700 1 -- 172.21.15.7:0/3146490391 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f89b81573e0 msgr2=0x7f89b8070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89bd7d6700 1 --2- 172.21.15.7:0/3146490391 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f89b81573e0 0x7f89b8070ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89bd7d6700 1 -- 172.21.15.7:0/3146490391 >> v1:172.21.15.171:6789/0 conn(0x7f89b815a460 legacy=0x7f89b815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89bd7d6700 1 -- 172.21.15.7:0/3146490391 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b80cb3f0 con 0x7f89b80710e0 2024-09-13T18:52:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89bd7d6700 1 --2- 172.21.15.7:0/3146490391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89b80710e0 0x7f89b8159f20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009970 tx=0x7f89a8009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2993acf697ec26 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89bcfd5700 1 -- 172.21.15.7:0/3146490391 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8011830 con 0x7f89b80710e0 2024-09-13T18:52:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89bcfd5700 1 -- 172.21.15.7:0/3146490391 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a8011990 con 0x7f89b80710e0 2024-09-13T18:52:54.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.681+0000 7f89bcfd5700 1 -- 172.21.15.7:0/3146490391 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a8011d20 con 0x7f89b80710e0 2024-09-13T18:52:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89befd9700 1 -- 172.21.15.7:0/3146490391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89b80710e0 msgr2=0x7f89b8159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89befd9700 1 --2- 172.21.15.7:0/3146490391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89b80710e0 0x7f89b8159f20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009970 tx=0x7f89a8009c80 comp rx=0 tx=0).stop 2024-09-13T18:52:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89befd9700 1 -- 172.21.15.7:0/3146490391 shutdown_connections 2024-09-13T18:52:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89befd9700 1 --2- 172.21.15.7:0/3146490391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89b80710e0 0x7f89b8159f20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89befd9700 1 --2- 172.21.15.7:0/3146490391 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f89b81573e0 0x7f89b8070ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89befd9700 1 -- 172.21.15.7:0/3146490391 >> 172.21.15.7:0/3146490391 conn(0x7f89b80b0e10 msgr2=0x7f89b80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89befd9700 1 -- 172.21.15.7:0/3146490391 shutdown_connections 2024-09-13T18:52:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89befd9700 1 -- 172.21.15.7:0/3146490391 wait complete. 2024-09-13T18:52:54.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89befd9700 1 Processor -- start 2024-09-13T18:52:54.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89befd9700 1 -- start start 2024-09-13T18:52:54.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89befd9700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f89b80710e0 0x7f89b8147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89befd9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89b81573e0 0x7f89b8157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89befd9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89b815a460 0x7f89b80ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89befd9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b814d330 con 0x7f89b815a460 2024-09-13T18:52:54.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89befd9700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b81580a0 con 0x7f89b81573e0 2024-09-13T18:52:54.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89befd9700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b8158360 con 0x7f89b80710e0 2024-09-13T18:52:54.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89be7d8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89b815a460 0x7f89b80ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89bd7d6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89b81573e0 0x7f89b8157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89bdfd7700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f89b80710e0 0x7f89b8147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89bd7d6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89b81573e0 0x7f89b8157b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:41672/0 (socket says 172.21.15.7:41672) 2024-09-13T18:52:54.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89bd7d6700 1 -- 172.21.15.7:0/1257235554 learned_addr learned my addr 172.21.15.7:0/1257235554 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:54.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.685+0000 7f89bdfd7700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f89b80710e0 0x7f89b8147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:57150/0 (socket says 172.21.15.7:57150) 2024-09-13T18:52:54.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.689+0000 7f89be7d8700 1 -- 172.21.15.7:0/1257235554 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f89b80710e0 msgr2=0x7f89b8147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:54.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.689+0000 7f89be7d8700 1 --2- 172.21.15.7:0/1257235554 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f89b80710e0 0x7f89b8147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.689+0000 7f89be7d8700 1 -- 172.21.15.7:0/1257235554 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89b81573e0 msgr2=0x7f89b8157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:54.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.689+0000 7f89be7d8700 1 --2- 172.21.15.7:0/1257235554 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89b81573e0 0x7f89b8157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.689+0000 7f89be7d8700 1 -- 172.21.15.7:0/1257235554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a8009570 con 0x7f89b815a460 2024-09-13T18:52:54.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.689+0000 7f89bdfd7700 1 --2- 172.21.15.7:0/1257235554 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f89b80710e0 0x7f89b8147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:54.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.689+0000 7f89bd7d6700 1 --2- 172.21.15.7:0/1257235554 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89b81573e0 0x7f89b8157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:54.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.689+0000 7f89be7d8700 1 --2- 172.21.15.7:0/1257235554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89b815a460 0x7f89b80ccc10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f89b000c390 tx=0x7f89b000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:54.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.689+0000 7f89ae7fc700 1 -- 172.21.15.7:0/1257235554 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b00052d0 con 0x7f89b815a460 2024-09-13T18:52:54.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.689+0000 7f89ae7fc700 1 -- 172.21.15.7:0/1257235554 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b0005430 con 0x7f89b815a460 2024-09-13T18:52:54.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.689+0000 7f89befd9700 1 -- 172.21.15.7:0/1257235554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b8158640 con 0x7f89b815a460 2024-09-13T18:52:54.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.689+0000 7f89ae7fc700 1 -- 172.21.15.7:0/1257235554 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b00055a0 con 0x7f89b815a460 2024-09-13T18:52:54.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.689+0000 7f89befd9700 1 -- 172.21.15.7:0/1257235554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b8158c00 con 0x7f89b815a460 2024-09-13T18:52:54.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.689+0000 7f89befd9700 1 -- 172.21.15.7:0/1257235554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b8159f20 con 0x7f89b815a460 2024-09-13T18:52:54.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.689+0000 7f89ae7fc700 1 -- 172.21.15.7:0/1257235554 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f89b00038e0 con 0x7f89b815a460 2024-09-13T18:52:54.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.693+0000 7f89ae7fc700 1 --2- 172.21.15.7:0/1257235554 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f899c066730 0x7f899c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:54.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.693+0000 7f89ae7fc700 1 -- 172.21.15.7:0/1257235554 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89b00832c0 con 0x7f89b815a460 2024-09-13T18:52:54.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.693+0000 7f89bdfd7700 1 --2- 172.21.15.7:0/1257235554 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f899c066730 0x7f899c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:54.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.693+0000 7f89bdfd7700 1 --2- 172.21.15.7:0/1257235554 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f899c066730 0x7f899c068bf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f89b400e740 tx=0x7f89b400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:54.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.701+0000 7f89ae7fc700 1 -- 172.21.15.7:0/1257235554 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f89b0050630 con 0x7f89b815a460 2024-09-13T18:52:54.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.885+0000 7f89befd9700 1 -- 172.21.15.7:0/1257235554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f89b8074660 con 0x7f89b815a460 2024-09-13T18:52:54.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.889+0000 7f89ae7fc700 1 -- 172.21.15.7:0/1257235554 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f89b0053c50 con 0x7f89b815a460 2024-09-13T18:52:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.889+0000 7f89befd9700 1 -- 172.21.15.7:0/1257235554 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f899c066730 msgr2=0x7f899c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.889+0000 7f89befd9700 1 --2- 172.21.15.7:0/1257235554 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f899c066730 0x7f899c068bf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f89b400e740 tx=0x7f89b400b040 comp rx=0 tx=0).stop 2024-09-13T18:52:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.889+0000 7f89befd9700 1 -- 172.21.15.7:0/1257235554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89b815a460 msgr2=0x7f89b80ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.889+0000 7f89befd9700 1 --2- 172.21.15.7:0/1257235554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89b815a460 0x7f89b80ccc10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f89b000c390 tx=0x7f89b000c6a0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.889+0000 7f89befd9700 1 -- 172.21.15.7:0/1257235554 shutdown_connections 2024-09-13T18:52:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.889+0000 7f89befd9700 1 --2- 172.21.15.7:0/1257235554 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f899c066730 0x7f899c068bf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.889+0000 7f89befd9700 1 --2- 172.21.15.7:0/1257235554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89b815a460 0x7f89b80ccc10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.889+0000 7f89befd9700 1 --2- 172.21.15.7:0/1257235554 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89b81573e0 0x7f89b8157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.889+0000 7f89befd9700 1 --2- 172.21.15.7:0/1257235554 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f89b80710e0 0x7f89b8147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:54.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.889+0000 7f89befd9700 1 -- 172.21.15.7:0/1257235554 >> 172.21.15.7:0/1257235554 conn(0x7f89b80b0e10 msgr2=0x7f89b814de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:54.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.889+0000 7f89befd9700 1 -- 172.21.15.7:0/1257235554 shutdown_connections 2024-09-13T18:52:54.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:54.889+0000 7f89befd9700 1 -- 172.21.15.7:0/1257235554 wait complete. 2024-09-13T18:52:54.897 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-13T18:52:54.908 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:52:54.909 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:52:54.917 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.029+0000 7f739ec20700 1 Processor -- start 2024-09-13T18:52:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.029+0000 7f739ec20700 1 -- start start 2024-09-13T18:52:55.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.029+0000 7f739ec20700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f739815cae0 0x7f73980cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.029+0000 7f739ec20700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73980d21e0 con 0x7f739806b310 2024-09-13T18:52:55.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.029+0000 7f739ec20700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73980d1940 con 0x7f739815cec0 2024-09-13T18:52:55.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.029+0000 7f739ec20700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73980d1c20 con 0x7f739815cae0 2024-09-13T18:52:55.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.029+0000 7f739e41f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f739806b310 0x7f73980b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38792/0 (socket says 172.21.15.7:38792) 2024-09-13T18:52:55.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.029+0000 7f739e41f700 1 -- 172.21.15.7:0/454771241 learned_addr learned my addr 172.21.15.7:0/454771241 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:55.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.029+0000 7f739dc1e700 1 --2- 172.21.15.7:0/454771241 >> v2:172.21.15.171:3300/0 conn(0x7f739815cae0 0x7f73980cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.029+0000 7f739cc1c700 1 -- 172.21.15.7:0/454771241 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3178869709 0 0) 0x7f73980d21e0 con 0x7f739806b310 2024-09-13T18:52:55.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.029+0000 7f739cc1c700 1 -- 172.21.15.7:0/454771241 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73840035d0 con 0x7f739806b310 2024-09-13T18:52:55.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.029+0000 7f739dc1e700 1 -- 172.21.15.7:0/454771241 >> v1:172.21.15.171:6789/0 conn(0x7f739815cec0 legacy=0x7f739806ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.029+0000 7f739dc1e700 1 -- 172.21.15.7:0/454771241 >> v1:172.21.15.7:6789/0 conn(0x7f739806b310 legacy=0x7f73980b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.029+0000 7f739dc1e700 1 -- 172.21.15.7:0/454771241 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73980d1f00 con 0x7f739815cae0 2024-09-13T18:52:55.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.029+0000 7f739dc1e700 1 --2- 172.21.15.7:0/454771241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f739815cae0 0x7f73980cf030 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7394009880 tx=0x7f7394009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98e365348e87870b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:55.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.029+0000 7f739cc1c700 1 -- 172.21.15.7:0/454771241 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394011830 con 0x7f739815cae0 2024-09-13T18:52:55.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.029+0000 7f739cc1c700 1 -- 172.21.15.7:0/454771241 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7394011990 con 0x7f739815cae0 2024-09-13T18:52:55.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.029+0000 7f739cc1c700 1 -- 172.21.15.7:0/454771241 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7394011d20 con 0x7f739815cae0 2024-09-13T18:52:55.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.033+0000 7f739ec20700 1 -- 172.21.15.7:0/454771241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f739815cae0 msgr2=0x7f73980cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.033+0000 7f739ec20700 1 --2- 172.21.15.7:0/454771241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f739815cae0 0x7f73980cf030 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7394009880 tx=0x7f7394009b90 comp rx=0 tx=0).stop 2024-09-13T18:52:55.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.033+0000 7f739ec20700 1 -- 172.21.15.7:0/454771241 shutdown_connections 2024-09-13T18:52:55.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.033+0000 7f739ec20700 1 --2- 172.21.15.7:0/454771241 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f739815cae0 0x7f73980cf030 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.033+0000 7f739ec20700 1 -- 172.21.15.7:0/454771241 >> 172.21.15.7:0/454771241 conn(0x7f73980d2c80 msgr2=0x7f73980d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:55.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.033+0000 7f739ec20700 1 -- 172.21.15.7:0/454771241 shutdown_connections 2024-09-13T18:52:55.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.033+0000 7f739ec20700 1 -- 172.21.15.7:0/454771241 wait complete. 2024-09-13T18:52:55.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739ec20700 1 Processor -- start 2024-09-13T18:52:55.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739ec20700 1 -- start start 2024-09-13T18:52:55.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739ec20700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f739806b310 0x7f739815a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739ec20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f739815cae0 0x7f739814fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739ec20700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f739815cec0 0x7f7398150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739ec20700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7398150850 con 0x7f739815cae0 2024-09-13T18:52:55.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739ec20700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739815e190 con 0x7f739806b310 2024-09-13T18:52:55.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739ec20700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739815e680 con 0x7f739815cec0 2024-09-13T18:52:55.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739d41d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f739815cae0 0x7f739814fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739dc1e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f739806b310 0x7f739815a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739e41f700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f739815cec0 0x7f7398150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739dc1e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f739806b310 0x7f739815a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:41688/0 (socket says 172.21.15.7:41688) 2024-09-13T18:52:55.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739d41d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f739815cae0 0x7f739814fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39090/0 (socket says 172.21.15.7:39090) 2024-09-13T18:52:55.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739dc1e700 1 -- 172.21.15.7:0/2554016455 learned_addr learned my addr 172.21.15.7:0/2554016455 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:55.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739d41d700 1 -- 172.21.15.7:0/2554016455 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f739815cec0 msgr2=0x7f7398150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739d41d700 1 --2- 172.21.15.7:0/2554016455 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f739815cec0 0x7f7398150310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739d41d700 1 -- 172.21.15.7:0/2554016455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f739806b310 msgr2=0x7f739815a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739d41d700 1 --2- 172.21.15.7:0/2554016455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f739806b310 0x7f739815a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739d41d700 1 -- 172.21.15.7:0/2554016455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73940094b0 con 0x7f739815cae0 2024-09-13T18:52:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739e41f700 1 --2- 172.21.15.7:0/2554016455 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f739815cec0 0x7f7398150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739d41d700 1 --2- 172.21.15.7:0/2554016455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f739815cae0 0x7f739814fdd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f738800c480 tx=0x7f738800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f738e7fc700 1 -- 172.21.15.7:0/2554016455 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73880052d0 con 0x7f739815cae0 2024-09-13T18:52:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f738e7fc700 1 -- 172.21.15.7:0/2554016455 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7388005430 con 0x7f739815cae0 2024-09-13T18:52:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739ec20700 1 -- 172.21.15.7:0/2554016455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f739815e960 con 0x7f739815cae0 2024-09-13T18:52:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f738e7fc700 1 -- 172.21.15.7:0/2554016455 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73880055a0 con 0x7f739815cae0 2024-09-13T18:52:55.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.037+0000 7f739ec20700 1 -- 172.21.15.7:0/2554016455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739815ef20 con 0x7f739815cae0 2024-09-13T18:52:55.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.041+0000 7f739ec20700 1 -- 172.21.15.7:0/2554016455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73980b4520 con 0x7f739815cae0 2024-09-13T18:52:55.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.041+0000 7f738e7fc700 1 -- 172.21.15.7:0/2554016455 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f73880038e0 con 0x7f739815cae0 2024-09-13T18:52:55.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.041+0000 7f738e7fc700 1 --2- 172.21.15.7:0/2554016455 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f737c066730 0x7f737c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.041+0000 7f738e7fc700 1 -- 172.21.15.7:0/2554016455 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7388083500 con 0x7f739815cae0 2024-09-13T18:52:55.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.045+0000 7f739dc1e700 1 --2- 172.21.15.7:0/2554016455 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f737c066730 0x7f737c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.045+0000 7f739dc1e700 1 --2- 172.21.15.7:0/2554016455 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f737c066730 0x7f737c068bf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7394009fd0 tx=0x7f7394008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:55.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.049+0000 7f738e7fc700 1 -- 172.21.15.7:0/2554016455 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f73880507e0 con 0x7f739815cae0 2024-09-13T18:52:55.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.221+0000 7f739ec20700 1 -- 172.21.15.7:0/2554016455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73980cf030 con 0x7f739815cae0 2024-09-13T18:52:55.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.221+0000 7f738e7fc700 1 -- 172.21.15.7:0/2554016455 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f7388053e00 con 0x7f739815cae0 2024-09-13T18:52:55.229 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:55.229 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.225+0000 7f739ec20700 1 -- 172.21.15.7:0/2554016455 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f737c066730 msgr2=0x7f737c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.225+0000 7f739ec20700 1 --2- 172.21.15.7:0/2554016455 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f737c066730 0x7f737c068bf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7394009fd0 tx=0x7f7394008030 comp rx=0 tx=0).stop 2024-09-13T18:52:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.225+0000 7f739ec20700 1 -- 172.21.15.7:0/2554016455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f739815cae0 msgr2=0x7f739814fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.225+0000 7f739ec20700 1 --2- 172.21.15.7:0/2554016455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f739815cae0 0x7f739814fdd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f738800c480 tx=0x7f738800c790 comp rx=0 tx=0).stop 2024-09-13T18:52:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.225+0000 7f739ec20700 1 -- 172.21.15.7:0/2554016455 shutdown_connections 2024-09-13T18:52:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.225+0000 7f739ec20700 1 --2- 172.21.15.7:0/2554016455 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f737c066730 0x7f737c068bf0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.225+0000 7f739ec20700 1 --2- 172.21.15.7:0/2554016455 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f739815cec0 0x7f7398150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.225+0000 7f739ec20700 1 --2- 172.21.15.7:0/2554016455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f739815cae0 0x7f739814fdd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.225+0000 7f739ec20700 1 --2- 172.21.15.7:0/2554016455 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f739806b310 0x7f739815a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.225+0000 7f739ec20700 1 -- 172.21.15.7:0/2554016455 >> 172.21.15.7:0/2554016455 conn(0x7f73980d2c80 msgr2=0x7f73980b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.225+0000 7f739ec20700 1 -- 172.21.15.7:0/2554016455 shutdown_connections 2024-09-13T18:52:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.225+0000 7f739ec20700 1 -- 172.21.15.7:0/2554016455 wait complete. 2024-09-13T18:52:55.244 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:52:15.554683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:52:16.896879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:52:55.244 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:52:55.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.357+0000 7fcb8359e700 1 Processor -- start 2024-09-13T18:52:55.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.357+0000 7fcb8359e700 1 -- start start 2024-09-13T18:52:55.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb8359e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcb8406db00 0x7fcb840b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb8359e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb840c9d20 con 0x7fcb8415aaa0 2024-09-13T18:52:55.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb8359e700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb840b28e0 con 0x7fcb84070720 2024-09-13T18:52:55.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb8359e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb840b2bc0 con 0x7fcb8406db00 2024-09-13T18:52:55.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb7bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcb8415aaa0 0x7fcb840b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38796/0 (socket says 172.21.15.7:38796) 2024-09-13T18:52:55.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb7bfff700 1 -- 172.21.15.7:0/2712988964 learned_addr learned my addr 172.21.15.7:0/2712988964 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:55.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb8259c700 1 --2- 172.21.15.7:0/2712988964 >> v2:172.21.15.7:3300/0 conn(0x7fcb8406db00 0x7fcb840b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb81d9b700 1 -- 172.21.15.7:0/2712988964 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1185933462 0 0) 0x7fcb840c9d20 con 0x7fcb8415aaa0 2024-09-13T18:52:55.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb81d9b700 1 -- 172.21.15.7:0/2712988964 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb700035d0 con 0x7fcb8415aaa0 2024-09-13T18:52:55.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb8259c700 1 -- 172.21.15.7:0/2712988964 >> v1:172.21.15.171:6789/0 conn(0x7fcb84070720 legacy=0x7fcb840ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb81d9b700 1 -- 172.21.15.7:0/2712988964 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3882893317 0 0) 0x7fcb700035d0 con 0x7fcb8415aaa0 2024-09-13T18:52:55.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb8259c700 1 -- 172.21.15.7:0/2712988964 >> v1:172.21.15.7:6789/0 conn(0x7fcb8415aaa0 legacy=0x7fcb840b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb8259c700 1 -- 172.21.15.7:0/2712988964 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb840c96d0 con 0x7fcb8406db00 2024-09-13T18:52:55.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb8259c700 1 --2- 172.21.15.7:0/2712988964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb8406db00 0x7fcb840b1fb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009850 tx=0x7fcb74009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9e1945b6a5f685 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:55.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb81d9b700 1 -- 172.21.15.7:0/2712988964 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c001670 con 0x7fcb8406db00 2024-09-13T18:52:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb81d9b700 1 -- 172.21.15.7:0/2712988964 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb74010590 con 0x7fcb8406db00 2024-09-13T18:52:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb81d9b700 1 -- 172.21.15.7:0/2712988964 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb74010940 con 0x7fcb8406db00 2024-09-13T18:52:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb8359e700 1 -- 172.21.15.7:0/2712988964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb8406db00 msgr2=0x7fcb840b1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb8359e700 1 --2- 172.21.15.7:0/2712988964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb8406db00 0x7fcb840b1fb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009850 tx=0x7fcb74009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb8359e700 1 -- 172.21.15.7:0/2712988964 shutdown_connections 2024-09-13T18:52:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb8359e700 1 --2- 172.21.15.7:0/2712988964 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb8406db00 0x7fcb840b1fb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb8359e700 1 -- 172.21.15.7:0/2712988964 >> 172.21.15.7:0/2712988964 conn(0x7fcb840cba60 msgr2=0x7fcb840cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb8359e700 1 -- 172.21.15.7:0/2712988964 shutdown_connections 2024-09-13T18:52:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.361+0000 7fcb8359e700 1 -- 172.21.15.7:0/2712988964 wait complete. 2024-09-13T18:52:55.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.365+0000 7fcb8359e700 1 Processor -- start 2024-09-13T18:52:55.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.365+0000 7fcb8359e700 1 -- start start 2024-09-13T18:52:55.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb8359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb8406db00 0x7fcb840afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb8359e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb84070720 0x7fcb840a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb8359e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb8415aaa0 0x7fcb840a89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb8359e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb840c6a30 con 0x7fcb8406db00 2024-09-13T18:52:55.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb8359e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb841316a0 con 0x7fcb84070720 2024-09-13T18:52:55.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb8359e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb84131980 con 0x7fcb8415aaa0 2024-09-13T18:52:55.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb8259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb8406db00 0x7fcb840afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb8259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb8406db00 0x7fcb840afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39100/0 (socket says 172.21.15.7:39100) 2024-09-13T18:52:55.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb8259c700 1 -- 172.21.15.7:0/2502042321 learned_addr learned my addr 172.21.15.7:0/2502042321 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:55.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb7bfff700 1 --2- 172.21.15.7:0/2502042321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb84070720 0x7fcb840a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb82d9d700 1 --2- 172.21.15.7:0/2502042321 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb8415aaa0 0x7fcb840a89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb8259c700 1 -- 172.21.15.7:0/2502042321 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb8415aaa0 msgr2=0x7fcb840a89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb8259c700 1 --2- 172.21.15.7:0/2502042321 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb8415aaa0 0x7fcb840a89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb8259c700 1 -- 172.21.15.7:0/2502042321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb84070720 msgr2=0x7fcb840a84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb8259c700 1 --2- 172.21.15.7:0/2502042321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb84070720 0x7fcb840a84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb8259c700 1 -- 172.21.15.7:0/2502042321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb740094b0 con 0x7fcb8406db00 2024-09-13T18:52:55.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb82d9d700 1 --2- 172.21.15.7:0/2502042321 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb8415aaa0 0x7fcb840a89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:55.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb8259c700 1 --2- 172.21.15.7:0/2502042321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb8406db00 0x7fcb840afd00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcb74013040 tx=0x7fcb74004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:55.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb7affd700 1 -- 172.21.15.7:0/2502042321 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb74004610 con 0x7fcb8406db00 2024-09-13T18:52:55.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb7affd700 1 -- 172.21.15.7:0/2502042321 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb740036c0 con 0x7fcb8406db00 2024-09-13T18:52:55.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb8359e700 1 -- 172.21.15.7:0/2502042321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb84131c60 con 0x7fcb8406db00 2024-09-13T18:52:55.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb7affd700 1 -- 172.21.15.7:0/2502042321 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb74003a70 con 0x7fcb8406db00 2024-09-13T18:52:55.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb8359e700 1 -- 172.21.15.7:0/2502042321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb840b0240 con 0x7fcb8406db00 2024-09-13T18:52:55.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.369+0000 7fcb8359e700 1 -- 172.21.15.7:0/2502042321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb840b4fe0 con 0x7fcb8406db00 2024-09-13T18:52:55.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.373+0000 7fcb7affd700 1 -- 172.21.15.7:0/2502042321 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcb74003bd0 con 0x7fcb8406db00 2024-09-13T18:52:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.373+0000 7fcb7affd700 1 --2- 172.21.15.7:0/2502042321 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb64066730 0x7fcb64068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.373+0000 7fcb7affd700 1 -- 172.21.15.7:0/2502042321 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb74085700 con 0x7fcb8406db00 2024-09-13T18:52:55.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.373+0000 7fcb7bfff700 1 --2- 172.21.15.7:0/2502042321 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb64066730 0x7fcb64068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.373+0000 7fcb7bfff700 1 --2- 172.21.15.7:0/2502042321 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb64066730 0x7fcb64068bf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcb84073ec0 tx=0x7fcb6c001ce0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:55.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.381+0000 7fcb7affd700 1 -- 172.21.15.7:0/2502042321 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcb74052c20 con 0x7fcb8406db00 2024-09-13T18:52:55.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.549+0000 7fcb8359e700 1 -- 172.21.15.7:0/2502042321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb84132180 con 0x7fcb8406db00 2024-09-13T18:52:55.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.553+0000 7fcb7affd700 1 -- 172.21.15.7:0/2502042321 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb74056240 con 0x7fcb8406db00 2024-09-13T18:52:55.558 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:52:55.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.553+0000 7fcb8359e700 1 -- 172.21.15.7:0/2502042321 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb64066730 msgr2=0x7fcb64068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.553+0000 7fcb8359e700 1 --2- 172.21.15.7:0/2502042321 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb64066730 0x7fcb64068bf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcb84073ec0 tx=0x7fcb6c001ce0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.553+0000 7fcb8359e700 1 -- 172.21.15.7:0/2502042321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb8406db00 msgr2=0x7fcb840afd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.553+0000 7fcb8359e700 1 --2- 172.21.15.7:0/2502042321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb8406db00 0x7fcb840afd00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcb74013040 tx=0x7fcb74004770 comp rx=0 tx=0).stop 2024-09-13T18:52:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.553+0000 7fcb8359e700 1 -- 172.21.15.7:0/2502042321 shutdown_connections 2024-09-13T18:52:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.553+0000 7fcb8359e700 1 --2- 172.21.15.7:0/2502042321 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb64066730 0x7fcb64068bf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.553+0000 7fcb8359e700 1 --2- 172.21.15.7:0/2502042321 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb8415aaa0 0x7fcb840a89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.553+0000 7fcb8359e700 1 --2- 172.21.15.7:0/2502042321 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb84070720 0x7fcb840a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.553+0000 7fcb8359e700 1 --2- 172.21.15.7:0/2502042321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb8406db00 0x7fcb840afd00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.553+0000 7fcb8359e700 1 -- 172.21.15.7:0/2502042321 >> 172.21.15.7:0/2502042321 conn(0x7fcb840cba60 msgr2=0x7fcb8415c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.553+0000 7fcb8359e700 1 -- 172.21.15.7:0/2502042321 shutdown_connections 2024-09-13T18:52:55.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.553+0000 7fcb8359e700 1 -- 172.21.15.7:0/2502042321 wait complete. 2024-09-13T18:52:55.573 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:52:55.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.685+0000 7f8bef59e700 1 Processor -- start 2024-09-13T18:52:55.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.685+0000 7f8bef59e700 1 -- start start 2024-09-13T18:52:55.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.685+0000 7f8bef59e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f8bf015cda0 0x7f8bf006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.685+0000 7f8bef59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8bf006a520 0x7f8bf006a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.685+0000 7f8bef59e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf00b8430 con 0x7f8bf00ceb70 2024-09-13T18:52:55.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.685+0000 7f8bef59e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf006be40 con 0x7f8bf015cda0 2024-09-13T18:52:55.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.685+0000 7f8bef59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00b64f0 con 0x7f8bf006a520 2024-09-13T18:52:55.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.685+0000 7f8be5d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8bf00ceb70 0x7f8bf00b5de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38810/0 (socket says 172.21.15.7:38810) 2024-09-13T18:52:55.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.685+0000 7f8be5d9b700 1 -- 172.21.15.7:0/3048806111 learned_addr learned my addr 172.21.15.7:0/3048806111 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:55.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.685+0000 7f8bee59c700 1 --2- 172.21.15.7:0/3048806111 >> v2:172.21.15.171:3300/0 conn(0x7f8bf015cda0 0x7f8bf006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.685+0000 7f8beed9d700 1 --2- 172.21.15.7:0/3048806111 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8bf006a520 0x7f8bf006a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.685+0000 7f8bedd9b700 1 -- 172.21.15.7:0/3048806111 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999125406 0 0) 0x7f8bf00b8430 con 0x7f8bf00ceb70 2024-09-13T18:52:55.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.685+0000 7f8bedd9b700 1 -- 172.21.15.7:0/3048806111 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bdc0035d0 con 0x7f8bf00ceb70 2024-09-13T18:52:55.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.685+0000 7f8bee59c700 1 -- 172.21.15.7:0/3048806111 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8bf006a520 msgr2=0x7f8bf006a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:55.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.685+0000 7f8bee59c700 1 --2- 172.21.15.7:0/3048806111 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8bf006a520 0x7f8bf006a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.685+0000 7f8bee59c700 1 -- 172.21.15.7:0/3048806111 >> v1:172.21.15.7:6789/0 conn(0x7f8bf00ceb70 legacy=0x7f8bf00b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.685+0000 7f8bee59c700 1 -- 172.21.15.7:0/3048806111 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf006ae90 con 0x7f8bf015cda0 2024-09-13T18:52:55.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.685+0000 7f8bee59c700 1 --2- 172.21.15.7:0/3048806111 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8bf015cda0 0x7f8bf006b900 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009850 tx=0x7f8be0009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8eba46a8942d128 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:55.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.689+0000 7f8bedd9b700 1 -- 172.21.15.7:0/3048806111 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be0011830 con 0x7f8bf015cda0 2024-09-13T18:52:55.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.689+0000 7f8bedd9b700 1 -- 172.21.15.7:0/3048806111 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be0011990 con 0x7f8bf015cda0 2024-09-13T18:52:55.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.689+0000 7f8bedd9b700 1 -- 172.21.15.7:0/3048806111 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be0011d00 con 0x7f8bf015cda0 2024-09-13T18:52:55.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.689+0000 7f8bef59e700 1 -- 172.21.15.7:0/3048806111 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8bf015cda0 msgr2=0x7f8bf006b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.689+0000 7f8bef59e700 1 --2- 172.21.15.7:0/3048806111 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8bf015cda0 0x7f8bf006b900 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8be0009850 tx=0x7f8be0009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:55.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.689+0000 7f8bef59e700 1 -- 172.21.15.7:0/3048806111 shutdown_connections 2024-09-13T18:52:55.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.689+0000 7f8bef59e700 1 --2- 172.21.15.7:0/3048806111 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8bf006a520 0x7f8bf006a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.689+0000 7f8bef59e700 1 --2- 172.21.15.7:0/3048806111 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8bf015cda0 0x7f8bf006b900 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.689+0000 7f8bef59e700 1 -- 172.21.15.7:0/3048806111 >> 172.21.15.7:0/3048806111 conn(0x7f8bf00d2cb0 msgr2=0x7f8bf00d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:55.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.689+0000 7f8bef59e700 1 -- 172.21.15.7:0/3048806111 shutdown_connections 2024-09-13T18:52:55.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.689+0000 7f8bef59e700 1 -- 172.21.15.7:0/3048806111 wait complete. 2024-09-13T18:52:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.689+0000 7f8bef59e700 1 Processor -- start 2024-09-13T18:52:55.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.689+0000 7f8bef59e700 1 -- start start 2024-09-13T18:52:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8bef59e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8bf006a520 0x7f8bf00c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8bef59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8bf00ceb70 0x7f8bf01499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8bef59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bf015cda0 0x7f8bf00c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8bef59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf014a0e0 con 0x7f8bf015cda0 2024-09-13T18:52:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8bef59e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf014a980 con 0x7f8bf006a520 2024-09-13T18:52:55.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8bef59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00c9d80 con 0x7f8bf00ceb70 2024-09-13T18:52:55.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8beed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bf015cda0 0x7f8bf00c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8bee59c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8bf006a520 0x7f8bf00c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8beed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bf015cda0 0x7f8bf00c67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39102/0 (socket says 172.21.15.7:39102) 2024-09-13T18:52:55.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8beed9d700 1 -- 172.21.15.7:0/3270840909 learned_addr learned my addr 172.21.15.7:0/3270840909 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:55.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8be5d9b700 1 --2- 172.21.15.7:0/3270840909 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8bf00ceb70 0x7f8bf01499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8beed9d700 1 -- 172.21.15.7:0/3270840909 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8bf00ceb70 msgr2=0x7f8bf01499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8beed9d700 1 --2- 172.21.15.7:0/3270840909 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8bf00ceb70 0x7f8bf01499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8beed9d700 1 -- 172.21.15.7:0/3270840909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8bf006a520 msgr2=0x7f8bf00c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8beed9d700 1 --2- 172.21.15.7:0/3270840909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8bf006a520 0x7f8bf00c9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8beed9d700 1 -- 172.21.15.7:0/3270840909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be00094b0 con 0x7f8bf015cda0 2024-09-13T18:52:55.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8bee59c700 1 --2- 172.21.15.7:0/3270840909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8bf006a520 0x7f8bf00c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:55.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8beed9d700 1 --2- 172.21.15.7:0/3270840909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bf015cda0 0x7f8bf00c67e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8be800efc0 tx=0x7f8be800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:55.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8be77fe700 1 -- 172.21.15.7:0/3270840909 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be800cad0 con 0x7f8bf015cda0 2024-09-13T18:52:55.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8be77fe700 1 -- 172.21.15.7:0/3270840909 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be800f040 con 0x7f8bf015cda0 2024-09-13T18:52:55.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8bef59e700 1 -- 172.21.15.7:0/3270840909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf00c9910 con 0x7f8bf015cda0 2024-09-13T18:52:55.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8be77fe700 1 -- 172.21.15.7:0/3270840909 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be800ce70 con 0x7f8bf015cda0 2024-09-13T18:52:55.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.693+0000 7f8bef59e700 1 -- 172.21.15.7:0/3270840909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf014a610 con 0x7f8bf015cda0 2024-09-13T18:52:55.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.697+0000 7f8bef59e700 1 -- 172.21.15.7:0/3270840909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf014a7d0 con 0x7f8bf015cda0 2024-09-13T18:52:55.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.697+0000 7f8be77fe700 1 -- 172.21.15.7:0/3270840909 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8be8014480 con 0x7f8bf015cda0 2024-09-13T18:52:55.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.697+0000 7f8be77fe700 1 --2- 172.21.15.7:0/3270840909 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8bd0066730 0x7f8bd0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:55.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.697+0000 7f8be77fe700 1 -- 172.21.15.7:0/3270840909 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8be8084c20 con 0x7f8bf015cda0 2024-09-13T18:52:55.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.697+0000 7f8bee59c700 1 --2- 172.21.15.7:0/3270840909 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8bd0066730 0x7f8bd0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:55.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.697+0000 7f8bee59c700 1 --2- 172.21.15.7:0/3270840909 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8bd0066730 0x7f8bd0068bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8be00090b0 tx=0x7f8be0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:55.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.705+0000 7f8be77fe700 1 -- 172.21.15.7:0/3270840909 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8be8051f00 con 0x7f8bf015cda0 2024-09-13T18:52:55.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.877+0000 7f8bef59e700 1 -- 172.21.15.7:0/3270840909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8bf014aaf0 con 0x7f8bf015cda0 2024-09-13T18:52:55.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.881+0000 7f8be77fe700 1 -- 172.21.15.7:0/3270840909 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8be8018020 con 0x7f8bf015cda0 2024-09-13T18:52:55.886 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:52:55.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.881+0000 7f8bef59e700 1 -- 172.21.15.7:0/3270840909 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8bd0066730 msgr2=0x7f8bd0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.881+0000 7f8bef59e700 1 --2- 172.21.15.7:0/3270840909 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8bd0066730 0x7f8bd0068bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8be00090b0 tx=0x7f8be0008030 comp rx=0 tx=0).stop 2024-09-13T18:52:55.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.881+0000 7f8bef59e700 1 -- 172.21.15.7:0/3270840909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bf015cda0 msgr2=0x7f8bf00c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:55.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.881+0000 7f8bef59e700 1 --2- 172.21.15.7:0/3270840909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bf015cda0 0x7f8bf00c67e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8be800efc0 tx=0x7f8be800c0c0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.881+0000 7f8bef59e700 1 -- 172.21.15.7:0/3270840909 shutdown_connections 2024-09-13T18:52:55.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.881+0000 7f8bef59e700 1 --2- 172.21.15.7:0/3270840909 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8bd0066730 0x7f8bd0068bf0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.881+0000 7f8bef59e700 1 --2- 172.21.15.7:0/3270840909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bf015cda0 0x7f8bf00c67e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.881+0000 7f8bef59e700 1 --2- 172.21.15.7:0/3270840909 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8bf00ceb70 0x7f8bf01499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.881+0000 7f8bef59e700 1 --2- 172.21.15.7:0/3270840909 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8bf006a520 0x7f8bf00c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:55.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.881+0000 7f8bef59e700 1 -- 172.21.15.7:0/3270840909 >> 172.21.15.7:0/3270840909 conn(0x7f8bf00d2cb0 msgr2=0x7f8bf00b5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:55.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.881+0000 7f8bef59e700 1 -- 172.21.15.7:0/3270840909 shutdown_connections 2024-09-13T18:52:55.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:55.881+0000 7f8bef59e700 1 -- 172.21.15.7:0/3270840909 wait complete. 2024-09-13T18:52:55.901 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:52:56.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.021+0000 7f72c3cf4700 1 Processor -- start 2024-09-13T18:52:56.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.021+0000 7f72c3cf4700 1 -- start start 2024-09-13T18:52:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.021+0000 7f72c3cf4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f72bc15cb60 0x7f72bc0ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.021+0000 7f72c3cf4700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f72bc071c80 0x7f72bc0720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.021+0000 7f72c3cf4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72bc072610 con 0x7f72bc160850 2024-09-13T18:52:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.021+0000 7f72c3cf4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72bc0ad6e0 con 0x7f72bc15cb60 2024-09-13T18:52:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.021+0000 7f72c3cf4700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc0b2b10 con 0x7f72bc071c80 2024-09-13T18:52:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.021+0000 7f72c1a90700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f72bc15cb60 0x7f72bc0ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.021+0000 7f72c2291700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f72bc160850 0x7f72bc160cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38822/0 (socket says 172.21.15.7:38822) 2024-09-13T18:52:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.021+0000 7f72c2291700 1 -- 172.21.15.7:0/1817854868 learned_addr learned my addr 172.21.15.7:0/1817854868 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.021+0000 7f72c128f700 1 --2- 172.21.15.7:0/1817854868 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f72bc071c80 0x7f72bc0720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.021+0000 7f72c0a8e700 1 -- 172.21.15.7:0/1817854868 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804906030 0 0) 0x7f72b4001490 con 0x7f72bc160850 2024-09-13T18:52:56.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.021+0000 7f72c0a8e700 1 -- 172.21.15.7:0/1817854868 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72a80035d0 con 0x7f72bc160850 2024-09-13T18:52:56.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.021+0000 7f72c1a90700 1 -- 172.21.15.7:0/1817854868 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f72bc071c80 msgr2=0x7f72bc0720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:56.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.021+0000 7f72c1a90700 1 --2- 172.21.15.7:0/1817854868 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f72bc071c80 0x7f72bc0720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.021+0000 7f72c1a90700 1 -- 172.21.15.7:0/1817854868 >> v1:172.21.15.7:6789/0 conn(0x7f72bc160850 legacy=0x7f72bc160cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:56.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.025+0000 7f72c1a90700 1 -- 172.21.15.7:0/1817854868 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc0b2df0 con 0x7f72bc15cb60 2024-09-13T18:52:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.025+0000 7f72c1a90700 1 --2- 172.21.15.7:0/1817854868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72bc15cb60 0x7f72bc0ae3a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009850 tx=0x7f72b8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1eaab9403a51878 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.025+0000 7f72c0a8e700 1 -- 172.21.15.7:0/1817854868 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b8011830 con 0x7f72bc15cb60 2024-09-13T18:52:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.025+0000 7f72c0a8e700 1 -- 172.21.15.7:0/1817854868 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b8011990 con 0x7f72bc15cb60 2024-09-13T18:52:56.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.025+0000 7f72c0a8e700 1 -- 172.21.15.7:0/1817854868 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b8011d40 con 0x7f72bc15cb60 2024-09-13T18:52:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.025+0000 7f72c3cf4700 1 -- 172.21.15.7:0/1817854868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72bc15cb60 msgr2=0x7f72bc0ae3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.025+0000 7f72c3cf4700 1 --2- 172.21.15.7:0/1817854868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72bc15cb60 0x7f72bc0ae3a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009850 tx=0x7f72b8009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.025+0000 7f72c3cf4700 1 -- 172.21.15.7:0/1817854868 shutdown_connections 2024-09-13T18:52:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.025+0000 7f72c3cf4700 1 --2- 172.21.15.7:0/1817854868 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f72bc071c80 0x7f72bc0720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.025+0000 7f72c3cf4700 1 --2- 172.21.15.7:0/1817854868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72bc15cb60 0x7f72bc0ae3a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.025+0000 7f72c3cf4700 1 -- 172.21.15.7:0/1817854868 >> 172.21.15.7:0/1817854868 conn(0x7f72bc0b1810 msgr2=0x7f72bc0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.025+0000 7f72c3cf4700 1 -- 172.21.15.7:0/1817854868 shutdown_connections 2024-09-13T18:52:56.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.025+0000 7f72c3cf4700 1 -- 172.21.15.7:0/1817854868 wait complete. 2024-09-13T18:52:56.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c3cf4700 1 Processor -- start 2024-09-13T18:52:56.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c3cf4700 1 -- start start 2024-09-13T18:52:56.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c3cf4700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f72bc071c80 0x7f72bc144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:56.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c3cf4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72bc15cb60 0x7f72bc1f32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:56.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c3cf4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72bc160850 0x7f72bc1f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:56.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c3cf4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc06bbf0 con 0x7f72bc15cb60 2024-09-13T18:52:56.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c3cf4700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc145110 con 0x7f72bc160850 2024-09-13T18:52:56.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c3cf4700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc1f3830 con 0x7f72bc071c80 2024-09-13T18:52:56.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c128f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72bc15cb60 0x7f72bc1f32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:56.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c1a90700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f72bc071c80 0x7f72bc144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:56.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c2291700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72bc160850 0x7f72bc1f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:56.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c128f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72bc15cb60 0x7f72bc1f32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39122/0 (socket says 172.21.15.7:39122) 2024-09-13T18:52:56.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c128f700 1 -- 172.21.15.7:0/1466508809 learned_addr learned my addr 172.21.15.7:0/1466508809 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:56.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c1a90700 1 -- 172.21.15.7:0/1466508809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72bc160850 msgr2=0x7f72bc1f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:56.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c1a90700 1 --2- 172.21.15.7:0/1466508809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72bc160850 0x7f72bc1f5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c1a90700 1 -- 172.21.15.7:0/1466508809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72bc15cb60 msgr2=0x7f72bc1f32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:56.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c1a90700 1 --2- 172.21.15.7:0/1466508809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72bc15cb60 0x7f72bc1f32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c1a90700 1 -- 172.21.15.7:0/1466508809 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b80094b0 con 0x7f72bc071c80 2024-09-13T18:52:56.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c1a90700 1 --2- 172.21.15.7:0/1466508809 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f72bc071c80 0x7f72bc144bd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f72b8016000 tx=0x7f72b8004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:56.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72b27fc700 1 -- 172.21.15.7:0/1466508809 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b8004590 con 0x7f72bc071c80 2024-09-13T18:52:56.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72b27fc700 1 -- 172.21.15.7:0/1466508809 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b80036c0 con 0x7f72bc071c80 2024-09-13T18:52:56.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c3cf4700 1 -- 172.21.15.7:0/1466508809 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72bc15d420 con 0x7f72bc071c80 2024-09-13T18:52:56.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72b27fc700 1 -- 172.21.15.7:0/1466508809 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b8003a70 con 0x7f72bc071c80 2024-09-13T18:52:56.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.029+0000 7f72c3cf4700 1 -- 172.21.15.7:0/1466508809 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72bc159680 con 0x7f72bc071c80 2024-09-13T18:52:56.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.033+0000 7f72c3cf4700 1 -- 172.21.15.7:0/1466508809 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72bc0ceab0 con 0x7f72bc071c80 2024-09-13T18:52:56.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.033+0000 7f72b27fc700 1 -- 172.21.15.7:0/1466508809 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f72b8003bd0 con 0x7f72bc071c80 2024-09-13T18:52:56.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.033+0000 7f72b27fc700 1 --2- 172.21.15.7:0/1466508809 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f72a0066730 0x7f72a0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:56.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.033+0000 7f72b27fc700 1 -- 172.21.15.7:0/1466508809 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72b8021030 con 0x7f72bc071c80 2024-09-13T18:52:56.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.033+0000 7f72c128f700 1 --2- 172.21.15.7:0/1466508809 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f72a0066730 0x7f72a0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:56.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.037+0000 7f72c128f700 1 --2- 172.21.15.7:0/1466508809 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f72a0066730 0x7f72a0068bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f72bc160700 tx=0x7f72ac009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:56.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.045+0000 7f72b27fc700 1 -- 172.21.15.7:0/1466508809 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f72b8052e50 con 0x7f72bc071c80 2024-09-13T18:52:56.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.217+0000 7f72c3cf4700 1 -- 172.21.15.7:0/1466508809 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f72bc15cf40 con 0x7f72bc071c80 2024-09-13T18:52:56.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.217+0000 7f72b27fc700 1 -- 172.21.15.7:0/1466508809 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f72b802b0a0 con 0x7f72bc071c80 2024-09-13T18:52:56.223 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:52:56.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.221+0000 7f72c3cf4700 1 -- 172.21.15.7:0/1466508809 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f72a0066730 msgr2=0x7f72a0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:56.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.221+0000 7f72c3cf4700 1 --2- 172.21.15.7:0/1466508809 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f72a0066730 0x7f72a0068bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f72bc160700 tx=0x7f72ac009240 comp rx=0 tx=0).stop 2024-09-13T18:52:56.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.221+0000 7f72c3cf4700 1 -- 172.21.15.7:0/1466508809 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f72bc071c80 msgr2=0x7f72bc144bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:56.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.221+0000 7f72c3cf4700 1 --2- 172.21.15.7:0/1466508809 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f72bc071c80 0x7f72bc144bd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f72b8016000 tx=0x7f72b8004770 comp rx=0 tx=0).stop 2024-09-13T18:52:56.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.221+0000 7f72c3cf4700 1 -- 172.21.15.7:0/1466508809 shutdown_connections 2024-09-13T18:52:56.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.221+0000 7f72c3cf4700 1 --2- 172.21.15.7:0/1466508809 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f72a0066730 0x7f72a0068bf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.221+0000 7f72c3cf4700 1 --2- 172.21.15.7:0/1466508809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f72bc160850 0x7f72bc1f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.221+0000 7f72c3cf4700 1 --2- 172.21.15.7:0/1466508809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72bc15cb60 0x7f72bc1f32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.221+0000 7f72c3cf4700 1 --2- 172.21.15.7:0/1466508809 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f72bc071c80 0x7f72bc144bd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.221+0000 7f72c3cf4700 1 -- 172.21.15.7:0/1466508809 >> 172.21.15.7:0/1466508809 conn(0x7f72bc0b1810 msgr2=0x7f72bc0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:56.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.221+0000 7f72c3cf4700 1 -- 172.21.15.7:0/1466508809 shutdown_connections 2024-09-13T18:52:56.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.221+0000 7f72c3cf4700 1 -- 172.21.15.7:0/1466508809 wait complete. 2024-09-13T18:52:56.243 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:52:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.357+0000 7fd4061b4700 1 Processor -- start 2024-09-13T18:52:56.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.357+0000 7fd4061b4700 1 -- start start 2024-09-13T18:52:56.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.357+0000 7fd4061b4700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd40015c4d0 0x7fd40006b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:56.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.357+0000 7fd4061b4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40006d120 con 0x7fd400159550 2024-09-13T18:52:56.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.357+0000 7fd4061b4700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40015e150 con 0x7fd40015a940 2024-09-13T18:52:56.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.357+0000 7fd4061b4700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40006bc20 con 0x7fd40015c4d0 2024-09-13T18:52:56.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.357+0000 7fd4051b2700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd400159550 0x7fd40015d330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38830/0 (socket says 172.21.15.7:38830) 2024-09-13T18:52:56.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.357+0000 7fd4051b2700 1 -- 172.21.15.7:0/653192270 learned_addr learned my addr 172.21.15.7:0/653192270 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:56.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.357+0000 7fd4059b3700 1 --2- 172.21.15.7:0/653192270 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd40015c4d0 0x7fd40006b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:56.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.357+0000 7fd3f7fff700 1 -- 172.21.15.7:0/653192270 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2032472743 0 0) 0x7fd40006d120 con 0x7fd400159550 2024-09-13T18:52:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.357+0000 7fd3f7fff700 1 -- 172.21.15.7:0/653192270 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3ec0035d0 con 0x7fd400159550 2024-09-13T18:52:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.361+0000 7fd4059b3700 1 -- 172.21.15.7:0/653192270 >> v1:172.21.15.171:6789/0 conn(0x7fd40015a940 legacy=0x7fd40015da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.361+0000 7fd4059b3700 1 -- 172.21.15.7:0/653192270 >> v1:172.21.15.7:6789/0 conn(0x7fd400159550 legacy=0x7fd40015d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.361+0000 7fd4059b3700 1 -- 172.21.15.7:0/653192270 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40006bf00 con 0x7fd40015c4d0 2024-09-13T18:52:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.361+0000 7fd4059b3700 1 --2- 172.21.15.7:0/653192270 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd40015c4d0 0x7fd40006b540 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8009dc0 tx=0x7fd3f800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea6e2ab623b73d36 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:56.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.369+0000 7fd3f7fff700 1 -- 172.21.15.7:0/653192270 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f800e070 con 0x7fd40015c4d0 2024-09-13T18:52:56.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.369+0000 7fd3f7fff700 1 -- 172.21.15.7:0/653192270 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f800bb20 con 0x7fd40015c4d0 2024-09-13T18:52:56.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.369+0000 7fd3f7fff700 1 -- 172.21.15.7:0/653192270 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8012430 con 0x7fd40015c4d0 2024-09-13T18:52:56.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.369+0000 7fd4061b4700 1 -- 172.21.15.7:0/653192270 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd40015c4d0 msgr2=0x7fd40006b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:56.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.369+0000 7fd4061b4700 1 --2- 172.21.15.7:0/653192270 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd40015c4d0 0x7fd40006b540 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8009dc0 tx=0x7fd3f800b330 comp rx=0 tx=0).stop 2024-09-13T18:52:56.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.369+0000 7fd4061b4700 1 -- 172.21.15.7:0/653192270 shutdown_connections 2024-09-13T18:52:56.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.369+0000 7fd4061b4700 1 --2- 172.21.15.7:0/653192270 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd40015c4d0 0x7fd40006b540 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.369+0000 7fd4061b4700 1 -- 172.21.15.7:0/653192270 >> 172.21.15.7:0/653192270 conn(0x7fd4000cf000 msgr2=0x7fd4000cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:56.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.369+0000 7fd4061b4700 1 -- 172.21.15.7:0/653192270 shutdown_connections 2024-09-13T18:52:56.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.369+0000 7fd4061b4700 1 -- 172.21.15.7:0/653192270 wait complete. 2024-09-13T18:52:56.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.369+0000 7fd4061b4700 1 Processor -- start 2024-09-13T18:52:56.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.369+0000 7fd4061b4700 1 -- start start 2024-09-13T18:52:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4061b4700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd400159550 0x7fd400162b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4061b4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40015a940 0x7fd4001555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4061b4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd40015c4d0 0x7fd400155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4061b4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4000cc6f0 con 0x7fd40015a940 2024-09-13T18:52:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4061b4700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4000cc330 con 0x7fd40015c4d0 2024-09-13T18:52:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4061b4700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd400151fe0 con 0x7fd400159550 2024-09-13T18:52:56.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4049b1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40015a940 0x7fd4001555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4051b2700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd400159550 0x7fd400162b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4049b1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40015a940 0x7fd4001555a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39136/0 (socket says 172.21.15.7:39136) 2024-09-13T18:52:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4049b1700 1 -- 172.21.15.7:0/2351544428 learned_addr learned my addr 172.21.15.7:0/2351544428 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:56.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4059b3700 1 --2- 172.21.15.7:0/2351544428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd40015c4d0 0x7fd400155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:56.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4049b1700 1 -- 172.21.15.7:0/2351544428 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd400159550 msgr2=0x7fd400162b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:56.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4049b1700 1 --2- 172.21.15.7:0/2351544428 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd400159550 0x7fd400162b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4049b1700 1 -- 172.21.15.7:0/2351544428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd40015c4d0 msgr2=0x7fd400155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:56.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4049b1700 1 --2- 172.21.15.7:0/2351544428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd40015c4d0 0x7fd400155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4049b1700 1 -- 172.21.15.7:0/2351544428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f8009a70 con 0x7fd40015a940 2024-09-13T18:52:56.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4049b1700 1 --2- 172.21.15.7:0/2351544428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40015a940 0x7fd4001555a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00c4d0 tx=0x7fd3fc00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:56.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/2351544428 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc0052d0 con 0x7fd40015a940 2024-09-13T18:52:56.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/2351544428 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3fc005430 con 0x7fd40015a940 2024-09-13T18:52:56.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/2351544428 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc0055a0 con 0x7fd40015a940 2024-09-13T18:52:56.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4061b4700 1 -- 172.21.15.7:0/2351544428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4001522c0 con 0x7fd40015a940 2024-09-13T18:52:56.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.373+0000 7fd4061b4700 1 -- 172.21.15.7:0/2351544428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4001630d0 con 0x7fd40015a940 2024-09-13T18:52:56.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.377+0000 7fd4061b4700 1 -- 172.21.15.7:0/2351544428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40006b540 con 0x7fd40015a940 2024-09-13T18:52:56.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.377+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/2351544428 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd3fc0038c0 con 0x7fd40015a940 2024-09-13T18:52:56.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.377+0000 7fd3f5ffb700 1 --2- 172.21.15.7:0/2351544428 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd3e4066760 0x7fd3e4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:56.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.377+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/2351544428 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3fc083400 con 0x7fd40015a940 2024-09-13T18:52:56.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.377+0000 7fd4051b2700 1 --2- 172.21.15.7:0/2351544428 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd3e4066760 0x7fd3e4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:56.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.377+0000 7fd4051b2700 1 --2- 172.21.15.7:0/2351544428 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd3e4066760 0x7fd3e4068c20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0007fd0 tx=0x7fd3f0007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:56.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.385+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/2351544428 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd3fc050800 con 0x7fd40015a940 2024-09-13T18:52:56.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.569+0000 7fd4061b4700 1 -- 172.21.15.7:0/2351544428 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd400070350 con 0x7fd40015a940 2024-09-13T18:52:56.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.573+0000 7fd3f5ffb700 1 -- 172.21.15.7:0/2351544428 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd3fc053e20 con 0x7fd40015a940 2024-09-13T18:52:56.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.573+0000 7fd4061b4700 1 -- 172.21.15.7:0/2351544428 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd3e4066760 msgr2=0x7fd3e4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:56.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.573+0000 7fd4061b4700 1 --2- 172.21.15.7:0/2351544428 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd3e4066760 0x7fd3e4068c20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0007fd0 tx=0x7fd3f0007de0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.573+0000 7fd4061b4700 1 -- 172.21.15.7:0/2351544428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40015a940 msgr2=0x7fd4001555a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:56.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.573+0000 7fd4061b4700 1 --2- 172.21.15.7:0/2351544428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40015a940 0x7fd4001555a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00c4d0 tx=0x7fd3fc00c7e0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.573+0000 7fd4061b4700 1 -- 172.21.15.7:0/2351544428 shutdown_connections 2024-09-13T18:52:56.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.573+0000 7fd4061b4700 1 --2- 172.21.15.7:0/2351544428 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd3e4066760 0x7fd3e4068c20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.573+0000 7fd4061b4700 1 --2- 172.21.15.7:0/2351544428 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd40015c4d0 0x7fd400155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.573+0000 7fd4061b4700 1 --2- 172.21.15.7:0/2351544428 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40015a940 0x7fd4001555a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.573+0000 7fd4061b4700 1 --2- 172.21.15.7:0/2351544428 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd400159550 0x7fd400162b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.573+0000 7fd4061b4700 1 -- 172.21.15.7:0/2351544428 >> 172.21.15.7:0/2351544428 conn(0x7fd4000cf000 msgr2=0x7fd400070800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:56.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.573+0000 7fd4061b4700 1 -- 172.21.15.7:0/2351544428 shutdown_connections 2024-09-13T18:52:56.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.573+0000 7fd4061b4700 1 -- 172.21.15.7:0/2351544428 wait complete. 2024-09-13T18:52:56.582 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-13T18:52:56.593 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi171.front.sepia.ceph.com... 2024-09-13T18:52:56.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:56.594 DEBUG:teuthology.orchestra.run.smithi171:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-13T18:52:56.620 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:52:56.620 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T18:52:56.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:52:56.621 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-13T18:52:56.657 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:52:56.657 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:52:56.665 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:52:56.665 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:52:56.716 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:56.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.821+0000 7f16df0e7700 1 Processor -- start 2024-09-13T18:52:56.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.821+0000 7f16df0e7700 1 -- start start 2024-09-13T18:52:56.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.821+0000 7f16df0e7700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f16d815cae0 0x7f16d80cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:56.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.821+0000 7f16df0e7700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f16d815cec0 0x7f16d80d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:56.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.821+0000 7f16df0e7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d80d21e0 con 0x7f16d806ac00 2024-09-13T18:52:56.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.821+0000 7f16df0e7700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d80d1750 con 0x7f16d815cec0 2024-09-13T18:52:56.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.821+0000 7f16df0e7700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80d1940 con 0x7f16d815cae0 2024-09-13T18:52:56.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.821+0000 7f16de8e6700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f16d806ac00 0x7f16d80cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38832/0 (socket says 172.21.15.7:38832) 2024-09-13T18:52:56.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.821+0000 7f16de8e6700 1 -- 172.21.15.7:0/980730873 learned_addr learned my addr 172.21.15.7:0/980730873 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:56.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.821+0000 7f16de0e5700 1 --2- 172.21.15.7:0/980730873 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f16d815cae0 0x7f16d80cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:56.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.821+0000 7f16dd8e4700 1 --2- 172.21.15.7:0/980730873 >> v2:172.21.15.171:3300/0 conn(0x7f16d815cec0 0x7f16d80d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:56.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16dd0e3700 1 -- 172.21.15.7:0/980730873 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140908643 0 0) 0x7f16d80d21e0 con 0x7f16d806ac00 2024-09-13T18:52:56.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16dd0e3700 1 -- 172.21.15.7:0/980730873 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16c40035d0 con 0x7f16d806ac00 2024-09-13T18:52:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16de0e5700 1 -- 172.21.15.7:0/980730873 >> v2:172.21.15.171:3300/0 conn(0x7f16d815cec0 msgr2=0x7f16d80d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16de0e5700 1 --2- 172.21.15.7:0/980730873 >> v2:172.21.15.171:3300/0 conn(0x7f16d815cec0 0x7f16d80d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16de0e5700 1 -- 172.21.15.7:0/980730873 >> v1:172.21.15.7:6789/0 conn(0x7f16d806ac00 legacy=0x7f16d80cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16de0e5700 1 -- 172.21.15.7:0/980730873 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d80d1c20 con 0x7f16d815cae0 2024-09-13T18:52:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16de0e5700 1 --2- 172.21.15.7:0/980730873 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f16d815cae0 0x7f16d80cf030 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009850 tx=0x7f16d4009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=473916da4d3378ab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16dd0e3700 1 -- 172.21.15.7:0/980730873 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4011830 con 0x7f16d815cae0 2024-09-13T18:52:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16dd0e3700 1 -- 172.21.15.7:0/980730873 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d4011990 con 0x7f16d815cae0 2024-09-13T18:52:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16dd0e3700 1 -- 172.21.15.7:0/980730873 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4011d40 con 0x7f16d815cae0 2024-09-13T18:52:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16df0e7700 1 -- 172.21.15.7:0/980730873 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f16d815cae0 msgr2=0x7f16d80cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16df0e7700 1 --2- 172.21.15.7:0/980730873 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f16d815cae0 0x7f16d80cf030 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f16d4009850 tx=0x7f16d4009b60 comp rx=0 tx=0).stop 2024-09-13T18:52:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16df0e7700 1 -- 172.21.15.7:0/980730873 shutdown_connections 2024-09-13T18:52:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16df0e7700 1 --2- 172.21.15.7:0/980730873 >> v2:172.21.15.171:3300/0 conn(0x7f16d815cec0 0x7f16d80d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16df0e7700 1 --2- 172.21.15.7:0/980730873 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f16d815cae0 0x7f16d80cf030 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16df0e7700 1 -- 172.21.15.7:0/980730873 >> 172.21.15.7:0/980730873 conn(0x7f16d80d2c80 msgr2=0x7f16d80d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16df0e7700 1 -- 172.21.15.7:0/980730873 shutdown_connections 2024-09-13T18:52:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16df0e7700 1 -- 172.21.15.7:0/980730873 wait complete. 2024-09-13T18:52:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.825+0000 7f16df0e7700 1 Processor -- start 2024-09-13T18:52:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16df0e7700 1 -- start start 2024-09-13T18:52:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16df0e7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16d806ac00 0x7f16d80c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16df0e7700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f16d815cae0 0x7f16d81588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16df0e7700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16d815cec0 0x7f16d8158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16df0e7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80b4610 con 0x7f16d806ac00 2024-09-13T18:52:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16df0e7700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80b3a90 con 0x7f16d815cec0 2024-09-13T18:52:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16df0e7700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80b3d70 con 0x7f16d815cae0 2024-09-13T18:52:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16de0e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16d806ac00 0x7f16d80c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16de0e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16d806ac00 0x7f16d80c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39152/0 (socket says 172.21.15.7:39152) 2024-09-13T18:52:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16de0e5700 1 -- 172.21.15.7:0/3176844106 learned_addr learned my addr 172.21.15.7:0/3176844106 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16dd8e4700 1 --2- 172.21.15.7:0/3176844106 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f16d815cae0 0x7f16d81588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16de8e6700 1 --2- 172.21.15.7:0/3176844106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16d815cec0 0x7f16d8158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16de0e5700 1 -- 172.21.15.7:0/3176844106 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f16d815cae0 msgr2=0x7f16d81588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16de0e5700 1 --2- 172.21.15.7:0/3176844106 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f16d815cae0 0x7f16d81588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16de0e5700 1 -- 172.21.15.7:0/3176844106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16d815cec0 msgr2=0x7f16d8158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16de0e5700 1 --2- 172.21.15.7:0/3176844106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16d815cec0 0x7f16d8158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16de0e5700 1 -- 172.21.15.7:0/3176844106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d40094b0 con 0x7f16d806ac00 2024-09-13T18:52:56.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16dd8e4700 1 --2- 172.21.15.7:0/3176844106 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f16d815cae0 0x7f16d81588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16de0e5700 1 --2- 172.21.15.7:0/3176844106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16d806ac00 0x7f16d80c8ce0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f16d40090b0 tx=0x7f16d4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16ceffd700 1 -- 172.21.15.7:0/3176844106 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4004480 con 0x7f16d806ac00 2024-09-13T18:52:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16ceffd700 1 -- 172.21.15.7:0/3176844106 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d401f070 con 0x7f16d806ac00 2024-09-13T18:52:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16df0e7700 1 -- 172.21.15.7:0/3176844106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d80b4050 con 0x7f16d806ac00 2024-09-13T18:52:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16ceffd700 1 -- 172.21.15.7:0/3176844106 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4003900 con 0x7f16d806ac00 2024-09-13T18:52:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.829+0000 7f16df0e7700 1 -- 172.21.15.7:0/3176844106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d80c9220 con 0x7f16d806ac00 2024-09-13T18:52:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.833+0000 7f16ceffd700 1 -- 172.21.15.7:0/3176844106 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f16d4021030 con 0x7f16d806ac00 2024-09-13T18:52:56.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.833+0000 7f16df0e7700 1 -- 172.21.15.7:0/3176844106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d81f3370 con 0x7f16d806ac00 2024-09-13T18:52:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.833+0000 7f16ceffd700 1 --2- 172.21.15.7:0/3176844106 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f16bc066730 0x7f16bc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:56.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.833+0000 7f16dd8e4700 1 --2- 172.21.15.7:0/3176844106 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f16bc066730 0x7f16bc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.833+0000 7f16ceffd700 1 -- 172.21.15.7:0/3176844106 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16d4085480 con 0x7f16d806ac00 2024-09-13T18:52:56.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.837+0000 7f16dd8e4700 1 --2- 172.21.15.7:0/3176844106 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f16bc066730 0x7f16bc068bf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f16d80ca2c0 tx=0x7f16c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:56.841+0000 7f16ceffd700 1 -- 172.21.15.7:0/3176844106 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f16d81f3370 con 0x7f16d806ac00 2024-09-13T18:52:57.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.017+0000 7f16df0e7700 1 -- 172.21.15.7:0/3176844106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16d81f3370 con 0x7f16d806ac00 2024-09-13T18:52:57.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.021+0000 7f16ceffd700 1 -- 172.21.15.7:0/3176844106 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f16d81f3370 con 0x7f16d806ac00 2024-09-13T18:52:57.028 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:57.028 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.025+0000 7f16df0e7700 1 -- 172.21.15.7:0/3176844106 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f16bc066730 msgr2=0x7f16bc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.025+0000 7f16df0e7700 1 --2- 172.21.15.7:0/3176844106 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f16bc066730 0x7f16bc068bf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f16d80ca2c0 tx=0x7f16c8008040 comp rx=0 tx=0).stop 2024-09-13T18:52:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.025+0000 7f16df0e7700 1 -- 172.21.15.7:0/3176844106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16d806ac00 msgr2=0x7f16d80c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.025+0000 7f16df0e7700 1 --2- 172.21.15.7:0/3176844106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16d806ac00 0x7f16d80c8ce0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f16d40090b0 tx=0x7f16d4004770 comp rx=0 tx=0).stop 2024-09-13T18:52:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.025+0000 7f16df0e7700 1 -- 172.21.15.7:0/3176844106 shutdown_connections 2024-09-13T18:52:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.025+0000 7f16df0e7700 1 --2- 172.21.15.7:0/3176844106 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f16bc066730 0x7f16bc068bf0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.025+0000 7f16df0e7700 1 --2- 172.21.15.7:0/3176844106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f16d815cec0 0x7f16d8158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.025+0000 7f16df0e7700 1 --2- 172.21.15.7:0/3176844106 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f16d815cae0 0x7f16d81588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.025+0000 7f16df0e7700 1 --2- 172.21.15.7:0/3176844106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16d806ac00 0x7f16d80c8ce0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.025+0000 7f16df0e7700 1 -- 172.21.15.7:0/3176844106 >> 172.21.15.7:0/3176844106 conn(0x7f16d80d2c80 msgr2=0x7f16d80caa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.025+0000 7f16df0e7700 1 -- 172.21.15.7:0/3176844106 shutdown_connections 2024-09-13T18:52:57.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.025+0000 7f16df0e7700 1 -- 172.21.15.7:0/3176844106 wait complete. 2024-09-13T18:52:57.044 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:52:15.554683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:52:16.896879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:52:57.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:52:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f746c804700 1 Processor -- start 2024-09-13T18:52:57.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f746c804700 1 -- start start 2024-09-13T18:52:57.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f746c804700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f7460036d80 0x7f746000adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f746c804700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7460050360 con 0x7f746004fc10 2024-09-13T18:52:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f746c804700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f746001d350 con 0x7f74600e2d60 2024-09-13T18:52:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f746c804700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f746001d630 con 0x7f7460036d80 2024-09-13T18:52:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f746659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f746004fc10 0x7f746000a6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38838/0 (socket says 172.21.15.7:38838) 2024-09-13T18:52:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f746659c700 1 -- 172.21.15.7:0/1731851941 learned_addr learned my addr 172.21.15.7:0/1731851941 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f746759e700 1 --2- 172.21.15.7:0/1731851941 >> v2:172.21.15.171:3300/0 conn(0x7f7460036d80 0x7f746000adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f7465d9b700 1 -- 172.21.15.7:0/1731851941 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984705445 0 0) 0x7f7460050360 con 0x7f746004fc10 2024-09-13T18:52:57.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f7465d9b700 1 -- 172.21.15.7:0/1731851941 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74500035d0 con 0x7f746004fc10 2024-09-13T18:52:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f7465d9b700 1 -- 172.21.15.7:0/1731851941 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4045051869 0 0) 0x7f74500035d0 con 0x7f746004fc10 2024-09-13T18:52:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f746759e700 1 -- 172.21.15.7:0/1731851941 >> v1:172.21.15.171:6789/0 conn(0x7f74600e2d60 legacy=0x7f746004f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f746759e700 1 -- 172.21.15.7:0/1731851941 >> v1:172.21.15.7:6789/0 conn(0x7f746004fc10 legacy=0x7f746000a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f746759e700 1 -- 172.21.15.7:0/1731851941 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746001d910 con 0x7f7460036d80 2024-09-13T18:52:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f746759e700 1 --2- 172.21.15.7:0/1731851941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7460036d80 0x7f746000adc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f746804f480 tx=0x7f7468074840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=293775ba4b46e357 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f7465d9b700 1 -- 172.21.15.7:0/1731851941 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c001670 con 0x7f7460036d80 2024-09-13T18:52:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f7465d9b700 1 -- 172.21.15.7:0/1731851941 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746808e070 con 0x7f7460036d80 2024-09-13T18:52:57.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.161+0000 7f7465d9b700 1 -- 172.21.15.7:0/1731851941 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7468073f20 con 0x7f7460036d80 2024-09-13T18:52:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f746c804700 1 -- 172.21.15.7:0/1731851941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7460036d80 msgr2=0x7f746000adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f746c804700 1 --2- 172.21.15.7:0/1731851941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7460036d80 0x7f746000adc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f746804f480 tx=0x7f7468074840 comp rx=0 tx=0).stop 2024-09-13T18:52:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f746c804700 1 -- 172.21.15.7:0/1731851941 shutdown_connections 2024-09-13T18:52:57.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f746c804700 1 --2- 172.21.15.7:0/1731851941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7460036d80 0x7f746000adc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f746c804700 1 -- 172.21.15.7:0/1731851941 >> 172.21.15.7:0/1731851941 conn(0x7f7460036140 msgr2=0x7f7460036550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f746c804700 1 -- 172.21.15.7:0/1731851941 shutdown_connections 2024-09-13T18:52:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f746c804700 1 -- 172.21.15.7:0/1731851941 wait complete. 2024-09-13T18:52:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f746c804700 1 Processor -- start 2024-09-13T18:52:57.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f746c804700 1 -- start start 2024-09-13T18:52:57.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f746c804700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7460036d80 0x7f74600387c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f746c804700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f746004fc10 0x7f74600dde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f746c804700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7460179750 0x7f7460179be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f746c804700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74600383a0 con 0x7f746004fc10 2024-09-13T18:52:57.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f746c804700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7460176020 con 0x7f7460179750 2024-09-13T18:52:57.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f746c804700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7460037480 con 0x7f7460036d80 2024-09-13T18:52:57.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f746659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f746004fc10 0x7f74600dde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f746659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f746004fc10 0x7f74600dde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39164/0 (socket says 172.21.15.7:39164) 2024-09-13T18:52:57.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f746659c700 1 -- 172.21.15.7:0/3353811407 learned_addr learned my addr 172.21.15.7:0/3353811407 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:57.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.165+0000 7f7466d9d700 1 --2- 172.21.15.7:0/3353811407 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7460036d80 0x7f74600387c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.169+0000 7f746759e700 1 --2- 172.21.15.7:0/3353811407 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7460179750 0x7f7460179be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.169+0000 7f746659c700 1 -- 172.21.15.7:0/3353811407 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7460036d80 msgr2=0x7f74600387c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.169+0000 7f746659c700 1 --2- 172.21.15.7:0/3353811407 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7460036d80 0x7f74600387c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.169+0000 7f746659c700 1 -- 172.21.15.7:0/3353811407 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7460179750 msgr2=0x7f7460179be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.169+0000 7f746659c700 1 --2- 172.21.15.7:0/3353811407 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7460179750 0x7f7460179be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.169+0000 7f746659c700 1 -- 172.21.15.7:0/3353811407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746804f5c0 con 0x7f746004fc10 2024-09-13T18:52:57.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.169+0000 7f746659c700 1 --2- 172.21.15.7:0/3353811407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f746004fc10 0x7f74600dde10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f745c00c770 tx=0x7f745c00ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:57.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.169+0000 7f744f7fe700 1 -- 172.21.15.7:0/3353811407 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c004ca0 con 0x7f746004fc10 2024-09-13T18:52:57.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.169+0000 7f744f7fe700 1 -- 172.21.15.7:0/3353811407 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c004e00 con 0x7f746004fc10 2024-09-13T18:52:57.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.169+0000 7f744f7fe700 1 -- 172.21.15.7:0/3353811407 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c0072f0 con 0x7f746004fc10 2024-09-13T18:52:57.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.169+0000 7f746c804700 1 -- 172.21.15.7:0/3353811407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7460037720 con 0x7f746004fc10 2024-09-13T18:52:57.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.169+0000 7f746c804700 1 -- 172.21.15.7:0/3353811407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7460037ce0 con 0x7f746004fc10 2024-09-13T18:52:57.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.169+0000 7f744f7fe700 1 -- 172.21.15.7:0/3353811407 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f745c007450 con 0x7f746004fc10 2024-09-13T18:52:57.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.169+0000 7f746c804700 1 -- 172.21.15.7:0/3353811407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7460004800 con 0x7f746004fc10 2024-09-13T18:52:57.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.169+0000 7f744f7fe700 1 --2- 172.21.15.7:0/3353811407 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7454066730 0x7f7454068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.169+0000 7f744f7fe700 1 -- 172.21.15.7:0/3353811407 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f745c083e10 con 0x7f746004fc10 2024-09-13T18:52:57.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.173+0000 7f7466d9d700 1 --2- 172.21.15.7:0/3353811407 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7454066730 0x7f7454068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.173+0000 7f7466d9d700 1 --2- 172.21.15.7:0/3353811407 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7454066730 0x7f7454068bf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7458000fb0 tx=0x7f7458009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:57.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.181+0000 7f744f7fe700 1 -- 172.21.15.7:0/3353811407 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f745c051180 con 0x7f746004fc10 2024-09-13T18:52:57.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.353+0000 7f746c804700 1 -- 172.21.15.7:0/3353811407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74600384e0 con 0x7f746004fc10 2024-09-13T18:52:57.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.353+0000 7f744f7fe700 1 -- 172.21.15.7:0/3353811407 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f74600384e0 con 0x7f746004fc10 2024-09-13T18:52:57.360 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:52:57.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.357+0000 7f746c804700 1 -- 172.21.15.7:0/3353811407 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7454066730 msgr2=0x7f7454068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.357+0000 7f746c804700 1 --2- 172.21.15.7:0/3353811407 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7454066730 0x7f7454068bf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7458000fb0 tx=0x7f7458009040 comp rx=0 tx=0).stop 2024-09-13T18:52:57.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.357+0000 7f746c804700 1 -- 172.21.15.7:0/3353811407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f746004fc10 msgr2=0x7f74600dde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.357+0000 7f746c804700 1 --2- 172.21.15.7:0/3353811407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f746004fc10 0x7f74600dde10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f745c00c770 tx=0x7f745c00ca80 comp rx=0 tx=0).stop 2024-09-13T18:52:57.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.357+0000 7f746c804700 1 -- 172.21.15.7:0/3353811407 shutdown_connections 2024-09-13T18:52:57.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.357+0000 7f746c804700 1 --2- 172.21.15.7:0/3353811407 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7454066730 0x7f7454068bf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.357+0000 7f746c804700 1 --2- 172.21.15.7:0/3353811407 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7460179750 0x7f7460179be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.357+0000 7f746c804700 1 --2- 172.21.15.7:0/3353811407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f746004fc10 0x7f74600dde10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.357+0000 7f746c804700 1 --2- 172.21.15.7:0/3353811407 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7460036d80 0x7f74600387c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.357+0000 7f746c804700 1 -- 172.21.15.7:0/3353811407 >> 172.21.15.7:0/3353811407 conn(0x7f7460036140 msgr2=0x7f7460034070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:57.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.357+0000 7f746c804700 1 -- 172.21.15.7:0/3353811407 shutdown_connections 2024-09-13T18:52:57.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.357+0000 7f746c804700 1 -- 172.21.15.7:0/3353811407 wait complete. 2024-09-13T18:52:57.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:52:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c637b700 1 Processor -- start 2024-09-13T18:52:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c637b700 1 -- start start 2024-09-13T18:52:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c637b700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fe3c006f3a0 0x7fe3c015d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c637b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe3c006f780 0x7fe3c0142c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c637b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe3c01401e0 0x7fe3c00b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c637b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c01431c0 con 0x7fe3c006f780 2024-09-13T18:52:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c637b700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c00cc340 con 0x7fe3c006f3a0 2024-09-13T18:52:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c637b700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c015d760 con 0x7fe3c01401e0 2024-09-13T18:52:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c4b78700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe3c006f780 0x7fe3c0142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c4b78700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe3c006f780 0x7fe3c0142c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39170/0 (socket says 172.21.15.7:39170) 2024-09-13T18:52:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c4b78700 1 -- 172.21.15.7:0/3472228423 learned_addr learned my addr 172.21.15.7:0/3472228423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c5379700 1 --2- 172.21.15.7:0/3472228423 >> v2:172.21.15.171:3300/0 conn(0x7fe3c006f3a0 0x7fe3c015d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c5b7a700 1 --2- 172.21.15.7:0/3472228423 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe3c01401e0 0x7fe3c00b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c4b78700 1 -- 172.21.15.7:0/3472228423 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe3c01401e0 msgr2=0x7fe3c00b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c4b78700 1 --2- 172.21.15.7:0/3472228423 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe3c01401e0 0x7fe3c00b2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c4b78700 1 -- 172.21.15.7:0/3472228423 >> v2:172.21.15.171:3300/0 conn(0x7fe3c006f3a0 msgr2=0x7fe3c015d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c4b78700 1 --2- 172.21.15.7:0/3472228423 >> v2:172.21.15.171:3300/0 conn(0x7fe3c006f3a0 0x7fe3c015d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c4b78700 1 -- 172.21.15.7:0/3472228423 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c015da40 con 0x7fe3c006f780 2024-09-13T18:52:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c5379700 1 --2- 172.21.15.7:0/3472228423 >> v2:172.21.15.171:3300/0 conn(0x7fe3c006f3a0 0x7fe3c015d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3c4b78700 1 --2- 172.21.15.7:0/3472228423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3c006f780 0x7fe3c0142c80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00b6ca0 tx=0x7fe3b0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55dd95fcb5cf30cf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3b7fff700 1 -- 172.21.15.7:0/3472228423 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b000b8a0 con 0x7fe3c006f780 2024-09-13T18:52:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3b7fff700 1 -- 172.21.15.7:0/3472228423 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b000ba00 con 0x7fe3c006f780 2024-09-13T18:52:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.493+0000 7fe3b7fff700 1 -- 172.21.15.7:0/3472228423 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b000bd90 con 0x7fe3c006f780 2024-09-13T18:52:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.497+0000 7fe3c637b700 1 -- 172.21.15.7:0/3472228423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3c006f780 msgr2=0x7fe3c0142c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.497+0000 7fe3c637b700 1 --2- 172.21.15.7:0/3472228423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3c006f780 0x7fe3c0142c80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00b6ca0 tx=0x7fe3b0009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.497+0000 7fe3c637b700 1 -- 172.21.15.7:0/3472228423 shutdown_connections 2024-09-13T18:52:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.497+0000 7fe3c637b700 1 --2- 172.21.15.7:0/3472228423 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe3c01401e0 0x7fe3c00b2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.497+0000 7fe3c637b700 1 --2- 172.21.15.7:0/3472228423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3c006f780 0x7fe3c0142c80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.497+0000 7fe3c637b700 1 --2- 172.21.15.7:0/3472228423 >> v2:172.21.15.171:3300/0 conn(0x7fe3c006f3a0 0x7fe3c015d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.497+0000 7fe3c637b700 1 -- 172.21.15.7:0/3472228423 >> 172.21.15.7:0/3472228423 conn(0x7fe3c0071ce0 msgr2=0x7fe3c00720c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:57.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.497+0000 7fe3c637b700 1 -- 172.21.15.7:0/3472228423 shutdown_connections 2024-09-13T18:52:57.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.497+0000 7fe3c637b700 1 -- 172.21.15.7:0/3472228423 wait complete. 2024-09-13T18:52:57.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.497+0000 7fe3c637b700 1 Processor -- start 2024-09-13T18:52:57.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.497+0000 7fe3c637b700 1 -- start start 2024-09-13T18:52:57.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c637b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3c006f3a0 0x7fe3c01f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c637b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe3c006f780 0x7fe3c006d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c637b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3c01401e0 0x7fe3c015f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c637b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c006ea10 con 0x7fe3c01401e0 2024-09-13T18:52:57.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c637b700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c0154a20 con 0x7fe3c006f3a0 2024-09-13T18:52:57.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c637b700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c006e420 con 0x7fe3c006f780 2024-09-13T18:52:57.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c5b7a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3c01401e0 0x7fe3c015f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c5b7a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3c01401e0 0x7fe3c015f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39174/0 (socket says 172.21.15.7:39174) 2024-09-13T18:52:57.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c5379700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3c006f3a0 0x7fe3c01f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c5b7a700 1 -- 172.21.15.7:0/61957388 learned_addr learned my addr 172.21.15.7:0/61957388 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:57.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c4b78700 1 --2- 172.21.15.7:0/61957388 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe3c006f780 0x7fe3c006d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c5b7a700 1 -- 172.21.15.7:0/61957388 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe3c006f780 msgr2=0x7fe3c006d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c5b7a700 1 --2- 172.21.15.7:0/61957388 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe3c006f780 0x7fe3c006d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c5b7a700 1 -- 172.21.15.7:0/61957388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3c006f3a0 msgr2=0x7fe3c01f2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c5b7a700 1 --2- 172.21.15.7:0/61957388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3c006f3a0 0x7fe3c01f2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c5b7a700 1 -- 172.21.15.7:0/61957388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b0009570 con 0x7fe3c01401e0 2024-09-13T18:52:57.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c5379700 1 --2- 172.21.15.7:0/61957388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3c006f3a0 0x7fe3c01f2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:57.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c5b7a700 1 --2- 172.21.15.7:0/61957388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3c01401e0 0x7fe3c015f630 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8007e60 tx=0x7fe3b800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:57.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3b5ffb700 1 -- 172.21.15.7:0/61957388 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b800ed30 con 0x7fe3c01401e0 2024-09-13T18:52:57.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3b5ffb700 1 -- 172.21.15.7:0/61957388 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b8014420 con 0x7fe3c01401e0 2024-09-13T18:52:57.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c637b700 1 -- 172.21.15.7:0/61957388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c01407d0 con 0x7fe3c01401e0 2024-09-13T18:52:57.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3b5ffb700 1 -- 172.21.15.7:0/61957388 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b80147d0 con 0x7fe3c01401e0 2024-09-13T18:52:57.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.501+0000 7fe3c637b700 1 -- 172.21.15.7:0/61957388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c01f2e90 con 0x7fe3c01401e0 2024-09-13T18:52:57.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.505+0000 7fe3c637b700 1 -- 172.21.15.7:0/61957388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c006eb50 con 0x7fe3c01401e0 2024-09-13T18:52:57.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.505+0000 7fe3b5ffb700 1 -- 172.21.15.7:0/61957388 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe3b8014930 con 0x7fe3c01401e0 2024-09-13T18:52:57.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.505+0000 7fe3b5ffb700 1 --2- 172.21.15.7:0/61957388 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe3a4066730 0x7fe3a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.505+0000 7fe3b5ffb700 1 -- 172.21.15.7:0/61957388 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3b8084600 con 0x7fe3c01401e0 2024-09-13T18:52:57.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.505+0000 7fe3c5379700 1 --2- 172.21.15.7:0/61957388 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe3a4066730 0x7fe3a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.505+0000 7fe3c5379700 1 --2- 172.21.15.7:0/61957388 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe3a4066730 0x7fe3a4068bf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009eb0 tx=0x7fe3bc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:57.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.513+0000 7fe3b5ffb700 1 -- 172.21.15.7:0/61957388 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe3c006eb50 con 0x7fe3c01401e0 2024-09-13T18:52:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.685+0000 7fe3c637b700 1 -- 172.21.15.7:0/61957388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3c006eb50 con 0x7fe3c01401e0 2024-09-13T18:52:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.689+0000 7fe3b5ffb700 1 -- 172.21.15.7:0/61957388 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe3c006eb50 con 0x7fe3c01401e0 2024-09-13T18:52:57.693 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:52:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.689+0000 7fe3c637b700 1 -- 172.21.15.7:0/61957388 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe3a4066730 msgr2=0x7fe3a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.689+0000 7fe3c637b700 1 --2- 172.21.15.7:0/61957388 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe3a4066730 0x7fe3a4068bf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009eb0 tx=0x7fe3bc008040 comp rx=0 tx=0).stop 2024-09-13T18:52:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.689+0000 7fe3c637b700 1 -- 172.21.15.7:0/61957388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3c01401e0 msgr2=0x7fe3c015f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.689+0000 7fe3c637b700 1 --2- 172.21.15.7:0/61957388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3c01401e0 0x7fe3c015f630 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8007e60 tx=0x7fe3b800e3c0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.689+0000 7fe3c637b700 1 -- 172.21.15.7:0/61957388 shutdown_connections 2024-09-13T18:52:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.689+0000 7fe3c637b700 1 --2- 172.21.15.7:0/61957388 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe3a4066730 0x7fe3a4068bf0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.689+0000 7fe3c637b700 1 --2- 172.21.15.7:0/61957388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3c01401e0 0x7fe3c015f630 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.689+0000 7fe3c637b700 1 --2- 172.21.15.7:0/61957388 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe3c006f780 0x7fe3c006d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.689+0000 7fe3c637b700 1 --2- 172.21.15.7:0/61957388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe3c006f3a0 0x7fe3c01f2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.689+0000 7fe3c637b700 1 -- 172.21.15.7:0/61957388 >> 172.21.15.7:0/61957388 conn(0x7fe3c0071ce0 msgr2=0x7fe3c00720c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:57.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.689+0000 7fe3c637b700 1 -- 172.21.15.7:0/61957388 shutdown_connections 2024-09-13T18:52:57.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.689+0000 7fe3c637b700 1 -- 172.21.15.7:0/61957388 wait complete. 2024-09-13T18:52:57.709 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:52:57.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.817+0000 7f2378caa700 1 Processor -- start 2024-09-13T18:52:57.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.817+0000 7f2378caa700 1 -- start start 2024-09-13T18:52:57.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f2378caa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f237415cda0 0x7f237406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f2378caa700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f237406a520 0x7f237406a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f2378caa700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23740b8430 con 0x7f23740ceb70 2024-09-13T18:52:57.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f2378caa700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237406be40 con 0x7f237415cda0 2024-09-13T18:52:57.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f2378caa700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23740b64f0 con 0x7f237406a520 2024-09-13T18:52:57.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f23737fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f237415cda0 0x7f237406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f23737fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f237415cda0 0x7f237406b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39190/0 (socket says 172.21.15.7:39190) 2024-09-13T18:52:57.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f23737fe700 1 -- 172.21.15.7:0/3615674026 learned_addr learned my addr 172.21.15.7:0/3615674026 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:57.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f2373fff700 1 --2- 172.21.15.7:0/3615674026 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f237406a520 0x7f237406a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f23737fe700 1 -- 172.21.15.7:0/3615674026 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f237406a520 msgr2=0x7f237406a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:57.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f23737fe700 1 --2- 172.21.15.7:0/3615674026 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f237406a520 0x7f237406a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f23737fe700 1 -- 172.21.15.7:0/3615674026 >> v1:172.21.15.171:6789/0 conn(0x7f23740ceb70 legacy=0x7f23740b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f23737fe700 1 -- 172.21.15.7:0/3615674026 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237406ae90 con 0x7f237415cda0 2024-09-13T18:52:57.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f23737fe700 1 --2- 172.21.15.7:0/3615674026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237415cda0 0x7f237406b900 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2364009910 tx=0x7f2364009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9bcf0009f8e0bc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:57.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f2372ffd700 1 -- 172.21.15.7:0/3615674026 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364011830 con 0x7f237415cda0 2024-09-13T18:52:57.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f2372ffd700 1 -- 172.21.15.7:0/3615674026 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2364011990 con 0x7f237415cda0 2024-09-13T18:52:57.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f2372ffd700 1 -- 172.21.15.7:0/3615674026 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364011d40 con 0x7f237415cda0 2024-09-13T18:52:57.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f2378caa700 1 -- 172.21.15.7:0/3615674026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237415cda0 msgr2=0x7f237406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f2378caa700 1 --2- 172.21.15.7:0/3615674026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237415cda0 0x7f237406b900 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2364009910 tx=0x7f2364009c20 comp rx=0 tx=0).stop 2024-09-13T18:52:57.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f2378caa700 1 -- 172.21.15.7:0/3615674026 shutdown_connections 2024-09-13T18:52:57.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f2378caa700 1 --2- 172.21.15.7:0/3615674026 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f237406a520 0x7f237406a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f2378caa700 1 --2- 172.21.15.7:0/3615674026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237415cda0 0x7f237406b900 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f2378caa700 1 -- 172.21.15.7:0/3615674026 >> 172.21.15.7:0/3615674026 conn(0x7f23740d2cb0 msgr2=0x7f23740d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:57.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f2378caa700 1 -- 172.21.15.7:0/3615674026 shutdown_connections 2024-09-13T18:52:57.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.821+0000 7f2378caa700 1 -- 172.21.15.7:0/3615674026 wait complete. 2024-09-13T18:52:57.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.825+0000 7f2378caa700 1 Processor -- start 2024-09-13T18:52:57.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.825+0000 7f2378caa700 1 -- start start 2024-09-13T18:52:57.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.825+0000 7f2378caa700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f237406a520 0x7f237414f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.825+0000 7f2378caa700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f23740ceb70 0x7f23741f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.825+0000 7f2378caa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237415cda0 0x7f23741f2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.825+0000 7f2378caa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237414f950 con 0x7f237415cda0 2024-09-13T18:52:57.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.825+0000 7f2378caa700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740c67e0 con 0x7f237406a520 2024-09-13T18:52:57.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.825+0000 7f2378caa700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23741f3080 con 0x7f23740ceb70 2024-09-13T18:52:57.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.825+0000 7f2373fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237415cda0 0x7f23741f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.825+0000 7f2373fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237415cda0 0x7f23741f2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39200/0 (socket says 172.21.15.7:39200) 2024-09-13T18:52:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.825+0000 7f2373fff700 1 -- 172.21.15.7:0/2220204135 learned_addr learned my addr 172.21.15.7:0/2220204135 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.825+0000 7f23737fe700 1 --2- 172.21.15.7:0/2220204135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f237406a520 0x7f237414f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.825+0000 7f236affd700 1 --2- 172.21.15.7:0/2220204135 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f23740ceb70 0x7f23741f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.829+0000 7f2373fff700 1 -- 172.21.15.7:0/2220204135 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f23740ceb70 msgr2=0x7f23741f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.829+0000 7f2373fff700 1 --2- 172.21.15.7:0/2220204135 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f23740ceb70 0x7f23741f3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.829+0000 7f2373fff700 1 -- 172.21.15.7:0/2220204135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f237406a520 msgr2=0x7f237414f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:57.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.829+0000 7f2373fff700 1 --2- 172.21.15.7:0/2220204135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f237406a520 0x7f237414f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:57.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.829+0000 7f2373fff700 1 -- 172.21.15.7:0/2220204135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2364009570 con 0x7f237415cda0 2024-09-13T18:52:57.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.829+0000 7f23737fe700 1 --2- 172.21.15.7:0/2220204135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f237406a520 0x7f237414f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:57.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.829+0000 7f2373fff700 1 --2- 172.21.15.7:0/2220204135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237415cda0 0x7f23741f2750 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f236c00efc0 tx=0x7f236c00c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:57.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.829+0000 7f2370ff9700 1 -- 172.21.15.7:0/2220204135 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c00cad0 con 0x7f237415cda0 2024-09-13T18:52:57.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.829+0000 7f2370ff9700 1 -- 172.21.15.7:0/2220204135 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c00f040 con 0x7f237415cda0 2024-09-13T18:52:57.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.829+0000 7f2378caa700 1 -- 172.21.15.7:0/2220204135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23741f3360 con 0x7f237415cda0 2024-09-13T18:52:57.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.829+0000 7f2370ff9700 1 -- 172.21.15.7:0/2220204135 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c00ce70 con 0x7f237415cda0 2024-09-13T18:52:57.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.829+0000 7f2378caa700 1 -- 172.21.15.7:0/2220204135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23740c6aa0 con 0x7f237415cda0 2024-09-13T18:52:57.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.829+0000 7f2378caa700 1 -- 172.21.15.7:0/2220204135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23740c6cc0 con 0x7f237415cda0 2024-09-13T18:52:57.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.833+0000 7f2370ff9700 1 -- 172.21.15.7:0/2220204135 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f236c014480 con 0x7f237415cda0 2024-09-13T18:52:57.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.833+0000 7f2370ff9700 1 --2- 172.21.15.7:0/2220204135 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2358066730 0x7f2358068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:57.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.833+0000 7f2370ff9700 1 -- 172.21.15.7:0/2220204135 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f236c085380 con 0x7f237415cda0 2024-09-13T18:52:57.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.833+0000 7f23737fe700 1 --2- 172.21.15.7:0/2220204135 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2358066730 0x7f2358068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:57.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.833+0000 7f23737fe700 1 --2- 172.21.15.7:0/2220204135 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2358066730 0x7f2358068bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2364000c00 tx=0x7f2364008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:57.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:57.837+0000 7f2370ff9700 1 -- 172.21.15.7:0/2220204135 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f236c052740 con 0x7f237415cda0 2024-09-13T18:52:58.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.013+0000 7f2378caa700 1 -- 172.21.15.7:0/2220204135 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23740d1120 con 0x7f237415cda0 2024-09-13T18:52:58.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.013+0000 7f2370ff9700 1 -- 172.21.15.7:0/2220204135 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f236c018020 con 0x7f237415cda0 2024-09-13T18:52:58.020 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:52:58.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.017+0000 7f2378caa700 1 -- 172.21.15.7:0/2220204135 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2358066730 msgr2=0x7f2358068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.017+0000 7f2378caa700 1 --2- 172.21.15.7:0/2220204135 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2358066730 0x7f2358068bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2364000c00 tx=0x7f2364008030 comp rx=0 tx=0).stop 2024-09-13T18:52:58.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.017+0000 7f2378caa700 1 -- 172.21.15.7:0/2220204135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237415cda0 msgr2=0x7f23741f2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.017+0000 7f2378caa700 1 --2- 172.21.15.7:0/2220204135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237415cda0 0x7f23741f2750 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f236c00efc0 tx=0x7f236c00c0c0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.017+0000 7f2378caa700 1 -- 172.21.15.7:0/2220204135 shutdown_connections 2024-09-13T18:52:58.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.017+0000 7f2378caa700 1 --2- 172.21.15.7:0/2220204135 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2358066730 0x7f2358068bf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.017+0000 7f2378caa700 1 --2- 172.21.15.7:0/2220204135 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237415cda0 0x7f23741f2750 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.017+0000 7f2378caa700 1 --2- 172.21.15.7:0/2220204135 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f23740ceb70 0x7f23741f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.017+0000 7f2378caa700 1 --2- 172.21.15.7:0/2220204135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f237406a520 0x7f237414f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.017+0000 7f2378caa700 1 -- 172.21.15.7:0/2220204135 >> 172.21.15.7:0/2220204135 conn(0x7f23740d2cb0 msgr2=0x7f23740d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:58.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.017+0000 7f2378caa700 1 -- 172.21.15.7:0/2220204135 shutdown_connections 2024-09-13T18:52:58.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.017+0000 7f2378caa700 1 -- 172.21.15.7:0/2220204135 wait complete. 2024-09-13T18:52:58.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:52:58.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.145+0000 7fa92eebc700 1 Processor -- start 2024-09-13T18:52:58.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.145+0000 7fa92eebc700 1 -- start start 2024-09-13T18:52:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.145+0000 7fa92eebc700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa928153e70 0x7fa9280cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.145+0000 7fa92eebc700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa9280b39d0 0x7fa9280703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.145+0000 7fa92eebc700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa928154250 con 0x7fa92806a310 2024-09-13T18:52:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.145+0000 7fa92eebc700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa92806c710 con 0x7fa9280b39d0 2024-09-13T18:52:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.145+0000 7fa92eebc700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92806c9f0 con 0x7fa928153e70 2024-09-13T18:52:58.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.145+0000 7fa92e6bb700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa9280b39d0 0x7fa9280703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.145+0000 7fa92deba700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa928153e70 0x7fa9280cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.145+0000 7fa92d6b9700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7fa92806a310 0x7fa9280b32c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:46334/0 (socket says 172.21.15.7:46334) 2024-09-13T18:52:58.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.145+0000 7fa92d6b9700 1 -- 172.21.15.7:0/3266548120 learned_addr learned my addr 172.21.15.7:0/3266548120 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:58.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.145+0000 7fa92e6bb700 1 -- 172.21.15.7:0/3266548120 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa928153e70 msgr2=0x7fa9280cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:58.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.145+0000 7fa92e6bb700 1 --2- 172.21.15.7:0/3266548120 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa928153e70 0x7fa9280cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.145+0000 7fa92e6bb700 1 -- 172.21.15.7:0/3266548120 >> v1:172.21.15.171:6789/0 conn(0x7fa92806a310 legacy=0x7fa9280b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.145+0000 7fa92e6bb700 1 -- 172.21.15.7:0/3266548120 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92806bad0 con 0x7fa9280b39d0 2024-09-13T18:52:58.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.145+0000 7fa92e6bb700 1 --2- 172.21.15.7:0/3266548120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9280b39d0 0x7fa9280703e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa920009dc0 tx=0x7fa92000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f50d8466c0d06d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:58.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.149+0000 7fa92ceb8700 1 -- 172.21.15.7:0/3266548120 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92000e070 con 0x7fa9280b39d0 2024-09-13T18:52:58.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.149+0000 7fa92ceb8700 1 -- 172.21.15.7:0/3266548120 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92000bae0 con 0x7fa9280b39d0 2024-09-13T18:52:58.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.149+0000 7fa92ceb8700 1 -- 172.21.15.7:0/3266548120 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92000be90 con 0x7fa9280b39d0 2024-09-13T18:52:58.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.149+0000 7fa92eebc700 1 -- 172.21.15.7:0/3266548120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9280b39d0 msgr2=0x7fa9280703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.149+0000 7fa92eebc700 1 --2- 172.21.15.7:0/3266548120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9280b39d0 0x7fa9280703e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa920009dc0 tx=0x7fa92000b330 comp rx=0 tx=0).stop 2024-09-13T18:52:58.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.149+0000 7fa92eebc700 1 -- 172.21.15.7:0/3266548120 shutdown_connections 2024-09-13T18:52:58.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.149+0000 7fa92eebc700 1 --2- 172.21.15.7:0/3266548120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa9280b39d0 0x7fa9280703e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.149+0000 7fa92eebc700 1 --2- 172.21.15.7:0/3266548120 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa928153e70 0x7fa9280cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.149+0000 7fa92eebc700 1 -- 172.21.15.7:0/3266548120 >> 172.21.15.7:0/3266548120 conn(0x7fa9280b0db0 msgr2=0x7fa9280b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:58.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.149+0000 7fa92eebc700 1 -- 172.21.15.7:0/3266548120 shutdown_connections 2024-09-13T18:52:58.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.149+0000 7fa92eebc700 1 -- 172.21.15.7:0/3266548120 wait complete. 2024-09-13T18:52:58.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92eebc700 1 Processor -- start 2024-09-13T18:52:58.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92eebc700 1 -- start start 2024-09-13T18:52:58.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92eebc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa92806a310 0x7fa9281f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92eebc700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9280b39d0 0x7fa9281f1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92eebc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa928153e70 0x7fa928156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92eebc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9281545d0 con 0x7fa92806a310 2024-09-13T18:52:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92eebc700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa928154820 con 0x7fa928153e70 2024-09-13T18:52:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92eebc700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa928154b00 con 0x7fa9280b39d0 2024-09-13T18:52:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92deba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa92806a310 0x7fa9281f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92deba700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa92806a310 0x7fa9281f3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39214/0 (socket says 172.21.15.7:39214) 2024-09-13T18:52:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92deba700 1 -- 172.21.15.7:0/262265405 learned_addr learned my addr 172.21.15.7:0/262265405 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92d6b9700 1 --2- 172.21.15.7:0/262265405 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9280b39d0 0x7fa9281f1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92e6bb700 1 --2- 172.21.15.7:0/262265405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa928153e70 0x7fa928156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92deba700 1 -- 172.21.15.7:0/262265405 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9280b39d0 msgr2=0x7fa9281f1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92deba700 1 --2- 172.21.15.7:0/262265405 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9280b39d0 0x7fa9281f1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92deba700 1 -- 172.21.15.7:0/262265405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa928153e70 msgr2=0x7fa928156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92deba700 1 --2- 172.21.15.7:0/262265405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa928153e70 0x7fa928156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92deba700 1 -- 172.21.15.7:0/262265405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa920009a70 con 0x7fa92806a310 2024-09-13T18:52:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.153+0000 7fa92d6b9700 1 --2- 172.21.15.7:0/262265405 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9280b39d0 0x7fa9281f1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.157+0000 7fa92deba700 1 --2- 172.21.15.7:0/262265405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa92806a310 0x7fa9281f3510 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa92400e840 tx=0x7fa92400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.157+0000 7fa91e7fc700 1 -- 172.21.15.7:0/262265405 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9240095a0 con 0x7fa92806a310 2024-09-13T18:52:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.157+0000 7fa91e7fc700 1 -- 172.21.15.7:0/262265405 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa924013680 con 0x7fa92806a310 2024-09-13T18:52:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.157+0000 7fa91e7fc700 1 -- 172.21.15.7:0/262265405 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9240139f0 con 0x7fa92806a310 2024-09-13T18:52:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.157+0000 7fa92eebc700 1 -- 172.21.15.7:0/262265405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa928154de0 con 0x7fa92806a310 2024-09-13T18:52:58.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.157+0000 7fa92eebc700 1 -- 172.21.15.7:0/262265405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9281570c0 con 0x7fa92806a310 2024-09-13T18:52:58.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.157+0000 7fa92eebc700 1 -- 172.21.15.7:0/262265405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa928072400 con 0x7fa92806a310 2024-09-13T18:52:58.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.157+0000 7fa91e7fc700 1 -- 172.21.15.7:0/262265405 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa924013b50 con 0x7fa92806a310 2024-09-13T18:52:58.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.157+0000 7fa91e7fc700 1 --2- 172.21.15.7:0/262265405 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa90c066730 0x7fa90c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.157+0000 7fa91e7fc700 1 -- 172.21.15.7:0/262265405 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa924084110 con 0x7fa92806a310 2024-09-13T18:52:58.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.157+0000 7fa92d6b9700 1 --2- 172.21.15.7:0/262265405 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa90c066730 0x7fa90c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.161+0000 7fa92d6b9700 1 --2- 172.21.15.7:0/262265405 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa90c066730 0x7fa90c068bf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa918007fd0 tx=0x7fa918009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:58.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.161+0000 7fa91e7fc700 1 -- 172.21.15.7:0/262265405 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa924051480 con 0x7fa92806a310 2024-09-13T18:52:58.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.345+0000 7fa92eebc700 1 -- 172.21.15.7:0/262265405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa9280b32c0 con 0x7fa92806a310 2024-09-13T18:52:58.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.345+0000 7fa91e7fc700 1 -- 172.21.15.7:0/262265405 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa924054aa0 con 0x7fa92806a310 2024-09-13T18:52:58.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.349+0000 7fa92eebc700 1 -- 172.21.15.7:0/262265405 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa90c066730 msgr2=0x7fa90c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.349+0000 7fa92eebc700 1 --2- 172.21.15.7:0/262265405 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa90c066730 0x7fa90c068bf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa918007fd0 tx=0x7fa918009040 comp rx=0 tx=0).stop 2024-09-13T18:52:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.349+0000 7fa92eebc700 1 -- 172.21.15.7:0/262265405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa92806a310 msgr2=0x7fa9281f3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.349+0000 7fa92eebc700 1 --2- 172.21.15.7:0/262265405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa92806a310 0x7fa9281f3510 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa92400e840 tx=0x7fa92400eb50 comp rx=0 tx=0).stop 2024-09-13T18:52:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.349+0000 7fa92eebc700 1 -- 172.21.15.7:0/262265405 shutdown_connections 2024-09-13T18:52:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.349+0000 7fa92eebc700 1 --2- 172.21.15.7:0/262265405 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa90c066730 0x7fa90c068bf0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.349+0000 7fa92eebc700 1 --2- 172.21.15.7:0/262265405 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa928153e70 0x7fa928156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.349+0000 7fa92eebc700 1 --2- 172.21.15.7:0/262265405 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa9280b39d0 0x7fa9281f1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.349+0000 7fa92eebc700 1 --2- 172.21.15.7:0/262265405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa92806a310 0x7fa9281f3510 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.349+0000 7fa92eebc700 1 -- 172.21.15.7:0/262265405 >> 172.21.15.7:0/262265405 conn(0x7fa9280b0db0 msgr2=0x7fa9280cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.349+0000 7fa92eebc700 1 -- 172.21.15.7:0/262265405 shutdown_connections 2024-09-13T18:52:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.349+0000 7fa92eebc700 1 -- 172.21.15.7:0/262265405 wait complete. 2024-09-13T18:52:58.357 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-13T18:52:58.368 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:52:58.368 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:52:58.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:52:58.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcffff700 1 Processor -- start 2024-09-13T18:52:58.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcffff700 1 -- start start 2024-09-13T18:52:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcffff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f5fd0165300 0x7f5fd00ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5fd006cb40 0x7f5fd00d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcffff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd00d1260 con 0x7f5fd0160ff0 2024-09-13T18:52:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcffff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd00ced70 con 0x7f5fd006cb40 2024-09-13T18:52:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcffff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd00d0c50 con 0x7f5fd0165300 2024-09-13T18:52:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcf7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5fd006cb40 0x7f5fd00d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcf7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5fd006cb40 0x7f5fd00d16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39224/0 (socket says 172.21.15.7:39224) 2024-09-13T18:52:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcf7fe700 1 -- 172.21.15.7:0/4052856965 learned_addr learned my addr 172.21.15.7:0/4052856965 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fce7fc700 1 --2- 172.21.15.7:0/4052856965 >> v2:172.21.15.171:3300/0 conn(0x7f5fd0165300 0x7f5fd00ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcf7fe700 1 -- 172.21.15.7:0/4052856965 >> v2:172.21.15.171:3300/0 conn(0x7f5fd0165300 msgr2=0x7f5fd00ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:58.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcf7fe700 1 --2- 172.21.15.7:0/4052856965 >> v2:172.21.15.171:3300/0 conn(0x7f5fd0165300 0x7f5fd00ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcf7fe700 1 -- 172.21.15.7:0/4052856965 >> v1:172.21.15.171:6789/0 conn(0x7f5fd0160ff0 legacy=0x7f5fd0164bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcf7fe700 1 -- 172.21.15.7:0/4052856965 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd00cf0b0 con 0x7f5fd006cb40 2024-09-13T18:52:58.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcf7fe700 1 --2- 172.21.15.7:0/4052856965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd006cb40 0x7f5fd00d16f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5fd006d000 tx=0x7f5fc0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b26d8b1ba216c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:58.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcdffb700 1 -- 172.21.15.7:0/4052856965 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0011830 con 0x7f5fd006cb40 2024-09-13T18:52:58.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcdffb700 1 -- 172.21.15.7:0/4052856965 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc0011990 con 0x7f5fd006cb40 2024-09-13T18:52:58.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcdffb700 1 -- 172.21.15.7:0/4052856965 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0011d20 con 0x7f5fd006cb40 2024-09-13T18:52:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcffff700 1 -- 172.21.15.7:0/4052856965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd006cb40 msgr2=0x7f5fd00d16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.485+0000 7f5fcffff700 1 --2- 172.21.15.7:0/4052856965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd006cb40 0x7f5fd00d16f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5fd006d000 tx=0x7f5fc0009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fcffff700 1 -- 172.21.15.7:0/4052856965 shutdown_connections 2024-09-13T18:52:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fcffff700 1 --2- 172.21.15.7:0/4052856965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd006cb40 0x7f5fd00d16f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fcffff700 1 --2- 172.21.15.7:0/4052856965 >> v2:172.21.15.171:3300/0 conn(0x7f5fd0165300 0x7f5fd00ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fcffff700 1 -- 172.21.15.7:0/4052856965 >> 172.21.15.7:0/4052856965 conn(0x7f5fd00d2cb0 msgr2=0x7f5fd00d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fcffff700 1 -- 172.21.15.7:0/4052856965 shutdown_connections 2024-09-13T18:52:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fcffff700 1 -- 172.21.15.7:0/4052856965 wait complete. 2024-09-13T18:52:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fcffff700 1 Processor -- start 2024-09-13T18:52:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fcffff700 1 -- start start 2024-09-13T18:52:58.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fcffff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5fd006cb40 0x7f5fd014e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fcffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd0160ff0 0x7f5fd015c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fcffff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fd0165300 0x7f5fd0148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fcffff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd014f160 con 0x7f5fd0160ff0 2024-09-13T18:52:58.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fcffff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd00c7f30 con 0x7f5fd0165300 2024-09-13T18:52:58.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fcffff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd015cf50 con 0x7f5fd006cb40 2024-09-13T18:52:58.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fce7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd0160ff0 0x7f5fd015c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fce7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd0160ff0 0x7f5fd015c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39238/0 (socket says 172.21.15.7:39238) 2024-09-13T18:52:58.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fce7fc700 1 -- 172.21.15.7:0/3778549779 learned_addr learned my addr 172.21.15.7:0/3778549779 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:58.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fceffd700 1 --2- 172.21.15.7:0/3778549779 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5fd006cb40 0x7f5fd014e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fcf7fe700 1 --2- 172.21.15.7:0/3778549779 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fd0165300 0x7f5fd0148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fce7fc700 1 -- 172.21.15.7:0/3778549779 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5fd006cb40 msgr2=0x7f5fd014e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fce7fc700 1 --2- 172.21.15.7:0/3778549779 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5fd006cb40 0x7f5fd014e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fce7fc700 1 -- 172.21.15.7:0/3778549779 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fd0165300 msgr2=0x7f5fd0148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fce7fc700 1 --2- 172.21.15.7:0/3778549779 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fd0165300 0x7f5fd0148d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fce7fc700 1 -- 172.21.15.7:0/3778549779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc0009570 con 0x7f5fd0160ff0 2024-09-13T18:52:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.489+0000 7f5fce7fc700 1 --2- 172.21.15.7:0/3778549779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd0160ff0 0x7f5fd015c8c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009d10 tx=0x7f5fc800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.493+0000 7f5fc77fe700 1 -- 172.21.15.7:0/3778549779 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8009b80 con 0x7f5fd0160ff0 2024-09-13T18:52:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.493+0000 7f5fc77fe700 1 -- 172.21.15.7:0/3778549779 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc800f040 con 0x7f5fd0160ff0 2024-09-13T18:52:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.493+0000 7f5fcffff700 1 -- 172.21.15.7:0/3778549779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd00c8160 con 0x7f5fd0160ff0 2024-09-13T18:52:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.493+0000 7f5fc77fe700 1 -- 172.21.15.7:0/3778549779 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8014480 con 0x7f5fd0160ff0 2024-09-13T18:52:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.493+0000 7f5fcffff700 1 -- 172.21.15.7:0/3778549779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd00c8720 con 0x7f5fd0160ff0 2024-09-13T18:52:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.493+0000 7f5fcffff700 1 -- 172.21.15.7:0/3778549779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd00ce830 con 0x7f5fd0160ff0 2024-09-13T18:52:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.493+0000 7f5fc77fe700 1 -- 172.21.15.7:0/3778549779 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5fc80145e0 con 0x7f5fd0160ff0 2024-09-13T18:52:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.493+0000 7f5fc77fe700 1 --2- 172.21.15.7:0/3778549779 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5fb0066730 0x7f5fb0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.493+0000 7f5fc77fe700 1 -- 172.21.15.7:0/3778549779 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5fc8084ce0 con 0x7f5fd0160ff0 2024-09-13T18:52:58.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.493+0000 7f5fceffd700 1 --2- 172.21.15.7:0/3778549779 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5fb0066730 0x7f5fb0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.497+0000 7f5fceffd700 1 --2- 172.21.15.7:0/3778549779 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5fb0066730 0x7f5fb0068bf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8000f60 tx=0x7f5fb8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:58.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.501+0000 7f5fc77fe700 1 -- 172.21.15.7:0/3778549779 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5fc8052060 con 0x7f5fd0160ff0 2024-09-13T18:52:58.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.677+0000 7f5fcffff700 1 -- 172.21.15.7:0/3778549779 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fd00b7c00 con 0x7f5fd0160ff0 2024-09-13T18:52:58.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.681+0000 7f5fc77fe700 1 -- 172.21.15.7:0/3778549779 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f5fc8055680 con 0x7f5fd0160ff0 2024-09-13T18:52:58.686 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:52:58.686 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:52:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.681+0000 7f5fcffff700 1 -- 172.21.15.7:0/3778549779 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5fb0066730 msgr2=0x7f5fb0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.681+0000 7f5fcffff700 1 --2- 172.21.15.7:0/3778549779 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5fb0066730 0x7f5fb0068bf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8000f60 tx=0x7f5fb8009040 comp rx=0 tx=0).stop 2024-09-13T18:52:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.681+0000 7f5fcffff700 1 -- 172.21.15.7:0/3778549779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd0160ff0 msgr2=0x7f5fd015c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.681+0000 7f5fcffff700 1 --2- 172.21.15.7:0/3778549779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd0160ff0 0x7f5fd015c8c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009d10 tx=0x7f5fc800c3a0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.681+0000 7f5fcffff700 1 -- 172.21.15.7:0/3778549779 shutdown_connections 2024-09-13T18:52:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.681+0000 7f5fcffff700 1 --2- 172.21.15.7:0/3778549779 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5fb0066730 0x7f5fb0068bf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.681+0000 7f5fcffff700 1 --2- 172.21.15.7:0/3778549779 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fd0165300 0x7f5fd0148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.681+0000 7f5fcffff700 1 --2- 172.21.15.7:0/3778549779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fd0160ff0 0x7f5fd015c8c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.681+0000 7f5fcffff700 1 --2- 172.21.15.7:0/3778549779 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5fd006cb40 0x7f5fd014e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.681+0000 7f5fcffff700 1 -- 172.21.15.7:0/3778549779 >> 172.21.15.7:0/3778549779 conn(0x7f5fd00d2cb0 msgr2=0x7f5fd006a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.681+0000 7f5fcffff700 1 -- 172.21.15.7:0/3778549779 shutdown_connections 2024-09-13T18:52:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.681+0000 7f5fcffff700 1 -- 172.21.15.7:0/3778549779 wait complete. 2024-09-13T18:52:58.702 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:52:15.554683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:52:16.896879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:52:58.702 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:52:58.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd45401a700 1 Processor -- start 2024-09-13T18:52:58.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd45401a700 1 -- start start 2024-09-13T18:52:58.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd45401a700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd44c15cb80 0x7fd44c0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd45401a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44c0cd440 con 0x7fd44c160710 2024-09-13T18:52:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd45401a700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44c0cd6f0 con 0x7fd44c0ae8e0 2024-09-13T18:52:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd45401a700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd44c0cd9d0 con 0x7fd44c15cb80 2024-09-13T18:52:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd4525b7700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd44c160710 0x7fd44c0ce790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38846/0 (socket says 172.21.15.7:38846) 2024-09-13T18:52:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd4525b7700 1 -- 172.21.15.7:0/4040804248 learned_addr learned my addr 172.21.15.7:0/4040804248 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:58.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd451db6700 1 --2- 172.21.15.7:0/4040804248 >> v2:172.21.15.171:3300/0 conn(0x7fd44c15cb80 0x7fd44c0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd450db4700 1 -- 172.21.15.7:0/4040804248 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361708148 0 0) 0x7fd44c0cd440 con 0x7fd44c160710 2024-09-13T18:52:58.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd450db4700 1 -- 172.21.15.7:0/4040804248 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4380035d0 con 0x7fd44c160710 2024-09-13T18:52:58.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd450db4700 1 -- 172.21.15.7:0/4040804248 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1512313440 0 0) 0x7fd44c0cd6f0 con 0x7fd44c0ae8e0 2024-09-13T18:52:58.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd450db4700 1 -- 172.21.15.7:0/4040804248 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd44c0cd440 con 0x7fd44c0ae8e0 2024-09-13T18:52:58.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd450db4700 1 -- 172.21.15.7:0/4040804248 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3522339329 0 0) 0x7fd4380035d0 con 0x7fd44c160710 2024-09-13T18:52:58.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd450db4700 1 -- 172.21.15.7:0/4040804248 >> v2:172.21.15.171:3300/0 conn(0x7fd44c15cb80 msgr2=0x7fd44c0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:58.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd450db4700 1 --2- 172.21.15.7:0/4040804248 >> v2:172.21.15.171:3300/0 conn(0x7fd44c15cb80 0x7fd44c0b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd450db4700 1 -- 172.21.15.7:0/4040804248 >> v1:172.21.15.171:6789/0 conn(0x7fd44c0ae8e0 legacy=0x7fd44c0ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd450db4700 1 -- 172.21.15.7:0/4040804248 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c0cdcb0 con 0x7fd44c160710 2024-09-13T18:52:58.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd450db4700 1 -- 172.21.15.7:0/4040804248 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7fd444001670 con 0x7fd44c160710 2024-09-13T18:52:58.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd450db4700 1 -- 172.21.15.7:0/4040804248 >> v1:172.21.15.7:6789/0 conn(0x7fd44c160710 legacy=0x7fd44c0ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd450db4700 1 --2- 172.21.15.7:0/4040804248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd438003c00 0x7fd438004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd450db4700 1 --2- 172.21.15.7:0/4040804248 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd438004870 0x7fd438004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd450db4700 1 --2- 172.21.15.7:0/4040804248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd438005230 0x7fd43800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd450db4700 1 -- 172.21.15.7:0/4040804248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c0cd9d0 con 0x7fd438003c00 2024-09-13T18:52:58.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd450db4700 1 -- 172.21.15.7:0/4040804248 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43800c100 con 0x7fd438005230 2024-09-13T18:52:58.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd450db4700 1 -- 172.21.15.7:0/4040804248 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43800c2c0 con 0x7fd438004870 2024-09-13T18:52:58.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd451db6700 1 --2- 172.21.15.7:0/4040804248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd438003c00 0x7fd438004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd4515b5700 1 --2- 172.21.15.7:0/4040804248 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd438004870 0x7fd438004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.821+0000 7fd4525b7700 1 --2- 172.21.15.7:0/4040804248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd438005230 0x7fd43800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd451db6700 1 -- 172.21.15.7:0/4040804248 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd438004870 msgr2=0x7fd438004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:58.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd451db6700 1 --2- 172.21.15.7:0/4040804248 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd438004870 0x7fd438004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd451db6700 1 -- 172.21.15.7:0/4040804248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd438005230 msgr2=0x7fd43800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:58.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd451db6700 1 --2- 172.21.15.7:0/4040804248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd438005230 0x7fd43800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd4525b7700 1 -- 172.21.15.7:0/4040804248 reap_dead start 2024-09-13T18:52:58.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd451db6700 1 -- 172.21.15.7:0/4040804248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4380035d0 con 0x7fd438003c00 2024-09-13T18:52:58.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd451db6700 1 --2- 172.21.15.7:0/4040804248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd438003c00 0x7fd438004000 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd448009fc0 tx=0x7fd4380058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1deb41ae08d697 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:58.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd450db4700 1 -- 172.21.15.7:0/4040804248 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44800ec30 con 0x7fd438003c00 2024-09-13T18:52:58.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd450db4700 1 -- 172.21.15.7:0/4040804248 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44800f040 con 0x7fd438003c00 2024-09-13T18:52:58.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd450db4700 1 -- 172.21.15.7:0/4040804248 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448014480 con 0x7fd438003c00 2024-09-13T18:52:58.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd45401a700 1 -- 172.21.15.7:0/4040804248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd438003c00 msgr2=0x7fd438004000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd45401a700 1 --2- 172.21.15.7:0/4040804248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd438003c00 0x7fd438004000 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd448009fc0 tx=0x7fd4380058a0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd45401a700 1 -- 172.21.15.7:0/4040804248 shutdown_connections 2024-09-13T18:52:58.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd45401a700 1 --2- 172.21.15.7:0/4040804248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd438003c00 0x7fd438004000 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd45401a700 1 -- 172.21.15.7:0/4040804248 >> 172.21.15.7:0/4040804248 conn(0x7fd44c070d00 msgr2=0x7fd44c071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:58.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd45401a700 1 -- 172.21.15.7:0/4040804248 shutdown_connections 2024-09-13T18:52:58.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd45401a700 1 -- 172.21.15.7:0/4040804248 wait complete. 2024-09-13T18:52:58.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd45401a700 1 Processor -- start 2024-09-13T18:52:58.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.825+0000 7fd45401a700 1 -- start start 2024-09-13T18:52:58.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd45401a700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd438003c00 0x7fd44c14ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd45401a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd44c15d930 0x7fd44c14c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd45401a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44c15ddb0 0x7fd44c14ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd45401a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c14d3b0 con 0x7fd44c15ddb0 2024-09-13T18:52:58.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd45401a700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c154690 con 0x7fd44c15d930 2024-09-13T18:52:58.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd45401a700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c152930 con 0x7fd438003c00 2024-09-13T18:52:58.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd4525b7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44c15ddb0 0x7fd44c14ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd4525b7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44c15ddb0 0x7fd44c14ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39256/0 (socket says 172.21.15.7:39256) 2024-09-13T18:52:58.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd4525b7700 1 -- 172.21.15.7:0/3061109966 learned_addr learned my addr 172.21.15.7:0/3061109966 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:58.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd451db6700 1 --2- 172.21.15.7:0/3061109966 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd438003c00 0x7fd44c14ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd4515b5700 1 --2- 172.21.15.7:0/3061109966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd44c15d930 0x7fd44c14c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd4525b7700 1 -- 172.21.15.7:0/3061109966 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd438003c00 msgr2=0x7fd44c14ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd4525b7700 1 --2- 172.21.15.7:0/3061109966 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd438003c00 0x7fd44c14ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd4525b7700 1 -- 172.21.15.7:0/3061109966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd44c15d930 msgr2=0x7fd44c14c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:58.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd4525b7700 1 --2- 172.21.15.7:0/3061109966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd44c15d930 0x7fd44c14c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:58.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd4525b7700 1 -- 172.21.15.7:0/3061109966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c152c10 con 0x7fd44c15ddb0 2024-09-13T18:52:58.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd451db6700 1 --2- 172.21.15.7:0/3061109966 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd438003c00 0x7fd44c14ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:58.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd4525b7700 1 --2- 172.21.15.7:0/3061109966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44c15ddb0 0x7fd44c14ce70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0af260 tx=0x7fd444007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:58.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd43e7fc700 1 -- 172.21.15.7:0/3061109966 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd444012560 con 0x7fd44c15ddb0 2024-09-13T18:52:58.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd43e7fc700 1 -- 172.21.15.7:0/3061109966 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd444012700 con 0x7fd44c15ddb0 2024-09-13T18:52:58.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd45401a700 1 -- 172.21.15.7:0/3061109966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd44c152ef0 con 0x7fd44c15ddb0 2024-09-13T18:52:58.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd43e7fc700 1 -- 172.21.15.7:0/3061109966 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd444012ab0 con 0x7fd44c15ddb0 2024-09-13T18:52:58.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd45401a700 1 -- 172.21.15.7:0/3061109966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44c150710 con 0x7fd44c15ddb0 2024-09-13T18:52:58.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.829+0000 7fd45401a700 1 -- 172.21.15.7:0/3061109966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44c04e7c0 con 0x7fd44c15ddb0 2024-09-13T18:52:58.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.833+0000 7fd43e7fc700 1 -- 172.21.15.7:0/3061109966 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd444012c10 con 0x7fd44c15ddb0 2024-09-13T18:52:58.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.833+0000 7fd43e7fc700 1 --2- 172.21.15.7:0/3061109966 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd430066730 0x7fd430068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:58.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.833+0000 7fd43e7fc700 1 -- 172.21.15.7:0/3061109966 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd444083fd0 con 0x7fd44c15ddb0 2024-09-13T18:52:58.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.833+0000 7fd451db6700 1 --2- 172.21.15.7:0/3061109966 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd430066730 0x7fd430068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:58.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.833+0000 7fd451db6700 1 --2- 172.21.15.7:0/3061109966 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd430066730 0x7fd430068bf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd448000f80 tx=0x7fd448013000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:58.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:58.837+0000 7fd43e7fc700 1 -- 172.21.15.7:0/3061109966 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd4440512b0 con 0x7fd44c15ddb0 2024-09-13T18:52:59.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.009+0000 7fd45401a700 1 -- 172.21.15.7:0/3061109966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd44c150430 con 0x7fd44c15ddb0 2024-09-13T18:52:59.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.009+0000 7fd43e7fc700 1 -- 172.21.15.7:0/3061109966 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd44c150430 con 0x7fd44c15ddb0 2024-09-13T18:52:59.016 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:52:59.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.013+0000 7fd45401a700 1 -- 172.21.15.7:0/3061109966 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd430066730 msgr2=0x7fd430068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.013+0000 7fd45401a700 1 --2- 172.21.15.7:0/3061109966 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd430066730 0x7fd430068bf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd448000f80 tx=0x7fd448013000 comp rx=0 tx=0).stop 2024-09-13T18:52:59.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.013+0000 7fd45401a700 1 -- 172.21.15.7:0/3061109966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44c15ddb0 msgr2=0x7fd44c14ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.013+0000 7fd45401a700 1 --2- 172.21.15.7:0/3061109966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44c15ddb0 0x7fd44c14ce70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd44c0af260 tx=0x7fd444007570 comp rx=0 tx=0).stop 2024-09-13T18:52:59.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.013+0000 7fd45401a700 1 -- 172.21.15.7:0/3061109966 shutdown_connections 2024-09-13T18:52:59.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.013+0000 7fd45401a700 1 --2- 172.21.15.7:0/3061109966 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd430066730 0x7fd430068bf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.013+0000 7fd45401a700 1 --2- 172.21.15.7:0/3061109966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44c15ddb0 0x7fd44c14ce70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.013+0000 7fd45401a700 1 --2- 172.21.15.7:0/3061109966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd44c15d930 0x7fd44c14c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.013+0000 7fd45401a700 1 --2- 172.21.15.7:0/3061109966 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd438003c00 0x7fd44c14ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.013+0000 7fd45401a700 1 -- 172.21.15.7:0/3061109966 >> 172.21.15.7:0/3061109966 conn(0x7fd44c070d00 msgr2=0x7fd44c0b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:59.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.013+0000 7fd45401a700 1 -- 172.21.15.7:0/3061109966 shutdown_connections 2024-09-13T18:52:59.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.013+0000 7fd45401a700 1 -- 172.21.15.7:0/3061109966 wait complete. 2024-09-13T18:52:59.031 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:52:59.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.145+0000 7f2b37fff700 1 Processor -- start 2024-09-13T18:52:59.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.145+0000 7f2b37fff700 1 -- start start 2024-09-13T18:52:59.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b37fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2b3815cda0 0x7f2b3806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b37fff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f2b380ceb70 0x7f2b380b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b37fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b380b8430 con 0x7f2b3806a520 2024-09-13T18:52:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b37fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b3806be40 con 0x7f2b3815cda0 2024-09-13T18:52:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b37fff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b380cf350 con 0x7f2b380ceb70 2024-09-13T18:52:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b36ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2b3815cda0 0x7f2b3806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b377fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2b3806a520 0x7f2b3806a930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38858/0 (socket says 172.21.15.7:38858) 2024-09-13T18:52:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b377fe700 1 -- 172.21.15.7:0/427838037 learned_addr learned my addr 172.21.15.7:0/427838037 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b2ffff700 1 --2- 172.21.15.7:0/427838037 >> v2:172.21.15.171:3300/0 conn(0x7f2b380ceb70 0x7f2b380b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b367fc700 1 -- 172.21.15.7:0/427838037 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761698540 0 0) 0x7f2b380b8430 con 0x7f2b3806a520 2024-09-13T18:52:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b367fc700 1 -- 172.21.15.7:0/427838037 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b240035d0 con 0x7f2b3806a520 2024-09-13T18:52:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b36ffd700 1 -- 172.21.15.7:0/427838037 >> v2:172.21.15.171:3300/0 conn(0x7f2b380ceb70 msgr2=0x7f2b380b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:59.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b36ffd700 1 --2- 172.21.15.7:0/427838037 >> v2:172.21.15.171:3300/0 conn(0x7f2b380ceb70 0x7f2b380b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b36ffd700 1 -- 172.21.15.7:0/427838037 >> v1:172.21.15.7:6789/0 conn(0x7f2b3806a520 legacy=0x7f2b3806a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b36ffd700 1 -- 172.21.15.7:0/427838037 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b380d0880 con 0x7f2b3815cda0 2024-09-13T18:52:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b36ffd700 1 --2- 172.21.15.7:0/427838037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b3815cda0 0x7f2b3806b900 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b3806cca0 tx=0x7f2b28009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04914b4fabaea8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b367fc700 1 -- 172.21.15.7:0/427838037 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b28011830 con 0x7f2b3815cda0 2024-09-13T18:52:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b367fc700 1 -- 172.21.15.7:0/427838037 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b28011990 con 0x7f2b3815cda0 2024-09-13T18:52:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b367fc700 1 -- 172.21.15.7:0/427838037 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b28011d20 con 0x7f2b3815cda0 2024-09-13T18:52:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b37fff700 1 -- 172.21.15.7:0/427838037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b3815cda0 msgr2=0x7f2b3806b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b37fff700 1 --2- 172.21.15.7:0/427838037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b3815cda0 0x7f2b3806b900 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b3806cca0 tx=0x7f2b28009b30 comp rx=0 tx=0).stop 2024-09-13T18:52:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b37fff700 1 -- 172.21.15.7:0/427838037 shutdown_connections 2024-09-13T18:52:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b37fff700 1 --2- 172.21.15.7:0/427838037 >> v2:172.21.15.171:3300/0 conn(0x7f2b380ceb70 0x7f2b380b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b37fff700 1 --2- 172.21.15.7:0/427838037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b3815cda0 0x7f2b3806b900 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b37fff700 1 -- 172.21.15.7:0/427838037 >> 172.21.15.7:0/427838037 conn(0x7f2b380d2cb0 msgr2=0x7f2b380d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b37fff700 1 -- 172.21.15.7:0/427838037 shutdown_connections 2024-09-13T18:52:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.149+0000 7f2b37fff700 1 -- 172.21.15.7:0/427838037 wait complete. 2024-09-13T18:52:59.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.153+0000 7f2b37fff700 1 Processor -- start 2024-09-13T18:52:59.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.153+0000 7f2b37fff700 1 -- start start 2024-09-13T18:52:59.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.153+0000 7f2b37fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b3806a520 0x7f2b381f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:59.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.153+0000 7f2b37fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2b380ceb70 0x7f2b380cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:59.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.153+0000 7f2b37fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2b3815cda0 0x7f2b380c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:59.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.153+0000 7f2b37fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b380b2110 con 0x7f2b3806a520 2024-09-13T18:52:59.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.153+0000 7f2b37fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b380cccb0 con 0x7f2b3815cda0 2024-09-13T18:52:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.153+0000 7f2b37fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b380ca930 con 0x7f2b380ceb70 2024-09-13T18:52:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.153+0000 7f2b36ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b3806a520 0x7f2b381f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.153+0000 7f2b36ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b3806a520 0x7f2b381f5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39280/0 (socket says 172.21.15.7:39280) 2024-09-13T18:52:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.153+0000 7f2b36ffd700 1 -- 172.21.15.7:0/560586906 learned_addr learned my addr 172.21.15.7:0/560586906 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.153+0000 7f2b2ffff700 1 --2- 172.21.15.7:0/560586906 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2b380ceb70 0x7f2b380cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.153+0000 7f2b377fe700 1 --2- 172.21.15.7:0/560586906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2b3815cda0 0x7f2b380c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.157+0000 7f2b36ffd700 1 -- 172.21.15.7:0/560586906 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2b380ceb70 msgr2=0x7f2b380cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.157+0000 7f2b36ffd700 1 --2- 172.21.15.7:0/560586906 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2b380ceb70 0x7f2b380cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.157+0000 7f2b36ffd700 1 -- 172.21.15.7:0/560586906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2b3815cda0 msgr2=0x7f2b380c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.157+0000 7f2b36ffd700 1 --2- 172.21.15.7:0/560586906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2b3815cda0 0x7f2b380c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.157+0000 7f2b36ffd700 1 -- 172.21.15.7:0/560586906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b280094b0 con 0x7f2b3806a520 2024-09-13T18:52:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.157+0000 7f2b36ffd700 1 --2- 172.21.15.7:0/560586906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b3806a520 0x7f2b381f5de0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009b70 tx=0x7f2b28004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.157+0000 7f2b2f7fe700 1 -- 172.21.15.7:0/560586906 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b28004480 con 0x7f2b3806a520 2024-09-13T18:52:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.157+0000 7f2b2f7fe700 1 -- 172.21.15.7:0/560586906 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2801f070 con 0x7f2b3806a520 2024-09-13T18:52:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.157+0000 7f2b37fff700 1 -- 172.21.15.7:0/560586906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b380cac10 con 0x7f2b3806a520 2024-09-13T18:52:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.157+0000 7f2b2f7fe700 1 -- 172.21.15.7:0/560586906 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b280038c0 con 0x7f2b3806a520 2024-09-13T18:52:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.157+0000 7f2b37fff700 1 -- 172.21.15.7:0/560586906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b380cb1d0 con 0x7f2b3806a520 2024-09-13T18:52:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.157+0000 7f2b37fff700 1 -- 172.21.15.7:0/560586906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b3806b900 con 0x7f2b3806a520 2024-09-13T18:52:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.157+0000 7f2b2f7fe700 1 -- 172.21.15.7:0/560586906 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2b28021030 con 0x7f2b3806a520 2024-09-13T18:52:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.161+0000 7f2b2f7fe700 1 --2- 172.21.15.7:0/560586906 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2b18066730 0x7f2b18068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.161+0000 7f2b2f7fe700 1 -- 172.21.15.7:0/560586906 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b28085630 con 0x7f2b3806a520 2024-09-13T18:52:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.161+0000 7f2b2ffff700 1 --2- 172.21.15.7:0/560586906 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2b18066730 0x7f2b18068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:59.167 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:52:59.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.161+0000 7f2b2ffff700 1 --2- 172.21.15.7:0/560586906 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2b18066730 0x7f2b18068bf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2b380b7a60 tx=0x7f2b20008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:59.169 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:52:59.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.173+0000 7f2b2f7fe700 1 -- 172.21.15.7:0/560586906 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2b28052ab0 con 0x7f2b3806a520 2024-09-13T18:52:59.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.345+0000 7f2b37fff700 1 -- 172.21.15.7:0/560586906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2b38066110 con 0x7f2b3806a520 2024-09-13T18:52:59.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.345+0000 7f2b2f7fe700 1 -- 172.21.15.7:0/560586906 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2b280560d0 con 0x7f2b3806a520 2024-09-13T18:52:59.351 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:52:59.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.345+0000 7f2b37fff700 1 -- 172.21.15.7:0/560586906 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2b18066730 msgr2=0x7f2b18068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.345+0000 7f2b37fff700 1 --2- 172.21.15.7:0/560586906 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2b18066730 0x7f2b18068bf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2b380b7a60 tx=0x7f2b20008040 comp rx=0 tx=0).stop 2024-09-13T18:52:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.345+0000 7f2b37fff700 1 -- 172.21.15.7:0/560586906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b3806a520 msgr2=0x7f2b381f5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.345+0000 7f2b37fff700 1 --2- 172.21.15.7:0/560586906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b3806a520 0x7f2b381f5de0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009b70 tx=0x7f2b28004770 comp rx=0 tx=0).stop 2024-09-13T18:52:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.349+0000 7f2b37fff700 1 -- 172.21.15.7:0/560586906 shutdown_connections 2024-09-13T18:52:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.349+0000 7f2b37fff700 1 --2- 172.21.15.7:0/560586906 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2b18066730 0x7f2b18068bf0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.349+0000 7f2b37fff700 1 --2- 172.21.15.7:0/560586906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2b3815cda0 0x7f2b380c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.349+0000 7f2b37fff700 1 --2- 172.21.15.7:0/560586906 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2b380ceb70 0x7f2b380cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.349+0000 7f2b37fff700 1 --2- 172.21.15.7:0/560586906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b3806a520 0x7f2b381f5de0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.349+0000 7f2b37fff700 1 -- 172.21.15.7:0/560586906 >> 172.21.15.7:0/560586906 conn(0x7f2b380d2cb0 msgr2=0x7f2b3815b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.349+0000 7f2b37fff700 1 -- 172.21.15.7:0/560586906 shutdown_connections 2024-09-13T18:52:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.349+0000 7f2b37fff700 1 -- 172.21.15.7:0/560586906 wait complete. 2024-09-13T18:52:59.371 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:52:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.481+0000 7f431e65a700 1 Processor -- start 2024-09-13T18:52:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.481+0000 7f431e65a700 1 -- start start 2024-09-13T18:52:59.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f431e65a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f43180ce270 0x7f431815caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:59.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f431e65a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f431815d480 con 0x7f431806d6f0 2024-09-13T18:52:59.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f431e65a700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f431806dad0 con 0x7f431815d030 2024-09-13T18:52:59.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f431e65a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43180ae3c0 con 0x7f43180ce270 2024-09-13T18:52:59.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f431d658700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f431806d6f0 0x7f43180adbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38864/0 (socket says 172.21.15.7:38864) 2024-09-13T18:52:59.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f431ce57700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f43180ce270 0x7f431815caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:59.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f431d658700 1 -- 172.21.15.7:0/3307124981 learned_addr learned my addr 172.21.15.7:0/3307124981 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:52:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f431ce57700 1 -- 172.21.15.7:0/3307124981 >> v1:172.21.15.171:6789/0 conn(0x7f431815d030 legacy=0x7f431806baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f431ce57700 1 -- 172.21.15.7:0/3307124981 >> v1:172.21.15.7:6789/0 conn(0x7f431806d6f0 legacy=0x7f43180adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f430ffff700 1 -- 172.21.15.7:0/3307124981 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2868836786 0 0) 0x7f4308001110 con 0x7f431806d6f0 2024-09-13T18:52:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f431ce57700 1 -- 172.21.15.7:0/3307124981 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43180705b0 con 0x7f43180ce270 2024-09-13T18:52:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f431ce57700 1 --2- 172.21.15.7:0/3307124981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43180ce270 0x7f431815caf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4314009880 tx=0x7f4314009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ef6fc21b68ec15 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f430ffff700 1 -- 172.21.15.7:0/3307124981 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314011830 con 0x7f43180ce270 2024-09-13T18:52:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f430ffff700 1 -- 172.21.15.7:0/3307124981 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4314011990 con 0x7f43180ce270 2024-09-13T18:52:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f430ffff700 1 -- 172.21.15.7:0/3307124981 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314011d20 con 0x7f43180ce270 2024-09-13T18:52:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f431e65a700 1 -- 172.21.15.7:0/3307124981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43180ce270 msgr2=0x7f431815caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f431e65a700 1 --2- 172.21.15.7:0/3307124981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43180ce270 0x7f431815caf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4314009880 tx=0x7f4314009b90 comp rx=0 tx=0).stop 2024-09-13T18:52:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f431e65a700 1 -- 172.21.15.7:0/3307124981 shutdown_connections 2024-09-13T18:52:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f431e65a700 1 --2- 172.21.15.7:0/3307124981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43180ce270 0x7f431815caf0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f431e65a700 1 -- 172.21.15.7:0/3307124981 >> 172.21.15.7:0/3307124981 conn(0x7f43180b0e10 msgr2=0x7f43180b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f431e65a700 1 -- 172.21.15.7:0/3307124981 shutdown_connections 2024-09-13T18:52:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.485+0000 7f431e65a700 1 -- 172.21.15.7:0/3307124981 wait complete. 2024-09-13T18:52:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.489+0000 7f431e65a700 1 Processor -- start 2024-09-13T18:52:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.489+0000 7f431e65a700 1 -- start start 2024-09-13T18:52:59.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.489+0000 7f431e65a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f431806d6f0 0x7f431814b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:59.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.489+0000 7f431e65a700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f43180ce270 0x7f431815a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.489+0000 7f431e65a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431815d030 0x7f43180ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.489+0000 7f431e65a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431814db10 con 0x7f431815d030 2024-09-13T18:52:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.489+0000 7f431e65a700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431815b300 con 0x7f431806d6f0 2024-09-13T18:52:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.489+0000 7f431e65a700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431815b5e0 con 0x7f43180ce270 2024-09-13T18:52:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.493+0000 7f431de59700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431815d030 0x7f43180ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.493+0000 7f431de59700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431815d030 0x7f43180ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39304/0 (socket says 172.21.15.7:39304) 2024-09-13T18:52:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.493+0000 7f431de59700 1 -- 172.21.15.7:0/1959866725 learned_addr learned my addr 172.21.15.7:0/1959866725 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.493+0000 7f431ce57700 1 --2- 172.21.15.7:0/1959866725 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f43180ce270 0x7f431815a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.493+0000 7f431d658700 1 --2- 172.21.15.7:0/1959866725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f431806d6f0 0x7f431814b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.493+0000 7f431ce57700 1 -- 172.21.15.7:0/1959866725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f431806d6f0 msgr2=0x7f431814b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.493+0000 7f431ce57700 1 --2- 172.21.15.7:0/1959866725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f431806d6f0 0x7f431814b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.493+0000 7f431ce57700 1 -- 172.21.15.7:0/1959866725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431815d030 msgr2=0x7f43180ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.493+0000 7f431ce57700 1 --2- 172.21.15.7:0/1959866725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431815d030 0x7f43180ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.493+0000 7f431ce57700 1 -- 172.21.15.7:0/1959866725 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43140094b0 con 0x7f43180ce270 2024-09-13T18:52:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.493+0000 7f431de59700 1 --2- 172.21.15.7:0/1959866725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431815d030 0x7f43180ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:52:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.493+0000 7f431ce57700 1 --2- 172.21.15.7:0/1959866725 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f43180ce270 0x7f431815a840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4314016000 tx=0x7f4314004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.493+0000 7f430dffb700 1 -- 172.21.15.7:0/1959866725 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314004610 con 0x7f43180ce270 2024-09-13T18:52:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.493+0000 7f430dffb700 1 -- 172.21.15.7:0/1959866725 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431401f070 con 0x7f43180ce270 2024-09-13T18:52:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.493+0000 7f430dffb700 1 -- 172.21.15.7:0/1959866725 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314003920 con 0x7f43180ce270 2024-09-13T18:52:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.493+0000 7f431e65a700 1 -- 172.21.15.7:0/1959866725 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431814d730 con 0x7f43180ce270 2024-09-13T18:52:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.493+0000 7f431e65a700 1 -- 172.21.15.7:0/1959866725 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431814c590 con 0x7f43180ce270 2024-09-13T18:52:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.493+0000 7f431e65a700 1 -- 172.21.15.7:0/1959866725 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431814d590 con 0x7f43180ce270 2024-09-13T18:52:59.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.497+0000 7f430dffb700 1 -- 172.21.15.7:0/1959866725 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4314003a80 con 0x7f43180ce270 2024-09-13T18:52:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.497+0000 7f430dffb700 1 --2- 172.21.15.7:0/1959866725 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f42fc066730 0x7f42fc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.497+0000 7f430dffb700 1 -- 172.21.15.7:0/1959866725 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4314021070 con 0x7f43180ce270 2024-09-13T18:52:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.497+0000 7f431d658700 1 --2- 172.21.15.7:0/1959866725 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f42fc066730 0x7f42fc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:59.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.497+0000 7f431d658700 1 --2- 172.21.15.7:0/1959866725 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f42fc066730 0x7f42fc068bf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4308007fd0 tx=0x7f4308007e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:59.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.505+0000 7f430dffb700 1 -- 172.21.15.7:0/1959866725 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4314052d80 con 0x7f43180ce270 2024-09-13T18:52:59.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.673+0000 7f431e65a700 1 -- 172.21.15.7:0/1959866725 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f431814d590 con 0x7f43180ce270 2024-09-13T18:52:59.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.677+0000 7f430dffb700 1 -- 172.21.15.7:0/1959866725 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f431814d590 con 0x7f43180ce270 2024-09-13T18:52:59.682 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:52:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.681+0000 7f431e65a700 1 -- 172.21.15.7:0/1959866725 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f42fc066730 msgr2=0x7f42fc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.681+0000 7f431e65a700 1 --2- 172.21.15.7:0/1959866725 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f42fc066730 0x7f42fc068bf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4308007fd0 tx=0x7f4308007e30 comp rx=0 tx=0).stop 2024-09-13T18:52:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.681+0000 7f431e65a700 1 -- 172.21.15.7:0/1959866725 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f43180ce270 msgr2=0x7f431815a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.681+0000 7f431e65a700 1 --2- 172.21.15.7:0/1959866725 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f43180ce270 0x7f431815a840 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4314016000 tx=0x7f4314004770 comp rx=0 tx=0).stop 2024-09-13T18:52:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.681+0000 7f431e65a700 1 -- 172.21.15.7:0/1959866725 shutdown_connections 2024-09-13T18:52:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.681+0000 7f431e65a700 1 --2- 172.21.15.7:0/1959866725 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f42fc066730 0x7f42fc068bf0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.681+0000 7f431e65a700 1 --2- 172.21.15.7:0/1959866725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f431815d030 0x7f43180ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.681+0000 7f431e65a700 1 --2- 172.21.15.7:0/1959866725 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f43180ce270 0x7f431815a840 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.681+0000 7f431e65a700 1 --2- 172.21.15.7:0/1959866725 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f431806d6f0 0x7f431814b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.681+0000 7f431e65a700 1 -- 172.21.15.7:0/1959866725 >> 172.21.15.7:0/1959866725 conn(0x7f43180b0e10 msgr2=0x7f4318158480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.681+0000 7f431e65a700 1 -- 172.21.15.7:0/1959866725 shutdown_connections 2024-09-13T18:52:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.681+0000 7f431e65a700 1 -- 172.21.15.7:0/1959866725 wait complete. 2024-09-13T18:52:59.702 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:52:59.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.809+0000 7f47a7fff700 1 Processor -- start 2024-09-13T18:52:59.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.809+0000 7f47a7fff700 1 -- start start 2024-09-13T18:52:59.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.809+0000 7f47a7fff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f47a8165160 0x7f47a80704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:59.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.809+0000 7f47a7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47a8070a00 0x7f47a8163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:59.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.809+0000 7f47a7fff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a8162570 con 0x7f47a806d8c0 2024-09-13T18:52:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.809+0000 7f47a7fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a8162e60 con 0x7f47a8070a00 2024-09-13T18:52:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.809+0000 7f47a7fff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a8071240 con 0x7f47a8165160 2024-09-13T18:52:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.809+0000 7f47a77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47a8070a00 0x7f47a8163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.809+0000 7f47a77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47a8070a00 0x7f47a8163280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39308/0 (socket says 172.21.15.7:39308) 2024-09-13T18:52:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.809+0000 7f47a77fe700 1 -- 172.21.15.7:0/1087696710 learned_addr learned my addr 172.21.15.7:0/1087696710 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.809+0000 7f47a67fc700 1 --2- 172.21.15.7:0/1087696710 >> v2:172.21.15.171:3300/0 conn(0x7f47a8165160 0x7f47a80704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.809+0000 7f47a77fe700 1 -- 172.21.15.7:0/1087696710 >> v2:172.21.15.171:3300/0 conn(0x7f47a8165160 msgr2=0x7f47a80704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:52:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.809+0000 7f47a77fe700 1 --2- 172.21.15.7:0/1087696710 >> v2:172.21.15.171:3300/0 conn(0x7f47a8165160 0x7f47a80704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.809+0000 7f47a77fe700 1 -- 172.21.15.7:0/1087696710 >> v1:172.21.15.171:6789/0 conn(0x7f47a806d8c0 legacy=0x7f47a8164a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.809+0000 7f47a77fe700 1 -- 172.21.15.7:0/1087696710 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a81627a0 con 0x7f47a8070a00 2024-09-13T18:52:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.813+0000 7f47a77fe700 1 --2- 172.21.15.7:0/1087696710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a8070a00 0x7f47a8163280 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f47a8158550 tx=0x7f47a0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ac60d8ad647aab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.813+0000 7f47a5ffb700 1 -- 172.21.15.7:0/1087696710 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0011830 con 0x7f47a8070a00 2024-09-13T18:52:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.813+0000 7f47a5ffb700 1 -- 172.21.15.7:0/1087696710 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a0011990 con 0x7f47a8070a00 2024-09-13T18:52:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.813+0000 7f47a5ffb700 1 -- 172.21.15.7:0/1087696710 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0011d20 con 0x7f47a8070a00 2024-09-13T18:52:59.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.813+0000 7f47a7fff700 1 -- 172.21.15.7:0/1087696710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a8070a00 msgr2=0x7f47a8163280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.813+0000 7f47a7fff700 1 --2- 172.21.15.7:0/1087696710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a8070a00 0x7f47a8163280 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f47a8158550 tx=0x7f47a0009bf0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.813+0000 7f47a7fff700 1 -- 172.21.15.7:0/1087696710 shutdown_connections 2024-09-13T18:52:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.813+0000 7f47a7fff700 1 --2- 172.21.15.7:0/1087696710 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a8070a00 0x7f47a8163280 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.813+0000 7f47a7fff700 1 --2- 172.21.15.7:0/1087696710 >> v2:172.21.15.171:3300/0 conn(0x7f47a8165160 0x7f47a80704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.813+0000 7f47a7fff700 1 -- 172.21.15.7:0/1087696710 >> 172.21.15.7:0/1087696710 conn(0x7f47a80b0e10 msgr2=0x7f47a80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:52:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.813+0000 7f47a7fff700 1 -- 172.21.15.7:0/1087696710 shutdown_connections 2024-09-13T18:52:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.813+0000 7f47a7fff700 1 -- 172.21.15.7:0/1087696710 wait complete. 2024-09-13T18:52:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.817+0000 7f47a7fff700 1 Processor -- start 2024-09-13T18:52:59.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.817+0000 7f47a7fff700 1 -- start start 2024-09-13T18:52:59.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.817+0000 7f47a7fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f47a806d8c0 0x7f47a806cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:59.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.817+0000 7f47a7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a8070a00 0x7f47a814fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:59.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.817+0000 7f47a7fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47a8165160 0x7f47a815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.817+0000 7f47a7fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8159270 con 0x7f47a8070a00 2024-09-13T18:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.817+0000 7f47a7fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a815d7a0 con 0x7f47a8165160 2024-09-13T18:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.817+0000 7f47a7fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a815da80 con 0x7f47a806d8c0 2024-09-13T18:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.817+0000 7f47a67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a8070a00 0x7f47a814fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.817+0000 7f47a67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a8070a00 0x7f47a814fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39320/0 (socket says 172.21.15.7:39320) 2024-09-13T18:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.817+0000 7f47a67fc700 1 -- 172.21.15.7:0/2645702025 learned_addr learned my addr 172.21.15.7:0/2645702025 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.817+0000 7f47a77fe700 1 --2- 172.21.15.7:0/2645702025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47a8165160 0x7f47a815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.817+0000 7f47a6ffd700 1 --2- 172.21.15.7:0/2645702025 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f47a806d8c0 0x7f47a806cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.817+0000 7f47a67fc700 1 -- 172.21.15.7:0/2645702025 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f47a806d8c0 msgr2=0x7f47a806cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.817+0000 7f47a67fc700 1 --2- 172.21.15.7:0/2645702025 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f47a806d8c0 0x7f47a806cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.817+0000 7f47a67fc700 1 -- 172.21.15.7:0/2645702025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47a8165160 msgr2=0x7f47a815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.817+0000 7f47a67fc700 1 --2- 172.21.15.7:0/2645702025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47a8165160 0x7f47a815f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:52:59.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.821+0000 7f47a67fc700 1 -- 172.21.15.7:0/2645702025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a0009570 con 0x7f47a8070a00 2024-09-13T18:52:59.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.821+0000 7f47a77fe700 1 --2- 172.21.15.7:0/2645702025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47a8165160 0x7f47a815f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:52:59.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.821+0000 7f47a67fc700 1 --2- 172.21.15.7:0/2645702025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a8070a00 0x7f47a814fbd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f479800efc0 tx=0x7f479800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:59.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.821+0000 7f479f7fe700 1 -- 172.21.15.7:0/2645702025 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798009b80 con 0x7f47a8070a00 2024-09-13T18:52:59.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.821+0000 7f479f7fe700 1 -- 172.21.15.7:0/2645702025 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479800f040 con 0x7f47a8070a00 2024-09-13T18:52:59.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.821+0000 7f47a7fff700 1 -- 172.21.15.7:0/2645702025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a8071010 con 0x7f47a8070a00 2024-09-13T18:52:59.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.821+0000 7f479f7fe700 1 -- 172.21.15.7:0/2645702025 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798014480 con 0x7f47a8070a00 2024-09-13T18:52:59.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.821+0000 7f47a7fff700 1 -- 172.21.15.7:0/2645702025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a815e040 con 0x7f47a8070a00 2024-09-13T18:52:59.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.821+0000 7f47a7fff700 1 -- 172.21.15.7:0/2645702025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a80704c0 con 0x7f47a8070a00 2024-09-13T18:52:59.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.821+0000 7f479f7fe700 1 -- 172.21.15.7:0/2645702025 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f47980145e0 con 0x7f47a8070a00 2024-09-13T18:52:59.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.825+0000 7f479f7fe700 1 --2- 172.21.15.7:0/2645702025 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4788066730 0x7f4788068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:52:59.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.825+0000 7f479f7fe700 1 -- 172.21.15.7:0/2645702025 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4798084a20 con 0x7f47a8070a00 2024-09-13T18:52:59.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.825+0000 7f47a6ffd700 1 --2- 172.21.15.7:0/2645702025 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4788066730 0x7f4788068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:52:59.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.825+0000 7f47a6ffd700 1 --2- 172.21.15.7:0/2645702025 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4788066730 0x7f4788068bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4790000f60 tx=0x7f4790009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:52:59.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:52:59.833+0000 7f479f7fe700 1 -- 172.21.15.7:0/2645702025 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4798051d90 con 0x7f47a8070a00 2024-09-13T18:53:00.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.029+0000 7f47a7fff700 1 -- 172.21.15.7:0/2645702025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f47a81647c0 con 0x7f47a8070a00 2024-09-13T18:53:00.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.029+0000 7f479f7fe700 1 -- 172.21.15.7:0/2645702025 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f47980553b0 con 0x7f47a8070a00 2024-09-13T18:53:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.033+0000 7f47a7fff700 1 -- 172.21.15.7:0/2645702025 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4788066730 msgr2=0x7f4788068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.033+0000 7f47a7fff700 1 --2- 172.21.15.7:0/2645702025 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4788066730 0x7f4788068bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4790000f60 tx=0x7f4790009040 comp rx=0 tx=0).stop 2024-09-13T18:53:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.033+0000 7f47a7fff700 1 -- 172.21.15.7:0/2645702025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a8070a00 msgr2=0x7f47a814fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.033+0000 7f47a7fff700 1 --2- 172.21.15.7:0/2645702025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a8070a00 0x7f47a814fbd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f479800efc0 tx=0x7f479800c3a0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.033+0000 7f47a7fff700 1 -- 172.21.15.7:0/2645702025 shutdown_connections 2024-09-13T18:53:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.033+0000 7f47a7fff700 1 --2- 172.21.15.7:0/2645702025 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4788066730 0x7f4788068bf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.033+0000 7f47a7fff700 1 --2- 172.21.15.7:0/2645702025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f47a8165160 0x7f47a815f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.033+0000 7f47a7fff700 1 --2- 172.21.15.7:0/2645702025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a8070a00 0x7f47a814fbd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.033+0000 7f47a7fff700 1 --2- 172.21.15.7:0/2645702025 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f47a806d8c0 0x7f47a806cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.033+0000 7f47a7fff700 1 -- 172.21.15.7:0/2645702025 >> 172.21.15.7:0/2645702025 conn(0x7f47a80b0e10 msgr2=0x7f47a80aec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.033+0000 7f47a7fff700 1 -- 172.21.15.7:0/2645702025 shutdown_connections 2024-09-13T18:53:00.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.033+0000 7f47a7fff700 1 -- 172.21.15.7:0/2645702025 wait complete. 2024-09-13T18:53:00.042 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-13T18:53:00.057 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2024-09-13T18:53:00.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:00.057 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-09-13T18:53:00.065 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:53:00.066 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-13T18:53:00.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:00.066 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-13T18:53:00.070 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:53:00.071 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:00.078 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:53:00.078 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:00.086 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:53:00.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.189+0000 7f0dc2117700 1 Processor -- start 2024-09-13T18:53:00.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.189+0000 7f0dc2117700 1 -- start start 2024-09-13T18:53:00.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.189+0000 7f0dc2117700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0dbc15cae0 0x7f0dbc0cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:00.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.189+0000 7f0dc2117700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dbc0d21e0 con 0x7f0dbc15cec0 2024-09-13T18:53:00.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.189+0000 7f0dc2117700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dbc0d1940 con 0x7f0dbc06b310 2024-09-13T18:53:00.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.189+0000 7f0dc2117700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dbc0d1c20 con 0x7f0dbc15cae0 2024-09-13T18:53:00.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.189+0000 7f0dc1115700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0dbc15cae0 0x7f0dbc0cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:00.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.189+0000 7f0dc0914700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0dbc15cec0 0x7f0dbc06ac00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38872/0 (socket says 172.21.15.7:38872) 2024-09-13T18:53:00.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.189+0000 7f0dc0914700 1 -- 172.21.15.7:0/307955011 learned_addr learned my addr 172.21.15.7:0/307955011 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:00.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.189+0000 7f0dc1115700 1 -- 172.21.15.7:0/307955011 >> v1:172.21.15.171:6789/0 conn(0x7f0dbc06b310 legacy=0x7f0dbc0b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.189+0000 7f0dc1115700 1 -- 172.21.15.7:0/307955011 >> v1:172.21.15.7:6789/0 conn(0x7f0dbc15cec0 legacy=0x7f0dbc06ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.189+0000 7f0dc1115700 1 -- 172.21.15.7:0/307955011 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc0d1f00 con 0x7f0dbc15cae0 2024-09-13T18:53:00.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.189+0000 7f0dc1115700 1 --2- 172.21.15.7:0/307955011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dbc15cae0 0x7f0dbc0cf030 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009880 tx=0x7f0db8009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52cf12c3e6771432 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.193+0000 7f0db3fff700 1 -- 172.21.15.7:0/307955011 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db8011830 con 0x7f0dbc15cae0 2024-09-13T18:53:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.193+0000 7f0db3fff700 1 -- 172.21.15.7:0/307955011 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db8011990 con 0x7f0dbc15cae0 2024-09-13T18:53:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.193+0000 7f0db3fff700 1 -- 172.21.15.7:0/307955011 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db8011d20 con 0x7f0dbc15cae0 2024-09-13T18:53:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.193+0000 7f0dc2117700 1 -- 172.21.15.7:0/307955011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dbc15cae0 msgr2=0x7f0dbc0cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.193+0000 7f0dc2117700 1 --2- 172.21.15.7:0/307955011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dbc15cae0 0x7f0dbc0cf030 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009880 tx=0x7f0db8009b90 comp rx=0 tx=0).stop 2024-09-13T18:53:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.193+0000 7f0dc2117700 1 -- 172.21.15.7:0/307955011 shutdown_connections 2024-09-13T18:53:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.193+0000 7f0dc2117700 1 --2- 172.21.15.7:0/307955011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dbc15cae0 0x7f0dbc0cf030 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.193+0000 7f0dc2117700 1 -- 172.21.15.7:0/307955011 >> 172.21.15.7:0/307955011 conn(0x7f0dbc0d2c80 msgr2=0x7f0dbc0d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.193+0000 7f0dc2117700 1 -- 172.21.15.7:0/307955011 shutdown_connections 2024-09-13T18:53:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.193+0000 7f0dc2117700 1 -- 172.21.15.7:0/307955011 wait complete. 2024-09-13T18:53:00.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc2117700 1 Processor -- start 2024-09-13T18:53:00.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc2117700 1 -- start start 2024-09-13T18:53:00.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc2117700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0dbc06b310 0x7f0dbc1f2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:00.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc2117700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0dbc15cae0 0x7f0dbc1f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:00.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc2117700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dbc15cec0 0x7f0dbc15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:00.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc2117700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc0a9800 con 0x7f0dbc15cec0 2024-09-13T18:53:00.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc2117700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc15f7c0 con 0x7f0dbc15cae0 2024-09-13T18:53:00.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc2117700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc1f2fc0 con 0x7f0dbc06b310 2024-09-13T18:53:00.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc1916700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dbc15cec0 0x7f0dbc15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:00.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc0914700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0dbc15cae0 0x7f0dbc1f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:00.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc1916700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dbc15cec0 0x7f0dbc15e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39350/0 (socket says 172.21.15.7:39350) 2024-09-13T18:53:00.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc1916700 1 -- 172.21.15.7:0/3714621645 learned_addr learned my addr 172.21.15.7:0/3714621645 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:00.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc1115700 1 --2- 172.21.15.7:0/3714621645 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0dbc06b310 0x7f0dbc1f2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:00.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc1916700 1 -- 172.21.15.7:0/3714621645 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0dbc06b310 msgr2=0x7f0dbc1f2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc1916700 1 --2- 172.21.15.7:0/3714621645 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0dbc06b310 0x7f0dbc1f2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc1916700 1 -- 172.21.15.7:0/3714621645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0dbc15cae0 msgr2=0x7f0dbc1f2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc1916700 1 --2- 172.21.15.7:0/3714621645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0dbc15cae0 0x7f0dbc1f2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc1916700 1 -- 172.21.15.7:0/3714621645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db80094b0 con 0x7f0dbc15cec0 2024-09-13T18:53:00.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc1916700 1 --2- 172.21.15.7:0/3714621645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dbc15cec0 0x7f0dbc15e2f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0db400cb50 tx=0x7f0db400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:00.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0db1ffb700 1 -- 172.21.15.7:0/3714621645 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db4003c20 con 0x7f0dbc15cec0 2024-09-13T18:53:00.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0db1ffb700 1 -- 172.21.15.7:0/3714621645 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db4003d80 con 0x7f0dbc15cec0 2024-09-13T18:53:00.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc2117700 1 -- 172.21.15.7:0/3714621645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dbc15eb40 con 0x7f0dbc15cec0 2024-09-13T18:53:00.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0db1ffb700 1 -- 172.21.15.7:0/3714621645 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db400aa20 con 0x7f0dbc15cec0 2024-09-13T18:53:00.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.197+0000 7f0dc2117700 1 -- 172.21.15.7:0/3714621645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dbc15f100 con 0x7f0dbc15cec0 2024-09-13T18:53:00.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.201+0000 7f0db1ffb700 1 -- 172.21.15.7:0/3714621645 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0db400ab80 con 0x7f0dbc15cec0 2024-09-13T18:53:00.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.201+0000 7f0dc2117700 1 -- 172.21.15.7:0/3714621645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dbc15f900 con 0x7f0dbc15cec0 2024-09-13T18:53:00.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.201+0000 7f0db1ffb700 1 --2- 172.21.15.7:0/3714621645 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0da0066730 0x7f0da0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:00.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.201+0000 7f0db1ffb700 1 -- 172.21.15.7:0/3714621645 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0db4084110 con 0x7f0dbc15cec0 2024-09-13T18:53:00.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.201+0000 7f0dc1115700 1 --2- 172.21.15.7:0/3714621645 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0da0066730 0x7f0da0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:00.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.201+0000 7f0dc1115700 1 --2- 172.21.15.7:0/3714621645 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0da0066730 0x7f0da0068bf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0db8000c00 tx=0x7f0db8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:00.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.209+0000 7f0db1ffb700 1 -- 172.21.15.7:0/3714621645 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0dbc15f900 con 0x7f0dbc15cec0 2024-09-13T18:53:00.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.389+0000 7f0dc2117700 1 -- 172.21.15.7:0/3714621645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0dbc15f900 con 0x7f0dbc15cec0 2024-09-13T18:53:00.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.393+0000 7f0db1ffb700 1 -- 172.21.15.7:0/3714621645 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7f0dbc15f900 con 0x7f0dbc15cec0 2024-09-13T18:53:00.399 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:00.399 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:53:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.393+0000 7f0dc2117700 1 -- 172.21.15.7:0/3714621645 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0da0066730 msgr2=0x7f0da0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.393+0000 7f0dc2117700 1 --2- 172.21.15.7:0/3714621645 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0da0066730 0x7f0da0068bf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0db8000c00 tx=0x7f0db8008030 comp rx=0 tx=0).stop 2024-09-13T18:53:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.393+0000 7f0dc2117700 1 -- 172.21.15.7:0/3714621645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dbc15cec0 msgr2=0x7f0dbc15e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.393+0000 7f0dc2117700 1 --2- 172.21.15.7:0/3714621645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dbc15cec0 0x7f0dbc15e2f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0db400cb50 tx=0x7f0db400ce60 comp rx=0 tx=0).stop 2024-09-13T18:53:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.393+0000 7f0dc2117700 1 -- 172.21.15.7:0/3714621645 shutdown_connections 2024-09-13T18:53:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.393+0000 7f0dc2117700 1 --2- 172.21.15.7:0/3714621645 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0da0066730 0x7f0da0068bf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.393+0000 7f0dc2117700 1 --2- 172.21.15.7:0/3714621645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dbc15cec0 0x7f0dbc15e2f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.393+0000 7f0dc2117700 1 --2- 172.21.15.7:0/3714621645 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0dbc15cae0 0x7f0dbc1f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.393+0000 7f0dc2117700 1 --2- 172.21.15.7:0/3714621645 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0dbc06b310 0x7f0dbc1f2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.393+0000 7f0dc2117700 1 -- 172.21.15.7:0/3714621645 >> 172.21.15.7:0/3714621645 conn(0x7f0dbc0d2c80 msgr2=0x7f0dbc0cbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:00.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.393+0000 7f0dc2117700 1 -- 172.21.15.7:0/3714621645 shutdown_connections 2024-09-13T18:53:00.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.393+0000 7f0dc2117700 1 -- 172.21.15.7:0/3714621645 wait complete. 2024-09-13T18:53:00.413 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:52:15.554683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:52:16.896879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:53:00.413 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:53:00.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.525+0000 7faba3fff700 1 Processor -- start 2024-09-13T18:53:00.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.525+0000 7faba3fff700 1 -- start start 2024-09-13T18:53:00.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.525+0000 7faba3fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faba4071b60 0x7faba40ad980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:00.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.525+0000 7faba3fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba40ae000 con 0x7faba4146160 2024-09-13T18:53:00.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.525+0000 7faba3fff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faba4070070 con 0x7faba40ce300 2024-09-13T18:53:00.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.525+0000 7faba3fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba40cc820 con 0x7faba4071b60 2024-09-13T18:53:00.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.525+0000 7faba2ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7faba4146160 0x7faba40cdbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38880/0 (socket says 172.21.15.7:38880) 2024-09-13T18:53:00.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.525+0000 7faba2ffd700 1 -- 172.21.15.7:0/2541675806 learned_addr learned my addr 172.21.15.7:0/2541675806 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:00.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.525+0000 7faba37fe700 1 --2- 172.21.15.7:0/2541675806 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faba4071b60 0x7faba40ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:00.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.529+0000 7faba1ffb700 1 -- 172.21.15.7:0/2541675806 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3976462958 0 0) 0x7faba40ae000 con 0x7faba4146160 2024-09-13T18:53:00.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.529+0000 7faba1ffb700 1 -- 172.21.15.7:0/2541675806 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab900035d0 con 0x7faba4146160 2024-09-13T18:53:00.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.529+0000 7faba1ffb700 1 -- 172.21.15.7:0/2541675806 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 47648901 0 0) 0x7faba4070070 con 0x7faba40ce300 2024-09-13T18:53:00.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.529+0000 7faba1ffb700 1 -- 172.21.15.7:0/2541675806 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faba40ae000 con 0x7faba40ce300 2024-09-13T18:53:00.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.529+0000 7faba37fe700 1 -- 172.21.15.7:0/2541675806 >> v1:172.21.15.171:6789/0 conn(0x7faba40ce300 legacy=0x7faba40ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.529+0000 7faba37fe700 1 -- 172.21.15.7:0/2541675806 >> v1:172.21.15.7:6789/0 conn(0x7faba4146160 legacy=0x7faba40cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.529+0000 7faba37fe700 1 -- 172.21.15.7:0/2541675806 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba4070350 con 0x7faba4071b60 2024-09-13T18:53:00.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.529+0000 7faba37fe700 1 --2- 172.21.15.7:0/2541675806 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faba4071b60 0x7faba40ad980 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009880 tx=0x7fab9c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1c17a7ac39cca05 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:00.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.529+0000 7faba1ffb700 1 -- 172.21.15.7:0/2541675806 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c011830 con 0x7faba4071b60 2024-09-13T18:53:00.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.529+0000 7faba1ffb700 1 -- 172.21.15.7:0/2541675806 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c011990 con 0x7faba4071b60 2024-09-13T18:53:00.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.529+0000 7faba1ffb700 1 -- 172.21.15.7:0/2541675806 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c011d00 con 0x7faba4071b60 2024-09-13T18:53:00.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.529+0000 7faba3fff700 1 -- 172.21.15.7:0/2541675806 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faba4071b60 msgr2=0x7faba40ad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.529+0000 7faba3fff700 1 --2- 172.21.15.7:0/2541675806 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faba4071b60 0x7faba40ad980 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009880 tx=0x7fab9c009b90 comp rx=0 tx=0).stop 2024-09-13T18:53:00.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.529+0000 7faba3fff700 1 -- 172.21.15.7:0/2541675806 shutdown_connections 2024-09-13T18:53:00.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.529+0000 7faba3fff700 1 --2- 172.21.15.7:0/2541675806 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faba4071b60 0x7faba40ad980 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.529+0000 7faba3fff700 1 -- 172.21.15.7:0/2541675806 >> 172.21.15.7:0/2541675806 conn(0x7faba40b0e10 msgr2=0x7faba40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:00.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.529+0000 7faba3fff700 1 -- 172.21.15.7:0/2541675806 shutdown_connections 2024-09-13T18:53:00.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.529+0000 7faba3fff700 1 -- 172.21.15.7:0/2541675806 wait complete. 2024-09-13T18:53:00.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba3fff700 1 Processor -- start 2024-09-13T18:53:00.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba3fff700 1 -- start start 2024-09-13T18:53:00.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba4071b60 0x7faba4143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:00.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba3fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faba40ce300 0x7faba413be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:00.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba3fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faba4146160 0x7faba415c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:00.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba3fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba413c670 con 0x7faba4071b60 2024-09-13T18:53:00.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba3fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba415c930 con 0x7faba40ce300 2024-09-13T18:53:00.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba3fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faba406b850 con 0x7faba4146160 2024-09-13T18:53:00.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba4071b60 0x7faba4143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:00.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba27fc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faba40ce300 0x7faba413be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:00.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba4071b60 0x7faba4143bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39352/0 (socket says 172.21.15.7:39352) 2024-09-13T18:53:00.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba2ffd700 1 -- 172.21.15.7:0/2187670610 learned_addr learned my addr 172.21.15.7:0/2187670610 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:00.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba37fe700 1 --2- 172.21.15.7:0/2187670610 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faba4146160 0x7faba415c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:00.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba2ffd700 1 -- 172.21.15.7:0/2187670610 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faba4146160 msgr2=0x7faba415c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba2ffd700 1 --2- 172.21.15.7:0/2187670610 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faba4146160 0x7faba415c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba2ffd700 1 -- 172.21.15.7:0/2187670610 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faba40ce300 msgr2=0x7faba413be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba2ffd700 1 --2- 172.21.15.7:0/2187670610 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faba40ce300 0x7faba413be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba2ffd700 1 -- 172.21.15.7:0/2187670610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c0094b0 con 0x7faba4071b60 2024-09-13T18:53:00.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba2ffd700 1 --2- 172.21.15.7:0/2187670610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba4071b60 0x7faba4143bd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fab9400c700 tx=0x7fab9400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:00.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7fab9bfff700 1 -- 172.21.15.7:0/2187670610 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab94005d90 con 0x7faba4071b60 2024-09-13T18:53:00.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba3fff700 1 -- 172.21.15.7:0/2187670610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba406bb30 con 0x7faba4071b60 2024-09-13T18:53:00.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7fab9bfff700 1 -- 172.21.15.7:0/2187670610 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab940070d0 con 0x7faba4071b60 2024-09-13T18:53:00.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7fab9bfff700 1 -- 172.21.15.7:0/2187670610 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab94007480 con 0x7faba4071b60 2024-09-13T18:53:00.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.533+0000 7faba3fff700 1 -- 172.21.15.7:0/2187670610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba406c0f0 con 0x7faba4071b60 2024-09-13T18:53:00.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.537+0000 7faba3fff700 1 -- 172.21.15.7:0/2187670610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab88005310 con 0x7faba4071b60 2024-09-13T18:53:00.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.537+0000 7fab9bfff700 1 -- 172.21.15.7:0/2187670610 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fab940075e0 con 0x7faba4071b60 2024-09-13T18:53:00.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.537+0000 7fab9bfff700 1 --2- 172.21.15.7:0/2187670610 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fab84066730 0x7fab84068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:00.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.537+0000 7fab9bfff700 1 -- 172.21.15.7:0/2187670610 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab940835e0 con 0x7faba4071b60 2024-09-13T18:53:00.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.537+0000 7faba27fc700 1 --2- 172.21.15.7:0/2187670610 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fab84066730 0x7fab84068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:00.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.537+0000 7faba27fc700 1 --2- 172.21.15.7:0/2187670610 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fab84066730 0x7fab84068bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faba406b640 tx=0x7fab8c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.541+0000 7fab9bfff700 1 -- 172.21.15.7:0/2187670610 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fab940508d0 con 0x7faba4071b60 2024-09-13T18:53:00.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.713+0000 7faba3fff700 1 -- 172.21.15.7:0/2187670610 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab88005b40 con 0x7faba4071b60 2024-09-13T18:53:00.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.713+0000 7fab9bfff700 1 -- 172.21.15.7:0/2187670610 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab94053ef0 con 0x7faba4071b60 2024-09-13T18:53:00.719 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:53:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.717+0000 7faba3fff700 1 -- 172.21.15.7:0/2187670610 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fab84066730 msgr2=0x7fab84068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.717+0000 7faba3fff700 1 --2- 172.21.15.7:0/2187670610 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fab84066730 0x7fab84068bf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faba406b640 tx=0x7fab8c009040 comp rx=0 tx=0).stop 2024-09-13T18:53:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.717+0000 7faba3fff700 1 -- 172.21.15.7:0/2187670610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba4071b60 msgr2=0x7faba4143bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.717+0000 7faba3fff700 1 --2- 172.21.15.7:0/2187670610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba4071b60 0x7faba4143bd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fab9400c700 tx=0x7fab9400ca10 comp rx=0 tx=0).stop 2024-09-13T18:53:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.717+0000 7faba3fff700 1 -- 172.21.15.7:0/2187670610 shutdown_connections 2024-09-13T18:53:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.717+0000 7faba3fff700 1 --2- 172.21.15.7:0/2187670610 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fab84066730 0x7fab84068bf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.717+0000 7faba3fff700 1 --2- 172.21.15.7:0/2187670610 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faba4146160 0x7faba415c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.717+0000 7faba3fff700 1 --2- 172.21.15.7:0/2187670610 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faba40ce300 0x7faba413be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.717+0000 7faba3fff700 1 --2- 172.21.15.7:0/2187670610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faba4071b60 0x7faba4143bd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.717+0000 7faba3fff700 1 -- 172.21.15.7:0/2187670610 >> 172.21.15.7:0/2187670610 conn(0x7faba40b0e10 msgr2=0x7faba406a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.717+0000 7faba3fff700 1 -- 172.21.15.7:0/2187670610 shutdown_connections 2024-09-13T18:53:00.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.717+0000 7faba3fff700 1 -- 172.21.15.7:0/2187670610 wait complete. 2024-09-13T18:53:00.737 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:53:00.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f1547fff700 1 Processor -- start 2024-09-13T18:53:00.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f1547fff700 1 -- start start 2024-09-13T18:53:00.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f1547fff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f154815cda0 0x7f154806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:00.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f1547fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15480ceb70 0x7f15480b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:00.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f1547fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f154806a520 0x7f154806a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:00.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f1547fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f154806be40 con 0x7f15480ceb70 2024-09-13T18:53:00.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f1547fff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15480b8430 con 0x7f154815cda0 2024-09-13T18:53:00.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f1547fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154806ae90 con 0x7f154806a520 2024-09-13T18:53:00.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f15467fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15480ceb70 0x7f15480b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:00.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f15467fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15480ceb70 0x7f15480b5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39364/0 (socket says 172.21.15.7:39364) 2024-09-13T18:53:00.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f15467fc700 1 -- 172.21.15.7:0/892737604 learned_addr learned my addr 172.21.15.7:0/892737604 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:00.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f15477fe700 1 --2- 172.21.15.7:0/892737604 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f154806a520 0x7f154806a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:00.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f1546ffd700 1 --2- 172.21.15.7:0/892737604 >> v2:172.21.15.171:3300/0 conn(0x7f154815cda0 0x7f154806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f15467fc700 1 -- 172.21.15.7:0/892737604 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f154806a520 msgr2=0x7f154806a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f15467fc700 1 --2- 172.21.15.7:0/892737604 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f154806a520 0x7f154806a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f15467fc700 1 -- 172.21.15.7:0/892737604 >> v2:172.21.15.171:3300/0 conn(0x7f154815cda0 msgr2=0x7f154806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f15467fc700 1 --2- 172.21.15.7:0/892737604 >> v2:172.21.15.171:3300/0 conn(0x7f154815cda0 0x7f154806b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f15467fc700 1 -- 172.21.15.7:0/892737604 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15480cf350 con 0x7f15480ceb70 2024-09-13T18:53:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f1546ffd700 1 --2- 172.21.15.7:0/892737604 >> v2:172.21.15.171:3300/0 conn(0x7f154815cda0 0x7f154806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f15467fc700 1 --2- 172.21.15.7:0/892737604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15480ceb70 0x7f15480b5de0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f15480d0800 tx=0x7f153000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1dcceea07e583d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f1545ffb700 1 -- 172.21.15.7:0/892737604 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153000e070 con 0x7f15480ceb70 2024-09-13T18:53:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f1545ffb700 1 -- 172.21.15.7:0/892737604 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153000b910 con 0x7f15480ceb70 2024-09-13T18:53:00.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.853+0000 7f1545ffb700 1 -- 172.21.15.7:0/892737604 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153000bcc0 con 0x7f15480ceb70 2024-09-13T18:53:00.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.857+0000 7f1547fff700 1 -- 172.21.15.7:0/892737604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15480ceb70 msgr2=0x7f15480b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.857+0000 7f1547fff700 1 --2- 172.21.15.7:0/892737604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15480ceb70 0x7f15480b5de0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f15480d0800 tx=0x7f153000b330 comp rx=0 tx=0).stop 2024-09-13T18:53:00.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.857+0000 7f1547fff700 1 -- 172.21.15.7:0/892737604 shutdown_connections 2024-09-13T18:53:00.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.857+0000 7f1547fff700 1 --2- 172.21.15.7:0/892737604 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f154806a520 0x7f154806a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.857+0000 7f1547fff700 1 --2- 172.21.15.7:0/892737604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15480ceb70 0x7f15480b5de0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.857+0000 7f1547fff700 1 --2- 172.21.15.7:0/892737604 >> v2:172.21.15.171:3300/0 conn(0x7f154815cda0 0x7f154806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.857+0000 7f1547fff700 1 -- 172.21.15.7:0/892737604 >> 172.21.15.7:0/892737604 conn(0x7f15480d2cb0 msgr2=0x7f15480d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.857+0000 7f1547fff700 1 -- 172.21.15.7:0/892737604 shutdown_connections 2024-09-13T18:53:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.857+0000 7f1547fff700 1 -- 172.21.15.7:0/892737604 wait complete. 2024-09-13T18:53:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f1547fff700 1 Processor -- start 2024-09-13T18:53:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f1547fff700 1 -- start start 2024-09-13T18:53:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f1547fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f154806a520 0x7f15480c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f1547fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f15480ceb70 0x7f15481f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f1547fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f154815cda0 0x7f1548144e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f1547fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15480c6350 con 0x7f154815cda0 2024-09-13T18:53:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f1547fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15480c8790 con 0x7f154806a520 2024-09-13T18:53:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f1547fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15481f6910 con 0x7f15480ceb70 2024-09-13T18:53:00.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f15477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f154815cda0 0x7f1548144e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:00.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f15477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f154815cda0 0x7f1548144e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39380/0 (socket says 172.21.15.7:39380) 2024-09-13T18:53:00.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f15477fe700 1 -- 172.21.15.7:0/226761579 learned_addr learned my addr 172.21.15.7:0/226761579 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:00.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f15467fc700 1 --2- 172.21.15.7:0/226761579 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f15480ceb70 0x7f15481f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:00.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f1546ffd700 1 --2- 172.21.15.7:0/226761579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f154806a520 0x7f15480c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f15477fe700 1 -- 172.21.15.7:0/226761579 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f15480ceb70 msgr2=0x7f15481f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f15477fe700 1 --2- 172.21.15.7:0/226761579 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f15480ceb70 0x7f15481f5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f15477fe700 1 -- 172.21.15.7:0/226761579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f154806a520 msgr2=0x7f15480c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f15477fe700 1 --2- 172.21.15.7:0/226761579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f154806a520 0x7f15480c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f15477fe700 1 -- 172.21.15.7:0/226761579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1530009b30 con 0x7f154815cda0 2024-09-13T18:53:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f15477fe700 1 --2- 172.21.15.7:0/226761579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f154815cda0 0x7f1548144e00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f154000eb90 tx=0x7f154000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.861+0000 7f153f7fe700 1 -- 172.21.15.7:0/226761579 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1540009740 con 0x7f154815cda0 2024-09-13T18:53:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.865+0000 7f153f7fe700 1 -- 172.21.15.7:0/226761579 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154000c850 con 0x7f154815cda0 2024-09-13T18:53:00.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.865+0000 7f1547fff700 1 -- 172.21.15.7:0/226761579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15480c8d30 con 0x7f154815cda0 2024-09-13T18:53:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.865+0000 7f153f7fe700 1 -- 172.21.15.7:0/226761579 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154000cbe0 con 0x7f154815cda0 2024-09-13T18:53:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.865+0000 7f1547fff700 1 -- 172.21.15.7:0/226761579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15481f6400 con 0x7f154815cda0 2024-09-13T18:53:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.865+0000 7f1547fff700 1 -- 172.21.15.7:0/226761579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15480c8ec0 con 0x7f154815cda0 2024-09-13T18:53:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.865+0000 7f153f7fe700 1 -- 172.21.15.7:0/226761579 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f154000cd40 con 0x7f154815cda0 2024-09-13T18:53:00.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.865+0000 7f153f7fe700 1 --2- 172.21.15.7:0/226761579 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1528066730 0x7f1528068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:00.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.865+0000 7f153f7fe700 1 -- 172.21.15.7:0/226761579 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1540084440 con 0x7f154815cda0 2024-09-13T18:53:00.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.865+0000 7f1546ffd700 1 --2- 172.21.15.7:0/226761579 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1528066730 0x7f1528068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:00.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.865+0000 7f1546ffd700 1 --2- 172.21.15.7:0/226761579 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1528066730 0x7f1528068bf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1538004430 tx=0x7f1538008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:00.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:00.873+0000 7f153f7fe700 1 -- 172.21.15.7:0/226761579 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f15400517b0 con 0x7f154815cda0 2024-09-13T18:53:01.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.045+0000 7f1547fff700 1 -- 172.21.15.7:0/226761579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f15480cf000 con 0x7f154815cda0 2024-09-13T18:53:01.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.045+0000 7f153f7fe700 1 -- 172.21.15.7:0/226761579 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1540054dd0 con 0x7f154815cda0 2024-09-13T18:53:01.052 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:53:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.049+0000 7f1547fff700 1 -- 172.21.15.7:0/226761579 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1528066730 msgr2=0x7f1528068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.049+0000 7f1547fff700 1 --2- 172.21.15.7:0/226761579 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1528066730 0x7f1528068bf0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1538004430 tx=0x7f1538008040 comp rx=0 tx=0).stop 2024-09-13T18:53:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.049+0000 7f1547fff700 1 -- 172.21.15.7:0/226761579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f154815cda0 msgr2=0x7f1548144e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.049+0000 7f1547fff700 1 --2- 172.21.15.7:0/226761579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f154815cda0 0x7f1548144e00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f154000eb90 tx=0x7f154000eea0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.049+0000 7f1547fff700 1 -- 172.21.15.7:0/226761579 shutdown_connections 2024-09-13T18:53:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.049+0000 7f1547fff700 1 --2- 172.21.15.7:0/226761579 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1528066730 0x7f1528068bf0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.049+0000 7f1547fff700 1 --2- 172.21.15.7:0/226761579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f154815cda0 0x7f1548144e00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.049+0000 7f1547fff700 1 --2- 172.21.15.7:0/226761579 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f15480ceb70 0x7f15481f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.049+0000 7f1547fff700 1 --2- 172.21.15.7:0/226761579 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f154806a520 0x7f15480c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.049+0000 7f1547fff700 1 -- 172.21.15.7:0/226761579 >> 172.21.15.7:0/226761579 conn(0x7f15480d2cb0 msgr2=0x7f15480d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:01.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.049+0000 7f1547fff700 1 -- 172.21.15.7:0/226761579 shutdown_connections 2024-09-13T18:53:01.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.049+0000 7f1547fff700 1 -- 172.21.15.7:0/226761579 wait complete. 2024-09-13T18:53:01.068 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:53:01.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.185+0000 7fd086265700 1 Processor -- start 2024-09-13T18:53:01.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.185+0000 7fd086265700 1 -- start start 2024-09-13T18:53:01.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.185+0000 7fd086265700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd08006b900 0x7fd0800b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:01.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.185+0000 7fd086265700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd08006a520 0x7fd08006a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:01.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd086265700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0800b8430 con 0x7fd08015cda0 2024-09-13T18:53:01.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd086265700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08006ae90 con 0x7fd08006b900 2024-09-13T18:53:01.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd086265700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08006bd60 con 0x7fd08006a520 2024-09-13T18:53:01.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd085263700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd08015cda0 0x7fd0800ceb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47536/0 (socket says 172.21.15.7:47536) 2024-09-13T18:53:01.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd085263700 1 -- 172.21.15.7:0/4005300678 learned_addr learned my addr 172.21.15.7:0/4005300678 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:01.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd084a62700 1 --2- 172.21.15.7:0/4005300678 >> v2:172.21.15.171:3300/0 conn(0x7fd08006b900 0x7fd0800b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:01.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd085a64700 1 --2- 172.21.15.7:0/4005300678 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd08006a520 0x7fd08006a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:01.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd073fff700 1 -- 172.21.15.7:0/4005300678 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24685113 0 0) 0x7fd0800b8430 con 0x7fd08015cda0 2024-09-13T18:53:01.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd073fff700 1 -- 172.21.15.7:0/4005300678 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd06c0035d0 con 0x7fd08015cda0 2024-09-13T18:53:01.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd084a62700 1 -- 172.21.15.7:0/4005300678 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd08006a520 msgr2=0x7fd08006a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:01.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd084a62700 1 --2- 172.21.15.7:0/4005300678 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd08006a520 0x7fd08006a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd084a62700 1 -- 172.21.15.7:0/4005300678 >> v1:172.21.15.7:6789/0 conn(0x7fd08015cda0 legacy=0x7fd0800ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:01.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd084a62700 1 -- 172.21.15.7:0/4005300678 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0800cf350 con 0x7fd08006b900 2024-09-13T18:53:01.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd085a64700 1 --2- 172.21.15.7:0/4005300678 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd08006a520 0x7fd08006a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:01.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd084a62700 1 --2- 172.21.15.7:0/4005300678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd08006b900 0x7fd0800b5de0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009850 tx=0x7fd07c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de007ba1eb6b1709 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:01.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd073fff700 1 -- 172.21.15.7:0/4005300678 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c011830 con 0x7fd08006b900 2024-09-13T18:53:01.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd073fff700 1 -- 172.21.15.7:0/4005300678 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c011990 con 0x7fd08006b900 2024-09-13T18:53:01.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd073fff700 1 -- 172.21.15.7:0/4005300678 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c011d00 con 0x7fd08006b900 2024-09-13T18:53:01.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd086265700 1 -- 172.21.15.7:0/4005300678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd08006b900 msgr2=0x7fd0800b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:01.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd086265700 1 --2- 172.21.15.7:0/4005300678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd08006b900 0x7fd0800b5de0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009850 tx=0x7fd07c009b60 comp rx=0 tx=0).stop 2024-09-13T18:53:01.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd086265700 1 -- 172.21.15.7:0/4005300678 shutdown_connections 2024-09-13T18:53:01.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd086265700 1 --2- 172.21.15.7:0/4005300678 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd08006a520 0x7fd08006a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd086265700 1 --2- 172.21.15.7:0/4005300678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd08006b900 0x7fd0800b5de0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd086265700 1 -- 172.21.15.7:0/4005300678 >> 172.21.15.7:0/4005300678 conn(0x7fd0800d2cb0 msgr2=0x7fd0800d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:01.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd086265700 1 -- 172.21.15.7:0/4005300678 shutdown_connections 2024-09-13T18:53:01.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.189+0000 7fd086265700 1 -- 172.21.15.7:0/4005300678 wait complete. 2024-09-13T18:53:01.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd086265700 1 Processor -- start 2024-09-13T18:53:01.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd086265700 1 -- start start 2024-09-13T18:53:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd086265700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08006a520 0x7fd0800cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd086265700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd08006b900 0x7fd0801f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd086265700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd08015cda0 0x7fd080148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:01.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd086265700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0800c6f20 con 0x7fd08006a520 2024-09-13T18:53:01.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd086265700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0801f6ce0 con 0x7fd08015cda0 2024-09-13T18:53:01.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd086265700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0800cc360 con 0x7fd08006b900 2024-09-13T18:53:01.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd085263700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08006a520 0x7fd0800cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:01.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd085263700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08006a520 0x7fd0800cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59726/0 (socket says 172.21.15.7:59726) 2024-09-13T18:53:01.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd085263700 1 -- 172.21.15.7:0/45868248 learned_addr learned my addr 172.21.15.7:0/45868248 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:01.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd085a64700 1 --2- 172.21.15.7:0/45868248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd08015cda0 0x7fd080148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:01.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd084a62700 1 --2- 172.21.15.7:0/45868248 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd08006b900 0x7fd0801f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd085263700 1 -- 172.21.15.7:0/45868248 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd08006b900 msgr2=0x7fd0801f67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd085263700 1 --2- 172.21.15.7:0/45868248 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd08006b900 0x7fd0801f67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd085263700 1 -- 172.21.15.7:0/45868248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd08015cda0 msgr2=0x7fd080148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd085263700 1 --2- 172.21.15.7:0/45868248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd08015cda0 0x7fd080148080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd085263700 1 -- 172.21.15.7:0/45868248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd078009a70 con 0x7fd08006a520 2024-09-13T18:53:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.193+0000 7fd085263700 1 --2- 172.21.15.7:0/45868248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08006a520 0x7fd0800cba80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd07400c4d0 tx=0x7fd07400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.197+0000 7fd071ffb700 1 -- 172.21.15.7:0/45868248 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074005d90 con 0x7fd08006a520 2024-09-13T18:53:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.197+0000 7fd071ffb700 1 -- 172.21.15.7:0/45868248 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07400ce70 con 0x7fd08006a520 2024-09-13T18:53:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.197+0000 7fd086265700 1 -- 172.21.15.7:0/45868248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07c0094b0 con 0x7fd08006a520 2024-09-13T18:53:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.197+0000 7fd071ffb700 1 -- 172.21.15.7:0/45868248 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0740072d0 con 0x7fd08006a520 2024-09-13T18:53:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.197+0000 7fd086265700 1 -- 172.21.15.7:0/45868248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0801485c0 con 0x7fd08006a520 2024-09-13T18:53:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.197+0000 7fd086265700 1 -- 172.21.15.7:0/45868248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08006a950 con 0x7fd08006a520 2024-09-13T18:53:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.197+0000 7fd071ffb700 1 -- 172.21.15.7:0/45868248 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd074007430 con 0x7fd08006a520 2024-09-13T18:53:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.197+0000 7fd071ffb700 1 --2- 172.21.15.7:0/45868248 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd064066730 0x7fd064068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.197+0000 7fd071ffb700 1 -- 172.21.15.7:0/45868248 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd074083860 con 0x7fd08006a520 2024-09-13T18:53:01.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.201+0000 7fd084a62700 1 --2- 172.21.15.7:0/45868248 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd064066730 0x7fd064068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:01.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.201+0000 7fd084a62700 1 --2- 172.21.15.7:0/45868248 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd064066730 0x7fd064068bf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0090b0 tx=0x7fd07c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:01.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.205+0000 7fd071ffb700 1 -- 172.21.15.7:0/45868248 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd074050d10 con 0x7fd08006a520 2024-09-13T18:53:01.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.377+0000 7fd086265700 1 -- 172.21.15.7:0/45868248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0800cd200 con 0x7fd08006a520 2024-09-13T18:53:01.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.377+0000 7fd071ffb700 1 -- 172.21.15.7:0/45868248 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd0740170a0 con 0x7fd08006a520 2024-09-13T18:53:01.650 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:53:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.381+0000 7fd086265700 1 -- 172.21.15.7:0/45868248 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd064066730 msgr2=0x7fd064068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.381+0000 7fd086265700 1 --2- 172.21.15.7:0/45868248 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd064066730 0x7fd064068bf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0090b0 tx=0x7fd07c008030 comp rx=0 tx=0).stop 2024-09-13T18:53:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.381+0000 7fd086265700 1 -- 172.21.15.7:0/45868248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08006a520 msgr2=0x7fd0800cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.381+0000 7fd086265700 1 --2- 172.21.15.7:0/45868248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08006a520 0x7fd0800cba80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd07400c4d0 tx=0x7fd07400c7e0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.381+0000 7fd086265700 1 -- 172.21.15.7:0/45868248 shutdown_connections 2024-09-13T18:53:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.381+0000 7fd086265700 1 --2- 172.21.15.7:0/45868248 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd064066730 0x7fd064068bf0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.381+0000 7fd086265700 1 --2- 172.21.15.7:0/45868248 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd08015cda0 0x7fd080148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.381+0000 7fd086265700 1 --2- 172.21.15.7:0/45868248 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd08006b900 0x7fd0801f67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.381+0000 7fd086265700 1 --2- 172.21.15.7:0/45868248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08006a520 0x7fd0800cba80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.381+0000 7fd086265700 1 -- 172.21.15.7:0/45868248 >> 172.21.15.7:0/45868248 conn(0x7fd0800d2cb0 msgr2=0x7fd0800d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:01.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.381+0000 7fd086265700 1 -- 172.21.15.7:0/45868248 shutdown_connections 2024-09-13T18:53:01.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.381+0000 7fd086265700 1 -- 172.21.15.7:0/45868248 wait complete. 2024-09-13T18:53:01.652 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:53:01.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.773+0000 7eff8df7f700 1 Processor -- start 2024-09-13T18:53:01.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.773+0000 7eff8df7f700 1 -- start start 2024-09-13T18:53:01.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.773+0000 7eff8df7f700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff88144340 0x7eff8806d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:01.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.773+0000 7eff8df7f700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7eff880cdc90 0x7eff8806fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:01.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.773+0000 7eff8df7f700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff88144720 con 0x7eff8806bdb0 2024-09-13T18:53:01.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.773+0000 7eff8df7f700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff8806dd40 con 0x7eff880cdc90 2024-09-13T18:53:01.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.773+0000 7eff8df7f700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8806da60 con 0x7eff88144340 2024-09-13T18:53:01.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.773+0000 7eff8cf7d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff88144340 0x7eff8806d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:01.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.773+0000 7eff8d77e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7eff880cdc90 0x7eff8806fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:01.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.773+0000 7eff87fff700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7eff8806bdb0 0x7eff880cd580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:58876/0 (socket says 172.21.15.7:58876) 2024-09-13T18:53:01.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.773+0000 7eff87fff700 1 -- 172.21.15.7:0/2845920278 learned_addr learned my addr 172.21.15.7:0/2845920278 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:01.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.773+0000 7eff8cf7d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff88144340 0x7eff8806d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:32812/0 (socket says 172.21.15.7:32812) 2024-09-13T18:53:01.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.773+0000 7eff877fe700 1 -- 172.21.15.7:0/2845920278 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2504842828 0 0) 0x7eff88144720 con 0x7eff8806bdb0 2024-09-13T18:53:01.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.777+0000 7eff877fe700 1 -- 172.21.15.7:0/2845920278 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff700035d0 con 0x7eff8806bdb0 2024-09-13T18:53:01.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.777+0000 7eff8cf7d700 1 -- 172.21.15.7:0/2845920278 >> v2:172.21.15.171:3300/0 conn(0x7eff880cdc90 msgr2=0x7eff8806fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:01.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.777+0000 7eff8cf7d700 1 --2- 172.21.15.7:0/2845920278 >> v2:172.21.15.171:3300/0 conn(0x7eff880cdc90 0x7eff8806fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.777+0000 7eff8cf7d700 1 -- 172.21.15.7:0/2845920278 >> v1:172.21.15.171:6789/0 conn(0x7eff8806bdb0 legacy=0x7eff880cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:01.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.777+0000 7eff8cf7d700 1 -- 172.21.15.7:0/2845920278 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8815e660 con 0x7eff88144340 2024-09-13T18:53:01.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.777+0000 7eff8d77e700 1 --2- 172.21.15.7:0/2845920278 >> v2:172.21.15.171:3300/0 conn(0x7eff880cdc90 0x7eff8806fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:01.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.777+0000 7eff8cf7d700 1 --2- 172.21.15.7:0/2845920278 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff88144340 0x7eff8806d480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009850 tx=0x7eff7c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9af7825ff3453a40 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:01.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.777+0000 7eff877fe700 1 -- 172.21.15.7:0/2845920278 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c011830 con 0x7eff88144340 2024-09-13T18:53:01.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.777+0000 7eff877fe700 1 -- 172.21.15.7:0/2845920278 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7c011990 con 0x7eff88144340 2024-09-13T18:53:01.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.777+0000 7eff877fe700 1 -- 172.21.15.7:0/2845920278 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7c011d40 con 0x7eff88144340 2024-09-13T18:53:01.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.777+0000 7eff8df7f700 1 -- 172.21.15.7:0/2845920278 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff88144340 msgr2=0x7eff8806d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:01.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.777+0000 7eff8df7f700 1 --2- 172.21.15.7:0/2845920278 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff88144340 0x7eff8806d480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009850 tx=0x7eff7c009b60 comp rx=0 tx=0).stop 2024-09-13T18:53:01.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.777+0000 7eff8df7f700 1 -- 172.21.15.7:0/2845920278 shutdown_connections 2024-09-13T18:53:01.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.777+0000 7eff8df7f700 1 --2- 172.21.15.7:0/2845920278 >> v2:172.21.15.171:3300/0 conn(0x7eff880cdc90 0x7eff8806fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.777+0000 7eff8df7f700 1 --2- 172.21.15.7:0/2845920278 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff88144340 0x7eff8806d480 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.777+0000 7eff8df7f700 1 -- 172.21.15.7:0/2845920278 >> 172.21.15.7:0/2845920278 conn(0x7eff880b0e10 msgr2=0x7eff880b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:01.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.777+0000 7eff8df7f700 1 -- 172.21.15.7:0/2845920278 shutdown_connections 2024-09-13T18:53:01.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.777+0000 7eff8df7f700 1 -- 172.21.15.7:0/2845920278 wait complete. 2024-09-13T18:53:01.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.781+0000 7eff8df7f700 1 Processor -- start 2024-09-13T18:53:01.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.781+0000 7eff8df7f700 1 -- start start 2024-09-13T18:53:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.781+0000 7eff8df7f700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff8806bdb0 0x7eff88142570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.781+0000 7eff8df7f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff880cdc90 0x7eff88141c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.781+0000 7eff8df7f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff88144340 0x7eff880c3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.781+0000 7eff8df7f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff88142ab0 con 0x7eff880cdc90 2024-09-13T18:53:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.781+0000 7eff8df7f700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff88143480 con 0x7eff88144340 2024-09-13T18:53:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.781+0000 7eff8df7f700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff880aa4d0 con 0x7eff8806bdb0 2024-09-13T18:53:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.781+0000 7eff87fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff880cdc90 0x7eff88141c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.781+0000 7eff87fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff880cdc90 0x7eff88141c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59740/0 (socket says 172.21.15.7:59740) 2024-09-13T18:53:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.781+0000 7eff87fff700 1 -- 172.21.15.7:0/1733085702 learned_addr learned my addr 172.21.15.7:0/1733085702 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.781+0000 7eff8d77e700 1 --2- 172.21.15.7:0/1733085702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff88144340 0x7eff880c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.781+0000 7eff8cf7d700 1 --2- 172.21.15.7:0/1733085702 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff8806bdb0 0x7eff88142570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.785+0000 7eff87fff700 1 -- 172.21.15.7:0/1733085702 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff8806bdb0 msgr2=0x7eff88142570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.785+0000 7eff87fff700 1 --2- 172.21.15.7:0/1733085702 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff8806bdb0 0x7eff88142570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.785+0000 7eff87fff700 1 -- 172.21.15.7:0/1733085702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff88144340 msgr2=0x7eff880c3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.785+0000 7eff87fff700 1 --2- 172.21.15.7:0/1733085702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff88144340 0x7eff880c3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.785+0000 7eff87fff700 1 -- 172.21.15.7:0/1733085702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff80009a70 con 0x7eff880cdc90 2024-09-13T18:53:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.785+0000 7eff8d77e700 1 --2- 172.21.15.7:0/1733085702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff88144340 0x7eff880c3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.785+0000 7eff87fff700 1 --2- 172.21.15.7:0/1733085702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff880cdc90 0x7eff88141c20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7eff7800c500 tx=0x7eff7800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.785+0000 7eff857fa700 1 -- 172.21.15.7:0/1733085702 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7800cea0 con 0x7eff880cdc90 2024-09-13T18:53:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.785+0000 7eff857fa700 1 -- 172.21.15.7:0/1733085702 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff78005db0 con 0x7eff880cdc90 2024-09-13T18:53:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.785+0000 7eff8df7f700 1 -- 172.21.15.7:0/1733085702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7c0094b0 con 0x7eff880cdc90 2024-09-13T18:53:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.785+0000 7eff857fa700 1 -- 172.21.15.7:0/1733085702 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff78007310 con 0x7eff880cdc90 2024-09-13T18:53:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.785+0000 7eff8df7f700 1 -- 172.21.15.7:0/1733085702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff880a9710 con 0x7eff880cdc90 2024-09-13T18:53:01.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.785+0000 7eff8df7f700 1 -- 172.21.15.7:0/1733085702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8806a310 con 0x7eff880cdc90 2024-09-13T18:53:01.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.785+0000 7eff857fa700 1 -- 172.21.15.7:0/1733085702 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7eff78007470 con 0x7eff880cdc90 2024-09-13T18:53:01.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.785+0000 7eff857fa700 1 --2- 172.21.15.7:0/1733085702 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7eff74066730 0x7eff74068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:01.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.789+0000 7eff857fa700 1 -- 172.21.15.7:0/1733085702 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff780834e0 con 0x7eff880cdc90 2024-09-13T18:53:01.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.789+0000 7eff8cf7d700 1 --2- 172.21.15.7:0/1733085702 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7eff74066730 0x7eff74068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.789+0000 7eff8cf7d700 1 --2- 172.21.15.7:0/1733085702 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7eff74066730 0x7eff74068bf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7eff7c0090b0 tx=0x7eff7c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.793+0000 7eff857fa700 1 -- 172.21.15.7:0/1733085702 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7eff780507b0 con 0x7eff880cdc90 2024-09-13T18:53:01.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.977+0000 7eff8df7f700 1 -- 172.21.15.7:0/1733085702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7eff881437a0 con 0x7eff880cdc90 2024-09-13T18:53:01.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.977+0000 7eff857fa700 1 -- 172.21.15.7:0/1733085702 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7eff78053dd0 con 0x7eff880cdc90 2024-09-13T18:53:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.981+0000 7eff8df7f700 1 -- 172.21.15.7:0/1733085702 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7eff74066730 msgr2=0x7eff74068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.981+0000 7eff8df7f700 1 --2- 172.21.15.7:0/1733085702 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7eff74066730 0x7eff74068bf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7eff7c0090b0 tx=0x7eff7c008030 comp rx=0 tx=0).stop 2024-09-13T18:53:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.981+0000 7eff8df7f700 1 -- 172.21.15.7:0/1733085702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff880cdc90 msgr2=0x7eff88141c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.981+0000 7eff8df7f700 1 --2- 172.21.15.7:0/1733085702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff880cdc90 0x7eff88141c20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7eff7800c500 tx=0x7eff7800c810 comp rx=0 tx=0).stop 2024-09-13T18:53:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.981+0000 7eff8df7f700 1 -- 172.21.15.7:0/1733085702 shutdown_connections 2024-09-13T18:53:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.981+0000 7eff8df7f700 1 --2- 172.21.15.7:0/1733085702 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7eff74066730 0x7eff74068bf0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.981+0000 7eff8df7f700 1 --2- 172.21.15.7:0/1733085702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff88144340 0x7eff880c3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.981+0000 7eff8df7f700 1 --2- 172.21.15.7:0/1733085702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff880cdc90 0x7eff88141c20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.981+0000 7eff8df7f700 1 --2- 172.21.15.7:0/1733085702 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff8806bdb0 0x7eff88142570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.981+0000 7eff8df7f700 1 -- 172.21.15.7:0/1733085702 >> 172.21.15.7:0/1733085702 conn(0x7eff880b0e10 msgr2=0x7eff8815d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.981+0000 7eff8df7f700 1 -- 172.21.15.7:0/1733085702 shutdown_connections 2024-09-13T18:53:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:01.981+0000 7eff8df7f700 1 -- 172.21.15.7:0/1733085702 wait complete. 2024-09-13T18:53:01.989 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-13T18:53:02.000 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:53:02.000 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:02.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:53:02.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.117+0000 7fcb4979f700 1 Processor -- start 2024-09-13T18:53:02.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.117+0000 7fcb4979f700 1 -- start start 2024-09-13T18:53:02.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb4979f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcb440aea40 0x7fcb44072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:02.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb4979f700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb4406dff0 0x7fcb4406e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:02.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb4979f700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb4415f400 con 0x7fcb4415e560 2024-09-13T18:53:02.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb4979f700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb440715b0 con 0x7fcb440aea40 2024-09-13T18:53:02.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb4979f700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb44072540 con 0x7fcb4406dff0 2024-09-13T18:53:02.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb43fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcb440aea40 0x7fcb44072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:02.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb43fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcb440aea40 0x7fcb44072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59750/0 (socket says 172.21.15.7:59750) 2024-09-13T18:53:02.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb43fff700 1 -- 172.21.15.7:0/2272253625 learned_addr learned my addr 172.21.15.7:0/2272253625 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:02.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb437fe700 1 --2- 172.21.15.7:0/2272253625 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb4406dff0 0x7fcb4406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:02.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb43fff700 1 -- 172.21.15.7:0/2272253625 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb4406dff0 msgr2=0x7fcb4406e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:02.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb43fff700 1 --2- 172.21.15.7:0/2272253625 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb4406dff0 0x7fcb4406e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb43fff700 1 -- 172.21.15.7:0/2272253625 >> v1:172.21.15.171:6789/0 conn(0x7fcb4415e560 legacy=0x7fcb4406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb43fff700 1 -- 172.21.15.7:0/2272253625 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb440ce2a0 con 0x7fcb440aea40 2024-09-13T18:53:02.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb437fe700 1 --2- 172.21.15.7:0/2272253625 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb4406dff0 0x7fcb4406e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:02.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb43fff700 1 --2- 172.21.15.7:0/2272253625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb440aea40 0x7fcb44072000 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009910 tx=0x7fcb34009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3dba34fc2129f57 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:02.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb42ffd700 1 -- 172.21.15.7:0/2272253625 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3400ba60 con 0x7fcb440aea40 2024-09-13T18:53:02.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb42ffd700 1 -- 172.21.15.7:0/2272253625 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3400bbc0 con 0x7fcb440aea40 2024-09-13T18:53:02.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb42ffd700 1 -- 172.21.15.7:0/2272253625 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34009e60 con 0x7fcb440aea40 2024-09-13T18:53:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb4979f700 1 -- 172.21.15.7:0/2272253625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb440aea40 msgr2=0x7fcb44072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb4979f700 1 --2- 172.21.15.7:0/2272253625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb440aea40 0x7fcb44072000 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009910 tx=0x7fcb34009c20 comp rx=0 tx=0).stop 2024-09-13T18:53:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb4979f700 1 -- 172.21.15.7:0/2272253625 shutdown_connections 2024-09-13T18:53:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb4979f700 1 --2- 172.21.15.7:0/2272253625 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb4406dff0 0x7fcb4406e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb4979f700 1 --2- 172.21.15.7:0/2272253625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb440aea40 0x7fcb44072000 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb4979f700 1 -- 172.21.15.7:0/2272253625 >> 172.21.15.7:0/2272253625 conn(0x7fcb440b1780 msgr2=0x7fcb440b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb4979f700 1 -- 172.21.15.7:0/2272253625 shutdown_connections 2024-09-13T18:53:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.121+0000 7fcb4979f700 1 -- 172.21.15.7:0/2272253625 wait complete. 2024-09-13T18:53:02.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb4979f700 1 Processor -- start 2024-09-13T18:53:02.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb4979f700 1 -- start start 2024-09-13T18:53:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb4979f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4406dff0 0x7fcb44156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb4979f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb4415e560 0x7fcb441f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb4979f700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb4415cf80 0x7fcb441f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb4979f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4414e790 con 0x7fcb4406dff0 2024-09-13T18:53:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb4979f700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb441560e0 con 0x7fcb4415e560 2024-09-13T18:53:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb4979f700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb44156380 con 0x7fcb4415cf80 2024-09-13T18:53:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb43fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4406dff0 0x7fcb44156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb43fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4406dff0 0x7fcb44156ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59766/0 (socket says 172.21.15.7:59766) 2024-09-13T18:53:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb43fff700 1 -- 172.21.15.7:0/4065885073 learned_addr learned my addr 172.21.15.7:0/4065885073 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb437fe700 1 --2- 172.21.15.7:0/4065885073 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb4415e560 0x7fcb441f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb48f9e700 1 --2- 172.21.15.7:0/4065885073 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb4415cf80 0x7fcb441f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb43fff700 1 -- 172.21.15.7:0/4065885073 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb4415cf80 msgr2=0x7fcb441f6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb43fff700 1 --2- 172.21.15.7:0/4065885073 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb4415cf80 0x7fcb441f6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb43fff700 1 -- 172.21.15.7:0/4065885073 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb4415e560 msgr2=0x7fcb441f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb43fff700 1 --2- 172.21.15.7:0/4065885073 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb4415e560 0x7fcb441f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb43fff700 1 -- 172.21.15.7:0/4065885073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb34009570 con 0x7fcb4406dff0 2024-09-13T18:53:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb43fff700 1 --2- 172.21.15.7:0/4065885073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4406dff0 0x7fcb44156ca0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcb34000c00 tx=0x7fcb34004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb40ff9700 1 -- 172.21.15.7:0/4065885073 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb340044d0 con 0x7fcb4406dff0 2024-09-13T18:53:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb4979f700 1 -- 172.21.15.7:0/4065885073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb44156660 con 0x7fcb4406dff0 2024-09-13T18:53:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb40ff9700 1 -- 172.21.15.7:0/4065885073 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3401f070 con 0x7fcb4406dff0 2024-09-13T18:53:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb40ff9700 1 -- 172.21.15.7:0/4065885073 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34003900 con 0x7fcb4406dff0 2024-09-13T18:53:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.125+0000 7fcb4979f700 1 -- 172.21.15.7:0/4065885073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb441571e0 con 0x7fcb4406dff0 2024-09-13T18:53:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.129+0000 7fcb4979f700 1 -- 172.21.15.7:0/4065885073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1c005310 con 0x7fcb4406dff0 2024-09-13T18:53:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.129+0000 7fcb40ff9700 1 -- 172.21.15.7:0/4065885073 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcb34003a60 con 0x7fcb4406dff0 2024-09-13T18:53:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.129+0000 7fcb40ff9700 1 --2- 172.21.15.7:0/4065885073 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb28066730 0x7fcb28068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.129+0000 7fcb40ff9700 1 -- 172.21.15.7:0/4065885073 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb34021070 con 0x7fcb4406dff0 2024-09-13T18:53:02.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.129+0000 7fcb437fe700 1 --2- 172.21.15.7:0/4065885073 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb28066730 0x7fcb28068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.129+0000 7fcb437fe700 1 --2- 172.21.15.7:0/4065885073 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb28066730 0x7fcb28068bf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcb4415cf10 tx=0x7fcb2c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:02.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.133+0000 7fcb40ff9700 1 -- 172.21.15.7:0/4065885073 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcb340528d0 con 0x7fcb4406dff0 2024-09-13T18:53:02.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.309+0000 7fcb4979f700 1 -- 172.21.15.7:0/4065885073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb1c005890 con 0x7fcb4406dff0 2024-09-13T18:53:02.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.313+0000 7fcb40ff9700 1 -- 172.21.15.7:0/4065885073 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7fcb34055ef0 con 0x7fcb4406dff0 2024-09-13T18:53:02.320 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:02.321 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:53:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.317+0000 7fcb4979f700 1 -- 172.21.15.7:0/4065885073 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb28066730 msgr2=0x7fcb28068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.317+0000 7fcb4979f700 1 --2- 172.21.15.7:0/4065885073 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb28066730 0x7fcb28068bf0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcb4415cf10 tx=0x7fcb2c008040 comp rx=0 tx=0).stop 2024-09-13T18:53:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.317+0000 7fcb4979f700 1 -- 172.21.15.7:0/4065885073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4406dff0 msgr2=0x7fcb44156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.317+0000 7fcb4979f700 1 --2- 172.21.15.7:0/4065885073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4406dff0 0x7fcb44156ca0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcb34000c00 tx=0x7fcb34004770 comp rx=0 tx=0).stop 2024-09-13T18:53:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.317+0000 7fcb4979f700 1 -- 172.21.15.7:0/4065885073 shutdown_connections 2024-09-13T18:53:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.317+0000 7fcb4979f700 1 --2- 172.21.15.7:0/4065885073 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb28066730 0x7fcb28068bf0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.317+0000 7fcb4979f700 1 --2- 172.21.15.7:0/4065885073 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb4415cf80 0x7fcb441f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.317+0000 7fcb4979f700 1 --2- 172.21.15.7:0/4065885073 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb4415e560 0x7fcb441f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.317+0000 7fcb4979f700 1 --2- 172.21.15.7:0/4065885073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4406dff0 0x7fcb44156ca0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.317+0000 7fcb4979f700 1 -- 172.21.15.7:0/4065885073 >> 172.21.15.7:0/4065885073 conn(0x7fcb440b1780 msgr2=0x7fcb440add90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.317+0000 7fcb4979f700 1 -- 172.21.15.7:0/4065885073 shutdown_connections 2024-09-13T18:53:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.317+0000 7fcb4979f700 1 -- 172.21.15.7:0/4065885073 wait complete. 2024-09-13T18:53:02.336 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:52:15.554683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:52:16.896879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:53:02.336 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:53:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.449+0000 7fc4fea3d700 1 Processor -- start 2024-09-13T18:53:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.449+0000 7fc4fea3d700 1 -- start start 2024-09-13T18:53:02.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fea3d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4f815ce40 0x7fc4f806aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:02.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fea3d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc4f80ce1d0 0x7fc4f80b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:02.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fea3d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f80ce600 con 0x7fc4f806b400 2024-09-13T18:53:02.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fea3d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f806b840 con 0x7fc4f80ce1d0 2024-09-13T18:53:02.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fea3d700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f80ce830 con 0x7fc4f815ce40 2024-09-13T18:53:02.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fcfda700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc4f80ce1d0 0x7fc4f80b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:02.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4f77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc4f806b400 0x7fc4f80cdac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47550/0 (socket says 172.21.15.7:47550) 2024-09-13T18:53:02.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4f77fe700 1 -- 172.21.15.7:0/3430558526 learned_addr learned my addr 172.21.15.7:0/3430558526 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:02.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4f7fff700 1 --2- 172.21.15.7:0/3430558526 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4f815ce40 0x7fc4f806aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4f6ffd700 1 -- 172.21.15.7:0/3430558526 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817672176 0 0) 0x7fc4f80ce600 con 0x7fc4f806b400 2024-09-13T18:53:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4f6ffd700 1 -- 172.21.15.7:0/3430558526 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4e00035d0 con 0x7fc4f806b400 2024-09-13T18:53:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fcfda700 1 -- 172.21.15.7:0/3430558526 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4f815ce40 msgr2=0x7fc4f806aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fcfda700 1 --2- 172.21.15.7:0/3430558526 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4f815ce40 0x7fc4f806aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fcfda700 1 -- 172.21.15.7:0/3430558526 >> v1:172.21.15.7:6789/0 conn(0x7fc4f806b400 legacy=0x7fc4f80cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fcfda700 1 -- 172.21.15.7:0/3430558526 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f80ceb10 con 0x7fc4f80ce1d0 2024-09-13T18:53:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fcfda700 1 --2- 172.21.15.7:0/3430558526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4f80ce1d0 0x7fc4f80b61c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009d90 tx=0x7fc4f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e2fa5447b85c8f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4f6ffd700 1 -- 172.21.15.7:0/3430558526 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f000e070 con 0x7fc4f80ce1d0 2024-09-13T18:53:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4f6ffd700 1 -- 172.21.15.7:0/3430558526 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f000bb20 con 0x7fc4f80ce1d0 2024-09-13T18:53:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4f6ffd700 1 -- 172.21.15.7:0/3430558526 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0012430 con 0x7fc4f80ce1d0 2024-09-13T18:53:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fea3d700 1 -- 172.21.15.7:0/3430558526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4f80ce1d0 msgr2=0x7fc4f80b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fea3d700 1 --2- 172.21.15.7:0/3430558526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4f80ce1d0 0x7fc4f80b61c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009d90 tx=0x7fc4f000b330 comp rx=0 tx=0).stop 2024-09-13T18:53:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fea3d700 1 -- 172.21.15.7:0/3430558526 shutdown_connections 2024-09-13T18:53:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fea3d700 1 --2- 172.21.15.7:0/3430558526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4f80ce1d0 0x7fc4f80b61c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fea3d700 1 --2- 172.21.15.7:0/3430558526 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4f815ce40 0x7fc4f806aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fea3d700 1 -- 172.21.15.7:0/3430558526 >> 172.21.15.7:0/3430558526 conn(0x7fc4f80d2f30 msgr2=0x7fc4f80d3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:02.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fea3d700 1 -- 172.21.15.7:0/3430558526 shutdown_connections 2024-09-13T18:53:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.453+0000 7fc4fea3d700 1 -- 172.21.15.7:0/3430558526 wait complete. 2024-09-13T18:53:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.457+0000 7fc4fea3d700 1 Processor -- start 2024-09-13T18:53:02.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.457+0000 7fc4fea3d700 1 -- start start 2024-09-13T18:53:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.457+0000 7fc4fea3d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4f806b400 0x7fc4f8157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.457+0000 7fc4fea3d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4f80ce1d0 0x7fc4f8159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.457+0000 7fc4fea3d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4f815ce40 0x7fc4f81484b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.457+0000 7fc4fea3d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f8159b60 con 0x7fc4f80ce1d0 2024-09-13T18:53:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.457+0000 7fc4fea3d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f8159e40 con 0x7fc4f806b400 2024-09-13T18:53:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.457+0000 7fc4fea3d700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f80c7970 con 0x7fc4f815ce40 2024-09-13T18:53:02.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.457+0000 7fc4f77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4f80ce1d0 0x7fc4f8159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.457+0000 7fc4f77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4f80ce1d0 0x7fc4f8159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59788/0 (socket says 172.21.15.7:59788) 2024-09-13T18:53:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.457+0000 7fc4f77fe700 1 -- 172.21.15.7:0/248827594 learned_addr learned my addr 172.21.15.7:0/248827594 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.457+0000 7fc4f7fff700 1 --2- 172.21.15.7:0/248827594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4f806b400 0x7fc4f8157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.457+0000 7fc4fcfda700 1 --2- 172.21.15.7:0/248827594 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4f815ce40 0x7fc4f81484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.461+0000 7fc4f77fe700 1 -- 172.21.15.7:0/248827594 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4f815ce40 msgr2=0x7fc4f81484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.461+0000 7fc4f77fe700 1 --2- 172.21.15.7:0/248827594 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4f815ce40 0x7fc4f81484b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.461+0000 7fc4f77fe700 1 -- 172.21.15.7:0/248827594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4f806b400 msgr2=0x7fc4f8157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.461+0000 7fc4f77fe700 1 --2- 172.21.15.7:0/248827594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4f806b400 0x7fc4f8157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.461+0000 7fc4f77fe700 1 -- 172.21.15.7:0/248827594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f0009a70 con 0x7fc4f80ce1d0 2024-09-13T18:53:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.461+0000 7fc4f77fe700 1 --2- 172.21.15.7:0/248827594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4f80ce1d0 0x7fc4f8159330 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00c4d0 tx=0x7fc4ec00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:02.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.461+0000 7fc4f4ff9700 1 -- 172.21.15.7:0/248827594 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec005d90 con 0x7fc4f80ce1d0 2024-09-13T18:53:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.461+0000 7fc4f4ff9700 1 -- 172.21.15.7:0/248827594 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ec0070d0 con 0x7fc4f80ce1d0 2024-09-13T18:53:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.461+0000 7fc4fea3d700 1 -- 172.21.15.7:0/248827594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f80c7c50 con 0x7fc4f80ce1d0 2024-09-13T18:53:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.461+0000 7fc4f4ff9700 1 -- 172.21.15.7:0/248827594 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec007480 con 0x7fc4f80ce1d0 2024-09-13T18:53:02.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.461+0000 7fc4fea3d700 1 -- 172.21.15.7:0/248827594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f8159870 con 0x7fc4f80ce1d0 2024-09-13T18:53:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.461+0000 7fc4fea3d700 1 -- 172.21.15.7:0/248827594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f80c8170 con 0x7fc4f80ce1d0 2024-09-13T18:53:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.461+0000 7fc4f4ff9700 1 -- 172.21.15.7:0/248827594 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc4ec0075e0 con 0x7fc4f80ce1d0 2024-09-13T18:53:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.465+0000 7fc4f4ff9700 1 --2- 172.21.15.7:0/248827594 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc4e4066730 0x7fc4e4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.465+0000 7fc4f4ff9700 1 -- 172.21.15.7:0/248827594 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4ec0834f0 con 0x7fc4f80ce1d0 2024-09-13T18:53:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.465+0000 7fc4f7fff700 1 --2- 172.21.15.7:0/248827594 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc4e4066730 0x7fc4e4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:02.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.465+0000 7fc4f7fff700 1 --2- 172.21.15.7:0/248827594 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc4e4066730 0x7fc4e4068bf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009bf0 tx=0x7fc4e8009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:02.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.473+0000 7fc4f4ff9700 1 -- 172.21.15.7:0/248827594 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc4f80c8170 con 0x7fc4f80ce1d0 2024-09-13T18:53:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.637+0000 7fc4fea3d700 1 -- 172.21.15.7:0/248827594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4f80c8170 con 0x7fc4f80ce1d0 2024-09-13T18:53:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.641+0000 7fc4f4ff9700 1 -- 172.21.15.7:0/248827594 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc4f80c8170 con 0x7fc4f80ce1d0 2024-09-13T18:53:02.646 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:53:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.641+0000 7fc4fea3d700 1 -- 172.21.15.7:0/248827594 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc4e4066730 msgr2=0x7fc4e4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.641+0000 7fc4fea3d700 1 --2- 172.21.15.7:0/248827594 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc4e4066730 0x7fc4e4068bf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009bf0 tx=0x7fc4e8009240 comp rx=0 tx=0).stop 2024-09-13T18:53:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.641+0000 7fc4fea3d700 1 -- 172.21.15.7:0/248827594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4f80ce1d0 msgr2=0x7fc4f8159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.641+0000 7fc4fea3d700 1 --2- 172.21.15.7:0/248827594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4f80ce1d0 0x7fc4f8159330 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00c4d0 tx=0x7fc4ec00c7e0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.641+0000 7fc4fea3d700 1 -- 172.21.15.7:0/248827594 shutdown_connections 2024-09-13T18:53:02.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.641+0000 7fc4fea3d700 1 --2- 172.21.15.7:0/248827594 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc4e4066730 0x7fc4e4068bf0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.641+0000 7fc4fea3d700 1 --2- 172.21.15.7:0/248827594 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc4f815ce40 0x7fc4f81484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.641+0000 7fc4fea3d700 1 --2- 172.21.15.7:0/248827594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4f80ce1d0 0x7fc4f8159330 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.641+0000 7fc4fea3d700 1 --2- 172.21.15.7:0/248827594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc4f806b400 0x7fc4f8157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.641+0000 7fc4fea3d700 1 -- 172.21.15.7:0/248827594 >> 172.21.15.7:0/248827594 conn(0x7fc4f80d2f30 msgr2=0x7fc4f815d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:02.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.641+0000 7fc4fea3d700 1 -- 172.21.15.7:0/248827594 shutdown_connections 2024-09-13T18:53:02.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.641+0000 7fc4fea3d700 1 -- 172.21.15.7:0/248827594 wait complete. 2024-09-13T18:53:02.661 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:53:02.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.773+0000 7f176f59e700 1 Processor -- start 2024-09-13T18:53:02.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.773+0000 7f176f59e700 1 -- start start 2024-09-13T18:53:02.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.773+0000 7f176f59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1768005b60 0x7f176800b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:02.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.773+0000 7f176f59e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17680e0d20 con 0x7f176800b990 2024-09-13T18:53:02.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.773+0000 7f176f59e700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17680e0110 con 0x7f17680df9c0 2024-09-13T18:53:02.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.773+0000 7f176f59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17680e03f0 con 0x7f1768005b60 2024-09-13T18:53:02.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.773+0000 7f176dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1768005b60 0x7f176800b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:02.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.773+0000 7f176dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1768005b60 0x7f176800b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59798/0 (socket says 172.21.15.7:59798) 2024-09-13T18:53:02.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.773+0000 7f176dd9b700 1 -- 172.21.15.7:0/2997881859 learned_addr learned my addr 172.21.15.7:0/2997881859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.773+0000 7f176dd9b700 1 -- 172.21.15.7:0/2997881859 >> v1:172.21.15.171:6789/0 conn(0x7f17680df9c0 legacy=0x7f176801c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.773+0000 7f176dd9b700 1 -- 172.21.15.7:0/2997881859 >> v1:172.21.15.7:6789/0 conn(0x7f176800b990 legacy=0x7f176804eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.773+0000 7f176dd9b700 1 -- 172.21.15.7:0/2997881859 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17680e06d0 con 0x7f1768005b60 2024-09-13T18:53:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.773+0000 7f176dd9b700 1 --2- 172.21.15.7:0/2997881859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1768005b60 0x7f176800b450 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f176000bbb0 tx=0x7f176000bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144857236180b138 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.773+0000 7f176d59a700 1 -- 172.21.15.7:0/2997881859 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176000dd90 con 0x7f1768005b60 2024-09-13T18:53:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.773+0000 7f176d59a700 1 -- 172.21.15.7:0/2997881859 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1760010470 con 0x7f1768005b60 2024-09-13T18:53:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.773+0000 7f176d59a700 1 -- 172.21.15.7:0/2997881859 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760010820 con 0x7f1768005b60 2024-09-13T18:53:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.777+0000 7f176f59e700 1 -- 172.21.15.7:0/2997881859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1768005b60 msgr2=0x7f176800b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.777+0000 7f176f59e700 1 --2- 172.21.15.7:0/2997881859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1768005b60 0x7f176800b450 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f176000bbb0 tx=0x7f176000bec0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.777+0000 7f176f59e700 1 -- 172.21.15.7:0/2997881859 shutdown_connections 2024-09-13T18:53:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.777+0000 7f176f59e700 1 --2- 172.21.15.7:0/2997881859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1768005b60 0x7f176800b450 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.777+0000 7f176f59e700 1 -- 172.21.15.7:0/2997881859 >> 172.21.15.7:0/2997881859 conn(0x7f1768036140 msgr2=0x7f1768036550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.777+0000 7f176f59e700 1 -- 172.21.15.7:0/2997881859 shutdown_connections 2024-09-13T18:53:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.777+0000 7f176f59e700 1 -- 172.21.15.7:0/2997881859 wait complete. 2024-09-13T18:53:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.777+0000 7f176f59e700 1 Processor -- start 2024-09-13T18:53:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.777+0000 7f176f59e700 1 -- start start 2024-09-13T18:53:02.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176f59e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1768005b60 0x7f176804dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:02.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f176800b990 0x7f17680dd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:02.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176f59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17680df9c0 0x7f17680dd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:02.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176f59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176804e320 con 0x7f176800b990 2024-09-13T18:53:02.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176f59e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17680319a0 con 0x7f1768005b60 2024-09-13T18:53:02.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176f59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17680ddfb0 con 0x7f17680df9c0 2024-09-13T18:53:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f176800b990 0x7f17680dd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f176800b990 0x7f17680dd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59800/0 (socket says 172.21.15.7:59800) 2024-09-13T18:53:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176dd9b700 1 -- 172.21.15.7:0/1943636580 learned_addr learned my addr 172.21.15.7:0/1943636580 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176e59c700 1 --2- 172.21.15.7:0/1943636580 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1768005b60 0x7f176804dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176ed9d700 1 --2- 172.21.15.7:0/1943636580 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17680df9c0 0x7f17680dd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176dd9b700 1 -- 172.21.15.7:0/1943636580 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17680df9c0 msgr2=0x7f17680dd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176dd9b700 1 --2- 172.21.15.7:0/1943636580 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17680df9c0 0x7f17680dd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176dd9b700 1 -- 172.21.15.7:0/1943636580 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1768005b60 msgr2=0x7f176804dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176dd9b700 1 --2- 172.21.15.7:0/1943636580 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1768005b60 0x7f176804dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176dd9b700 1 -- 172.21.15.7:0/1943636580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176000b810 con 0x7f176800b990 2024-09-13T18:53:02.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176dd9b700 1 --2- 172.21.15.7:0/1943636580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f176800b990 0x7f17680dd180 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1768052140 tx=0x7f1760004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:02.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f1766ffd700 1 -- 172.21.15.7:0/1943636580 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176001f070 con 0x7f176800b990 2024-09-13T18:53:02.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f1766ffd700 1 -- 172.21.15.7:0/1943636580 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17600038d0 con 0x7f176800b990 2024-09-13T18:53:02.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176f59e700 1 -- 172.21.15.7:0/1943636580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17680ddcd0 con 0x7f176800b990 2024-09-13T18:53:02.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f1766ffd700 1 -- 172.21.15.7:0/1943636580 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760003c80 con 0x7f176800b990 2024-09-13T18:53:02.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176f59e700 1 -- 172.21.15.7:0/1943636580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1768031630 con 0x7f176800b990 2024-09-13T18:53:02.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.781+0000 7f176f59e700 1 -- 172.21.15.7:0/1943636580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1768004800 con 0x7f176800b990 2024-09-13T18:53:02.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.785+0000 7f1766ffd700 1 -- 172.21.15.7:0/1943636580 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1760023410 con 0x7f176800b990 2024-09-13T18:53:02.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.785+0000 7f1766ffd700 1 --2- 172.21.15.7:0/1943636580 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1750066760 0x7f1750068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:02.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.785+0000 7f1766ffd700 1 -- 172.21.15.7:0/1943636580 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17600876b0 con 0x7f176800b990 2024-09-13T18:53:02.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.785+0000 7f176e59c700 1 --2- 172.21.15.7:0/1943636580 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1750066760 0x7f1750068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:02.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.785+0000 7f176e59c700 1 --2- 172.21.15.7:0/1943636580 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1750066760 0x7f1750068c20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1758000fb0 tx=0x7f1758009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:02.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.793+0000 7f1766ffd700 1 -- 172.21.15.7:0/1943636580 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1760054b50 con 0x7f176800b990 2024-09-13T18:53:02.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.957+0000 7f176f59e700 1 -- 172.21.15.7:0/1943636580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f17680317c0 con 0x7f176800b990 2024-09-13T18:53:02.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.957+0000 7f1766ffd700 1 -- 172.21.15.7:0/1943636580 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1760058170 con 0x7f176800b990 2024-09-13T18:53:02.962 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:53:02.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.961+0000 7f176f59e700 1 -- 172.21.15.7:0/1943636580 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1750066760 msgr2=0x7f1750068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.961+0000 7f176f59e700 1 --2- 172.21.15.7:0/1943636580 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1750066760 0x7f1750068c20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1758000fb0 tx=0x7f1758009040 comp rx=0 tx=0).stop 2024-09-13T18:53:02.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.961+0000 7f176f59e700 1 -- 172.21.15.7:0/1943636580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f176800b990 msgr2=0x7f17680dd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:02.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.961+0000 7f176f59e700 1 --2- 172.21.15.7:0/1943636580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f176800b990 0x7f17680dd180 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1768052140 tx=0x7f1760004770 comp rx=0 tx=0).stop 2024-09-13T18:53:02.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.961+0000 7f176f59e700 1 -- 172.21.15.7:0/1943636580 shutdown_connections 2024-09-13T18:53:02.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.961+0000 7f176f59e700 1 --2- 172.21.15.7:0/1943636580 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1750066760 0x7f1750068c20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.961+0000 7f176f59e700 1 --2- 172.21.15.7:0/1943636580 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17680df9c0 0x7f17680dd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.961+0000 7f176f59e700 1 --2- 172.21.15.7:0/1943636580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f176800b990 0x7f17680dd180 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.961+0000 7f176f59e700 1 --2- 172.21.15.7:0/1943636580 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1768005b60 0x7f176804dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:02.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.961+0000 7f176f59e700 1 -- 172.21.15.7:0/1943636580 >> 172.21.15.7:0/1943636580 conn(0x7f1768036140 msgr2=0x7f17680e1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:02.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.961+0000 7f176f59e700 1 -- 172.21.15.7:0/1943636580 shutdown_connections 2024-09-13T18:53:02.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:02.961+0000 7f176f59e700 1 -- 172.21.15.7:0/1943636580 wait complete. 2024-09-13T18:53:02.984 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:53:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.097+0000 7f9ae1926700 1 Processor -- start 2024-09-13T18:53:03.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.097+0000 7f9ae1926700 1 -- start start 2024-09-13T18:53:03.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.097+0000 7f9ae1926700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9adc0ce270 0x7f9adc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.097+0000 7f9ae1926700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f9adc0adbd0 0x7f9adc0b3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.097+0000 7f9ae1926700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9adc15caf0 0x7f9adc15cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.097+0000 7f9ae1926700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9adc0ce650 con 0x7f9adc0ce270 2024-09-13T18:53:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.097+0000 7f9ae1926700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9adc15d4a0 con 0x7f9adc0adbd0 2024-09-13T18:53:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.097+0000 7f9ae1926700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc0ae3c0 con 0x7f9adc15caf0 2024-09-13T18:53:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae0924700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9adc0ce270 0x7f9adc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae0924700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9adc0ce270 0x7f9adc06d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59810/0 (socket says 172.21.15.7:59810) 2024-09-13T18:53:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae1125700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9adc15caf0 0x7f9adc15cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae0924700 1 -- 172.21.15.7:0/2692736465 learned_addr learned my addr 172.21.15.7:0/2692736465 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9adbfff700 1 --2- 172.21.15.7:0/2692736465 >> v2:172.21.15.171:3300/0 conn(0x7f9adc0adbd0 0x7f9adc0b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae0924700 1 -- 172.21.15.7:0/2692736465 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9adc15caf0 msgr2=0x7f9adc15cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae0924700 1 --2- 172.21.15.7:0/2692736465 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9adc15caf0 0x7f9adc15cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae0924700 1 -- 172.21.15.7:0/2692736465 >> v2:172.21.15.171:3300/0 conn(0x7f9adc0adbd0 msgr2=0x7f9adc0b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae0924700 1 --2- 172.21.15.7:0/2692736465 >> v2:172.21.15.171:3300/0 conn(0x7f9adc0adbd0 0x7f9adc0b3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae0924700 1 -- 172.21.15.7:0/2692736465 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc06cc00 con 0x7f9adc0ce270 2024-09-13T18:53:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae0924700 1 --2- 172.21.15.7:0/2692736465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9adc0ce270 0x7f9adc06d6f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9adc15dae0 tx=0x7f9acc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243c8b90f4ec3362 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9adb7fe700 1 -- 172.21.15.7:0/2692736465 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc011830 con 0x7f9adc0ce270 2024-09-13T18:53:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9adb7fe700 1 -- 172.21.15.7:0/2692736465 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9acc011990 con 0x7f9adc0ce270 2024-09-13T18:53:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9adb7fe700 1 -- 172.21.15.7:0/2692736465 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc011d40 con 0x7f9adc0ce270 2024-09-13T18:53:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae1926700 1 -- 172.21.15.7:0/2692736465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9adc0ce270 msgr2=0x7f9adc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae1926700 1 --2- 172.21.15.7:0/2692736465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9adc0ce270 0x7f9adc06d6f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9adc15dae0 tx=0x7f9acc009c20 comp rx=0 tx=0).stop 2024-09-13T18:53:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae1926700 1 -- 172.21.15.7:0/2692736465 shutdown_connections 2024-09-13T18:53:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae1926700 1 --2- 172.21.15.7:0/2692736465 >> v2:172.21.15.171:3300/0 conn(0x7f9adc0adbd0 0x7f9adc0b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae1926700 1 --2- 172.21.15.7:0/2692736465 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9adc15caf0 0x7f9adc15cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae1926700 1 --2- 172.21.15.7:0/2692736465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9adc0ce270 0x7f9adc06d6f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae1926700 1 -- 172.21.15.7:0/2692736465 >> 172.21.15.7:0/2692736465 conn(0x7f9adc0b0e10 msgr2=0x7f9adc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae1926700 1 -- 172.21.15.7:0/2692736465 shutdown_connections 2024-09-13T18:53:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.101+0000 7f9ae1926700 1 -- 172.21.15.7:0/2692736465 wait complete. 2024-09-13T18:53:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9ae1926700 1 Processor -- start 2024-09-13T18:53:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9ae1926700 1 -- start start 2024-09-13T18:53:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9ae1926700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9adc0adbd0 0x7f9adc144080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9ae1926700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9adc0ce270 0x7f9adc1468d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9ae1926700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9adc15caf0 0x7f9adc15b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9ae1926700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc1445c0 con 0x7f9adc15caf0 2024-09-13T18:53:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9ae1926700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc144c50 con 0x7f9adc0ce270 2024-09-13T18:53:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9ae1926700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc1477e0 con 0x7f9adc0adbd0 2024-09-13T18:53:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9ae1125700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9adc15caf0 0x7f9adc15b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9ae1125700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9adc15caf0 0x7f9adc15b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59818/0 (socket says 172.21.15.7:59818) 2024-09-13T18:53:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9ae1125700 1 -- 172.21.15.7:0/2577449663 learned_addr learned my addr 172.21.15.7:0/2577449663 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9ae0924700 1 --2- 172.21.15.7:0/2577449663 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9adc0adbd0 0x7f9adc144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9adbfff700 1 --2- 172.21.15.7:0/2577449663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9adc0ce270 0x7f9adc1468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9ae1125700 1 -- 172.21.15.7:0/2577449663 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9adc0adbd0 msgr2=0x7f9adc144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9ae1125700 1 --2- 172.21.15.7:0/2577449663 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9adc0adbd0 0x7f9adc144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9ae1125700 1 -- 172.21.15.7:0/2577449663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9adc0ce270 msgr2=0x7f9adc1468d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9ae1125700 1 --2- 172.21.15.7:0/2577449663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9adc0ce270 0x7f9adc1468d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9ae1125700 1 -- 172.21.15.7:0/2577449663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc009570 con 0x7f9adc15caf0 2024-09-13T18:53:03.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.105+0000 7f9ae1125700 1 --2- 172.21.15.7:0/2577449663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9adc15caf0 0x7f9adc15b7b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400efc0 tx=0x7f9ad400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:03.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.109+0000 7f9ad97fa700 1 -- 172.21.15.7:0/2577449663 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad400cad0 con 0x7f9adc15caf0 2024-09-13T18:53:03.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.109+0000 7f9ad97fa700 1 -- 172.21.15.7:0/2577449663 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad400f040 con 0x7f9adc15caf0 2024-09-13T18:53:03.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.109+0000 7f9ae1926700 1 -- 172.21.15.7:0/2577449663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9adc147220 con 0x7f9adc15caf0 2024-09-13T18:53:03.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.109+0000 7f9ad97fa700 1 -- 172.21.15.7:0/2577449663 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad400ce70 con 0x7f9adc15caf0 2024-09-13T18:53:03.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.109+0000 7f9ae1926700 1 -- 172.21.15.7:0/2577449663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc1448e0 con 0x7f9adc15caf0 2024-09-13T18:53:03.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.109+0000 7f9ae1926700 1 -- 172.21.15.7:0/2577449663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9adc144dc0 con 0x7f9adc15caf0 2024-09-13T18:53:03.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.109+0000 7f9ad97fa700 1 -- 172.21.15.7:0/2577449663 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9ad4014480 con 0x7f9adc15caf0 2024-09-13T18:53:03.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.109+0000 7f9ad97fa700 1 --2- 172.21.15.7:0/2577449663 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9ac8066730 0x7f9ac8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.109+0000 7f9ad97fa700 1 -- 172.21.15.7:0/2577449663 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ad4084b50 con 0x7f9adc15caf0 2024-09-13T18:53:03.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.113+0000 7f9ae0924700 1 --2- 172.21.15.7:0/2577449663 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9ac8066730 0x7f9ac8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.113+0000 7f9ae0924700 1 --2- 172.21.15.7:0/2577449663 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9ac8066730 0x7f9ac8068bf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9acc000c00 tx=0x7f9acc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:03.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.117+0000 7f9ad97fa700 1 -- 172.21.15.7:0/2577449663 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9ad4051e30 con 0x7f9adc15caf0 2024-09-13T18:53:03.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.289+0000 7f9ae1926700 1 -- 172.21.15.7:0/2577449663 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9adc071370 con 0x7f9adc15caf0 2024-09-13T18:53:03.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.289+0000 7f9ad97fa700 1 -- 172.21.15.7:0/2577449663 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9ad4018020 con 0x7f9adc15caf0 2024-09-13T18:53:03.296 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:53:03.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.293+0000 7f9ae1926700 1 -- 172.21.15.7:0/2577449663 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9ac8066730 msgr2=0x7f9ac8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:03.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.293+0000 7f9ae1926700 1 --2- 172.21.15.7:0/2577449663 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9ac8066730 0x7f9ac8068bf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9acc000c00 tx=0x7f9acc008030 comp rx=0 tx=0).stop 2024-09-13T18:53:03.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.293+0000 7f9ae1926700 1 -- 172.21.15.7:0/2577449663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9adc15caf0 msgr2=0x7f9adc15b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:03.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.293+0000 7f9ae1926700 1 --2- 172.21.15.7:0/2577449663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9adc15caf0 0x7f9adc15b7b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400efc0 tx=0x7f9ad400c0c0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.293+0000 7f9ae1926700 1 -- 172.21.15.7:0/2577449663 shutdown_connections 2024-09-13T18:53:03.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.293+0000 7f9ae1926700 1 --2- 172.21.15.7:0/2577449663 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9ac8066730 0x7f9ac8068bf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.293+0000 7f9ae1926700 1 --2- 172.21.15.7:0/2577449663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9adc15caf0 0x7f9adc15b7b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.293+0000 7f9ae1926700 1 --2- 172.21.15.7:0/2577449663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9adc0ce270 0x7f9adc1468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.293+0000 7f9ae1926700 1 --2- 172.21.15.7:0/2577449663 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9adc0adbd0 0x7f9adc144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.293+0000 7f9ae1926700 1 -- 172.21.15.7:0/2577449663 >> 172.21.15.7:0/2577449663 conn(0x7f9adc0b0e10 msgr2=0x7f9adc152560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:03.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.293+0000 7f9ae1926700 1 -- 172.21.15.7:0/2577449663 shutdown_connections 2024-09-13T18:53:03.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.293+0000 7f9ae1926700 1 -- 172.21.15.7:0/2577449663 wait complete. 2024-09-13T18:53:03.315 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:53:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb023fff700 1 Processor -- start 2024-09-13T18:53:03.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb023fff700 1 -- start start 2024-09-13T18:53:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb023fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb02406d8c0 0x7fb024164a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb023fff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fb024164f90 0x7fb0240704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb023fff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb024162570 con 0x7fb024070a00 2024-09-13T18:53:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb023fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb024162e60 con 0x7fb02406d8c0 2024-09-13T18:53:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb023fff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb024165420 con 0x7fb024164f90 2024-09-13T18:53:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb022ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb02406d8c0 0x7fb024164a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb022ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb02406d8c0 0x7fb024164a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59824/0 (socket says 172.21.15.7:59824) 2024-09-13T18:53:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb022ffd700 1 -- 172.21.15.7:0/3007436836 learned_addr learned my addr 172.21.15.7:0/3007436836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb0227fc700 1 --2- 172.21.15.7:0/3007436836 >> v2:172.21.15.171:3300/0 conn(0x7fb024164f90 0x7fb0240704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb022ffd700 1 -- 172.21.15.7:0/3007436836 >> v2:172.21.15.171:3300/0 conn(0x7fb024164f90 msgr2=0x7fb0240704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb022ffd700 1 --2- 172.21.15.7:0/3007436836 >> v2:172.21.15.171:3300/0 conn(0x7fb024164f90 0x7fb0240704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb022ffd700 1 -- 172.21.15.7:0/3007436836 >> v1:172.21.15.171:6789/0 conn(0x7fb024070a00 legacy=0x7fb024161460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb022ffd700 1 -- 172.21.15.7:0/3007436836 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb024071240 con 0x7fb02406d8c0 2024-09-13T18:53:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb022ffd700 1 --2- 172.21.15.7:0/3007436836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb02406d8c0 0x7fb024164a50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb024158550 tx=0x7fb00c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73056ecb8745e78d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:03.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb021ffb700 1 -- 172.21.15.7:0/3007436836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00b8a0 con 0x7fb02406d8c0 2024-09-13T18:53:03.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb021ffb700 1 -- 172.21.15.7:0/3007436836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c00ba00 con 0x7fb02406d8c0 2024-09-13T18:53:03.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.425+0000 7fb021ffb700 1 -- 172.21.15.7:0/3007436836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00bd90 con 0x7fb02406d8c0 2024-09-13T18:53:03.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.429+0000 7fb023fff700 1 -- 172.21.15.7:0/3007436836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb02406d8c0 msgr2=0x7fb024164a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.429+0000 7fb023fff700 1 --2- 172.21.15.7:0/3007436836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb02406d8c0 0x7fb024164a50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb024158550 tx=0x7fb00c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.429+0000 7fb023fff700 1 -- 172.21.15.7:0/3007436836 shutdown_connections 2024-09-13T18:53:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.429+0000 7fb023fff700 1 --2- 172.21.15.7:0/3007436836 >> v2:172.21.15.171:3300/0 conn(0x7fb024164f90 0x7fb0240704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.429+0000 7fb023fff700 1 --2- 172.21.15.7:0/3007436836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb02406d8c0 0x7fb024164a50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.429+0000 7fb023fff700 1 -- 172.21.15.7:0/3007436836 >> 172.21.15.7:0/3007436836 conn(0x7fb0240b0e10 msgr2=0x7fb0240b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:03.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.429+0000 7fb023fff700 1 -- 172.21.15.7:0/3007436836 shutdown_connections 2024-09-13T18:53:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.429+0000 7fb023fff700 1 -- 172.21.15.7:0/3007436836 wait complete. 2024-09-13T18:53:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.429+0000 7fb023fff700 1 Processor -- start 2024-09-13T18:53:03.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.429+0000 7fb023fff700 1 -- start start 2024-09-13T18:53:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.429+0000 7fb023fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb02406d8c0 0x7fb024146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.429+0000 7fb023fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024070a00 0x7fb024156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.429+0000 7fb023fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb024164f90 0x7fb0241570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.429+0000 7fb023fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0241566e0 con 0x7fb024070a00 2024-09-13T18:53:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.429+0000 7fb023fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0241575f0 con 0x7fb02406d8c0 2024-09-13T18:53:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.429+0000 7fb023fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb024155ad0 con 0x7fb024164f90 2024-09-13T18:53:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.429+0000 7fb0227fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024070a00 0x7fb024156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb0227fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024070a00 0x7fb024156b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59832/0 (socket says 172.21.15.7:59832) 2024-09-13T18:53:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb0227fc700 1 -- 172.21.15.7:0/2486257889 learned_addr learned my addr 172.21.15.7:0/2486257889 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb022ffd700 1 --2- 172.21.15.7:0/2486257889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb02406d8c0 0x7fb024146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb0237fe700 1 --2- 172.21.15.7:0/2486257889 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb024164f90 0x7fb0241570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb0227fc700 1 -- 172.21.15.7:0/2486257889 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb024164f90 msgr2=0x7fb0241570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb0227fc700 1 --2- 172.21.15.7:0/2486257889 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb024164f90 0x7fb0241570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb0227fc700 1 -- 172.21.15.7:0/2486257889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb02406d8c0 msgr2=0x7fb024146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb0227fc700 1 --2- 172.21.15.7:0/2486257889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb02406d8c0 0x7fb024146530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb0227fc700 1 -- 172.21.15.7:0/2486257889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c009570 con 0x7fb024070a00 2024-09-13T18:53:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb0237fe700 1 --2- 172.21.15.7:0/2486257889 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb024164f90 0x7fb0241570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:03.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb022ffd700 1 --2- 172.21.15.7:0/2486257889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb02406d8c0 0x7fb024146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb0227fc700 1 --2- 172.21.15.7:0/2486257889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024070a00 0x7fb024156b70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb014007e60 tx=0x7fb01400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb01bfff700 1 -- 172.21.15.7:0/2486257889 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01400eea0 con 0x7fb024070a00 2024-09-13T18:53:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb01bfff700 1 -- 172.21.15.7:0/2486257889 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb014014420 con 0x7fb024070a00 2024-09-13T18:53:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb023fff700 1 -- 172.21.15.7:0/2486257889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb024071010 con 0x7fb024070a00 2024-09-13T18:53:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb01bfff700 1 -- 172.21.15.7:0/2486257889 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140147b0 con 0x7fb024070a00 2024-09-13T18:53:03.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb023fff700 1 -- 172.21.15.7:0/2486257889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb024156090 con 0x7fb024070a00 2024-09-13T18:53:03.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb023fff700 1 -- 172.21.15.7:0/2486257889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02406dd30 con 0x7fb024070a00 2024-09-13T18:53:03.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.433+0000 7fb01bfff700 1 -- 172.21.15.7:0/2486257889 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb014014910 con 0x7fb024070a00 2024-09-13T18:53:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.437+0000 7fb01bfff700 1 --2- 172.21.15.7:0/2486257889 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb008066730 0x7fb008068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.437+0000 7fb01bfff700 1 -- 172.21.15.7:0/2486257889 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb014084990 con 0x7fb024070a00 2024-09-13T18:53:03.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.437+0000 7fb022ffd700 1 --2- 172.21.15.7:0/2486257889 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb008066730 0x7fb008068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.437+0000 7fb022ffd700 1 --2- 172.21.15.7:0/2486257889 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb008066730 0x7fb008068bf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb00c000c00 tx=0x7fb00c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.445+0000 7fb01bfff700 1 -- 172.21.15.7:0/2486257889 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb014051c80 con 0x7fb024070a00 2024-09-13T18:53:03.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.625+0000 7fb023fff700 1 -- 172.21.15.7:0/2486257889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb0241647c0 con 0x7fb024070a00 2024-09-13T18:53:03.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.625+0000 7fb01bfff700 1 -- 172.21.15.7:0/2486257889 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb0140552a0 con 0x7fb024070a00 2024-09-13T18:53:03.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.625+0000 7fb023fff700 1 -- 172.21.15.7:0/2486257889 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb008066730 msgr2=0x7fb008068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:03.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.625+0000 7fb023fff700 1 --2- 172.21.15.7:0/2486257889 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb008066730 0x7fb008068bf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb00c000c00 tx=0x7fb00c008030 comp rx=0 tx=0).stop 2024-09-13T18:53:03.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.625+0000 7fb023fff700 1 -- 172.21.15.7:0/2486257889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024070a00 msgr2=0x7fb024156b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:03.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.625+0000 7fb023fff700 1 --2- 172.21.15.7:0/2486257889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024070a00 0x7fb024156b70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb014007e60 tx=0x7fb01400e3c0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.625+0000 7fb023fff700 1 -- 172.21.15.7:0/2486257889 shutdown_connections 2024-09-13T18:53:03.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.625+0000 7fb023fff700 1 --2- 172.21.15.7:0/2486257889 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb008066730 0x7fb008068bf0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.625+0000 7fb023fff700 1 --2- 172.21.15.7:0/2486257889 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb024164f90 0x7fb0241570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.625+0000 7fb023fff700 1 --2- 172.21.15.7:0/2486257889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb024070a00 0x7fb024156b70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.625+0000 7fb023fff700 1 --2- 172.21.15.7:0/2486257889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb02406d8c0 0x7fb024146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.625+0000 7fb023fff700 1 -- 172.21.15.7:0/2486257889 >> 172.21.15.7:0/2486257889 conn(0x7fb0240b0e10 msgr2=0x7fb024163180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:03.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.625+0000 7fb023fff700 1 -- 172.21.15.7:0/2486257889 shutdown_connections 2024-09-13T18:53:03.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.625+0000 7fb023fff700 1 -- 172.21.15.7:0/2486257889 wait complete. 2024-09-13T18:53:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-13T18:53:03.646 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi171.front.sepia.ceph.com... 2024-09-13T18:53:03.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:03.646 DEBUG:teuthology.orchestra.run.smithi171:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-09-13T18:53:03.660 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:53:03.660 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T18:53:03.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:03.661 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-13T18:53:03.669 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-13T18:53:03.669 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8720e9700 1 Processor -- start 2024-09-13T18:53:03.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8720e9700 1 -- start start 2024-09-13T18:53:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8720e9700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe86c159a50 0x7fe86c06c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8720e9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe86c06cec0 0x7fe86c159e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8720e9700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe86c06d700 con 0x7fe86c15a370 2024-09-13T18:53:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8720e9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe86c06dcc0 con 0x7fe86c06cec0 2024-09-13T18:53:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8720e9700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c06d9e0 con 0x7fe86c159a50 2024-09-13T18:53:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8708e6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe86c06cec0 0x7fe86c159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8708e6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe86c06cec0 0x7fe86c159e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59834/0 (socket says 172.21.15.7:59834) 2024-09-13T18:53:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8708e6700 1 -- 172.21.15.7:0/3082359138 learned_addr learned my addr 172.21.15.7:0/3082359138 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8710e7700 1 --2- 172.21.15.7:0/3082359138 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe86c159a50 0x7fe86c06c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8708e6700 1 -- 172.21.15.7:0/3082359138 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe86c159a50 msgr2=0x7fe86c06c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8708e6700 1 --2- 172.21.15.7:0/3082359138 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe86c159a50 0x7fe86c06c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8708e6700 1 -- 172.21.15.7:0/3082359138 >> v1:172.21.15.171:6789/0 conn(0x7fe86c15a370 legacy=0x7fe86c15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8708e6700 1 -- 172.21.15.7:0/3082359138 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c15b200 con 0x7fe86c06cec0 2024-09-13T18:53:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8708e6700 1 --2- 172.21.15.7:0/3082359138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe86c06cec0 0x7fe86c159e30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe86400d150 tx=0x7fe86400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d9d9bbeb669a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:03.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe85ffff700 1 -- 172.21.15.7:0/3082359138 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe864010070 con 0x7fe86c06cec0 2024-09-13T18:53:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe85ffff700 1 -- 172.21.15.7:0/3082359138 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86400dd80 con 0x7fe86c06cec0 2024-09-13T18:53:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe85ffff700 1 -- 172.21.15.7:0/3082359138 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86400b310 con 0x7fe86c06cec0 2024-09-13T18:53:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8720e9700 1 -- 172.21.15.7:0/3082359138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe86c06cec0 msgr2=0x7fe86c159e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:03.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8720e9700 1 --2- 172.21.15.7:0/3082359138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe86c06cec0 0x7fe86c159e30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe86400d150 tx=0x7fe86400d460 comp rx=0 tx=0).stop 2024-09-13T18:53:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8720e9700 1 -- 172.21.15.7:0/3082359138 shutdown_connections 2024-09-13T18:53:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8720e9700 1 --2- 172.21.15.7:0/3082359138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe86c06cec0 0x7fe86c159e30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8720e9700 1 --2- 172.21.15.7:0/3082359138 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe86c159a50 0x7fe86c06c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8720e9700 1 -- 172.21.15.7:0/3082359138 >> 172.21.15.7:0/3082359138 conn(0x7fe86c0b1f50 msgr2=0x7fe86c0cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8720e9700 1 -- 172.21.15.7:0/3082359138 shutdown_connections 2024-09-13T18:53:03.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.781+0000 7fe8720e9700 1 -- 172.21.15.7:0/3082359138 wait complete. 2024-09-13T18:53:03.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.785+0000 7fe8720e9700 1 Processor -- start 2024-09-13T18:53:03.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.785+0000 7fe8720e9700 1 -- start start 2024-09-13T18:53:03.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8720e9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe86c06cec0 0x7fe86c150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8720e9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe86c159a50 0x7fe86c152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8720e9700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe86c15a370 0x7fe86c151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8720e9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c152950 con 0x7fe86c159a50 2024-09-13T18:53:03.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8720e9700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c0a84b0 con 0x7fe86c06cec0 2024-09-13T18:53:03.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8720e9700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c151cd0 con 0x7fe86c15a370 2024-09-13T18:53:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8708e6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe86c159a50 0x7fe86c152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8708e6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe86c159a50 0x7fe86c152d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59850/0 (socket says 172.21.15.7:59850) 2024-09-13T18:53:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8708e6700 1 -- 172.21.15.7:0/596219505 learned_addr learned my addr 172.21.15.7:0/596219505 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8710e7700 1 --2- 172.21.15.7:0/596219505 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe86c06cec0 0x7fe86c150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8718e8700 1 --2- 172.21.15.7:0/596219505 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe86c15a370 0x7fe86c151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8708e6700 1 -- 172.21.15.7:0/596219505 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe86c15a370 msgr2=0x7fe86c151210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8708e6700 1 --2- 172.21.15.7:0/596219505 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe86c15a370 0x7fe86c151210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8708e6700 1 -- 172.21.15.7:0/596219505 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe86c06cec0 msgr2=0x7fe86c150850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8708e6700 1 --2- 172.21.15.7:0/596219505 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe86c06cec0 0x7fe86c150850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8708e6700 1 -- 172.21.15.7:0/596219505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86400bd40 con 0x7fe86c159a50 2024-09-13T18:53:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8718e8700 1 --2- 172.21.15.7:0/596219505 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe86c15a370 0x7fe86c151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:03.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8708e6700 1 --2- 172.21.15.7:0/596219505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe86c159a50 0x7fe86c152d70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe864017040 tx=0x7fe864004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:03.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe85dffb700 1 -- 172.21.15.7:0/596219505 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe864009a10 con 0x7fe86c159a50 2024-09-13T18:53:03.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe85dffb700 1 -- 172.21.15.7:0/596219505 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe864009b90 con 0x7fe86c159a50 2024-09-13T18:53:03.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8720e9700 1 -- 172.21.15.7:0/596219505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86c151fb0 con 0x7fe86c159a50 2024-09-13T18:53:03.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe85dffb700 1 -- 172.21.15.7:0/596219505 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe864004030 con 0x7fe86c159a50 2024-09-13T18:53:03.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.789+0000 7fe8720e9700 1 -- 172.21.15.7:0/596219505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe86c152570 con 0x7fe86c159a50 2024-09-13T18:53:03.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.793+0000 7fe8720e9700 1 -- 172.21.15.7:0/596219505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe86c159e30 con 0x7fe86c159a50 2024-09-13T18:53:03.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.793+0000 7fe85dffb700 1 -- 172.21.15.7:0/596219505 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe864004190 con 0x7fe86c159a50 2024-09-13T18:53:03.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.793+0000 7fe85dffb700 1 --2- 172.21.15.7:0/596219505 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe850066730 0x7fe850068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:03.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.793+0000 7fe85dffb700 1 -- 172.21.15.7:0/596219505 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8640878d0 con 0x7fe86c159a50 2024-09-13T18:53:03.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.793+0000 7fe8710e7700 1 --2- 172.21.15.7:0/596219505 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe850066730 0x7fe850068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:03.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.797+0000 7fe8710e7700 1 --2- 172.21.15.7:0/596219505 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe850066730 0x7fe850068bf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe860006b40 tx=0x7fe860008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:03.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:03.801+0000 7fe85dffb700 1 -- 172.21.15.7:0/596219505 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe864054c30 con 0x7fe86c159a50 2024-09-13T18:53:04.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.009+0000 7fe8720e9700 1 -- 172.21.15.7:0/596219505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe86c155cf0 con 0x7fe86c159a50 2024-09-13T18:53:04.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.009+0000 7fe85dffb700 1 -- 172.21.15.7:0/596219505 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4725 (secure 0 0 0) 0x7fe864019090 con 0x7fe86c159a50 2024-09-13T18:53:04.017 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:04.017 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1309419125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1309419125},{"type":"v1","addr":"172.21.15.7:6835","nonce":1309419125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/951828843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":951828843},{"type":"v1","addr":"172.21.15.7:6833","nonce":951828843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4226627580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4226627580},{"type":"v1","addr":"172.21.15.171:6835","nonce":4226627580}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:52:17.908937+0000","modified":"2024-09-13T18:52:20.930515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6837/3658296803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3658296803},{"type":"v1","addr":"172.21.15.171:6837","nonce":3658296803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:04.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.013+0000 7fe8720e9700 1 -- 172.21.15.7:0/596219505 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe850066730 msgr2=0x7fe850068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:04.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.013+0000 7fe8720e9700 1 --2- 172.21.15.7:0/596219505 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe850066730 0x7fe850068bf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe860006b40 tx=0x7fe860008040 comp rx=0 tx=0).stop 2024-09-13T18:53:04.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.013+0000 7fe8720e9700 1 -- 172.21.15.7:0/596219505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe86c159a50 msgr2=0x7fe86c152d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:04.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.013+0000 7fe8720e9700 1 --2- 172.21.15.7:0/596219505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe86c159a50 0x7fe86c152d70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe864017040 tx=0x7fe864004af0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.013+0000 7fe8720e9700 1 -- 172.21.15.7:0/596219505 shutdown_connections 2024-09-13T18:53:04.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.013+0000 7fe8720e9700 1 --2- 172.21.15.7:0/596219505 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe850066730 0x7fe850068bf0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.013+0000 7fe8720e9700 1 --2- 172.21.15.7:0/596219505 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe86c15a370 0x7fe86c151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.013+0000 7fe8720e9700 1 --2- 172.21.15.7:0/596219505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe86c159a50 0x7fe86c152d70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.013+0000 7fe8720e9700 1 --2- 172.21.15.7:0/596219505 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe86c06cec0 0x7fe86c150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.013+0000 7fe8720e9700 1 -- 172.21.15.7:0/596219505 >> 172.21.15.7:0/596219505 conn(0x7fe86c0b1f50 msgr2=0x7fe86c15b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:04.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.013+0000 7fe8720e9700 1 -- 172.21.15.7:0/596219505 shutdown_connections 2024-09-13T18:53:04.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.013+0000 7fe8720e9700 1 -- 172.21.15.7:0/596219505 wait complete. 2024-09-13T18:53:04.021 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-13T18:53:04.033 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:53:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.141+0000 7fe21b47a700 1 Processor -- start 2024-09-13T18:53:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.141+0000 7fe21b47a700 1 -- start start 2024-09-13T18:53:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe21b47a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe21406fb30 0x7fe2140adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe21b47a700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe21406bb90 0x7fe2140cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe21b47a700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe21406a780 con 0x7fe2140b3380 2024-09-13T18:53:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe21b47a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe21406cd30 con 0x7fe21406fb30 2024-09-13T18:53:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe21b47a700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe214159470 con 0x7fe21406bb90 2024-09-13T18:53:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe21a478700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe21406fb30 0x7fe2140adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe21a478700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe21406fb30 0x7fe2140adbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59858/0 (socket says 172.21.15.7:59858) 2024-09-13T18:53:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe219c77700 1 --2- 172.21.15.7:0/2592090246 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe21406bb90 0x7fe2140cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe21a478700 1 -- 172.21.15.7:0/2592090246 learned_addr learned my addr 172.21.15.7:0/2592090246 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe219c77700 1 -- 172.21.15.7:0/2592090246 >> v2:172.21.15.7:3300/0 conn(0x7fe21406fb30 msgr2=0x7fe2140adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe219c77700 1 --2- 172.21.15.7:0/2592090246 >> v2:172.21.15.7:3300/0 conn(0x7fe21406fb30 0x7fe2140adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe219c77700 1 -- 172.21.15.7:0/2592090246 >> v1:172.21.15.171:6789/0 conn(0x7fe2140b3380 legacy=0x7fe21406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe219c77700 1 -- 172.21.15.7:0/2592090246 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21406c0b0 con 0x7fe21406bb90 2024-09-13T18:53:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe21a478700 1 --2- 172.21.15.7:0/2592090246 >> v2:172.21.15.7:3300/0 conn(0x7fe21406fb30 0x7fe2140adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe219476700 1 -- 172.21.15.7:0/2592090246 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 755571251 0 0) 0x7fe21406a780 con 0x7fe2140b3380 2024-09-13T18:53:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe219c77700 1 --2- 172.21.15.7:0/2592090246 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe21406bb90 0x7fe2140cbf00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe21000baf0 tx=0x7fe21000be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61023fe4b15a114d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe219476700 1 -- 172.21.15.7:0/2592090246 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21000dc70 con 0x7fe21406bb90 2024-09-13T18:53:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe219476700 1 -- 172.21.15.7:0/2592090246 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21000ddd0 con 0x7fe21406bb90 2024-09-13T18:53:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe219476700 1 -- 172.21.15.7:0/2592090246 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210010690 con 0x7fe21406bb90 2024-09-13T18:53:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe21b47a700 1 -- 172.21.15.7:0/2592090246 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe21406bb90 msgr2=0x7fe2140cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe21b47a700 1 --2- 172.21.15.7:0/2592090246 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe21406bb90 0x7fe2140cbf00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe21000baf0 tx=0x7fe21000be00 comp rx=0 tx=0).stop 2024-09-13T18:53:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe21b47a700 1 -- 172.21.15.7:0/2592090246 shutdown_connections 2024-09-13T18:53:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe21b47a700 1 --2- 172.21.15.7:0/2592090246 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe21406bb90 0x7fe2140cbf00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe21b47a700 1 --2- 172.21.15.7:0/2592090246 >> v2:172.21.15.7:3300/0 conn(0x7fe21406fb30 0x7fe2140adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe21b47a700 1 -- 172.21.15.7:0/2592090246 >> 172.21.15.7:0/2592090246 conn(0x7fe2140b0f00 msgr2=0x7fe2140b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe21b47a700 1 -- 172.21.15.7:0/2592090246 shutdown_connections 2024-09-13T18:53:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.145+0000 7fe21b47a700 1 -- 172.21.15.7:0/2592090246 wait complete. 2024-09-13T18:53:04.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe21b47a700 1 Processor -- start 2024-09-13T18:53:04.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe21b47a700 1 -- start start 2024-09-13T18:53:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe21b47a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe21406bb90 0x7fe214142240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:04.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe21b47a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21406fb30 0x7fe21415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe21b47a700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe2140b3380 0x7fe2140ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe21b47a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21415c590 con 0x7fe21406fb30 2024-09-13T18:53:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe21b47a700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe214159d90 con 0x7fe21406bb90 2024-09-13T18:53:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe21b47a700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21414f910 con 0x7fe2140b3380 2024-09-13T18:53:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe219c77700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21406fb30 0x7fe21415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe219c77700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21406fb30 0x7fe21415bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59870/0 (socket says 172.21.15.7:59870) 2024-09-13T18:53:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe219c77700 1 -- 172.21.15.7:0/2573105637 learned_addr learned my addr 172.21.15.7:0/2573105637 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe21a478700 1 --2- 172.21.15.7:0/2573105637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe21406bb90 0x7fe214142240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe21ac79700 1 --2- 172.21.15.7:0/2573105637 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe2140b3380 0x7fe2140ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe219c77700 1 -- 172.21.15.7:0/2573105637 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe2140b3380 msgr2=0x7fe2140ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe219c77700 1 --2- 172.21.15.7:0/2573105637 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe2140b3380 0x7fe2140ccf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe219c77700 1 -- 172.21.15.7:0/2573105637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe21406bb90 msgr2=0x7fe214142240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:04.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe219c77700 1 --2- 172.21.15.7:0/2573105637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe21406bb90 0x7fe214142240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe219c77700 1 -- 172.21.15.7:0/2573105637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe204009570 con 0x7fe21406fb30 2024-09-13T18:53:04.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe219c77700 1 --2- 172.21.15.7:0/2573105637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21406fb30 0x7fe21415bdb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe21000b2c0 tx=0x7fe210004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:04.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.149+0000 7fe20affd700 1 -- 172.21.15.7:0/2573105637 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21001f070 con 0x7fe21406fb30 2024-09-13T18:53:04.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.153+0000 7fe20affd700 1 -- 172.21.15.7:0/2573105637 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe210003990 con 0x7fe21406fb30 2024-09-13T18:53:04.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.153+0000 7fe21b47a700 1 -- 172.21.15.7:0/2573105637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21000b750 con 0x7fe21406fb30 2024-09-13T18:53:04.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.153+0000 7fe20affd700 1 -- 172.21.15.7:0/2573105637 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210003d60 con 0x7fe21406fb30 2024-09-13T18:53:04.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.153+0000 7fe21b47a700 1 -- 172.21.15.7:0/2573105637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2141597d0 con 0x7fe21406fb30 2024-09-13T18:53:04.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.153+0000 7fe21b47a700 1 -- 172.21.15.7:0/2573105637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2140b37b0 con 0x7fe21406fb30 2024-09-13T18:53:04.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.153+0000 7fe20affd700 1 -- 172.21.15.7:0/2573105637 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe210019400 con 0x7fe21406fb30 2024-09-13T18:53:04.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.153+0000 7fe20affd700 1 --2- 172.21.15.7:0/2573105637 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe1f8066730 0x7fe1f8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:04.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.153+0000 7fe20affd700 1 -- 172.21.15.7:0/2573105637 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe210087680 con 0x7fe21406fb30 2024-09-13T18:53:04.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.153+0000 7fe21a478700 1 --2- 172.21.15.7:0/2573105637 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe1f8066730 0x7fe1f8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:04.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.157+0000 7fe21a478700 1 --2- 172.21.15.7:0/2573105637 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe1f8066730 0x7fe1f8068bf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe204006d50 tx=0x7fe2040092e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.161+0000 7fe20affd700 1 -- 172.21.15.7:0/2573105637 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe210054b20 con 0x7fe21406fb30 2024-09-13T18:53:04.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.341+0000 7fe21b47a700 1 -- 172.21.15.7:0/2573105637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2140711e0 con 0x7fe21406fb30 2024-09-13T18:53:04.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.341+0000 7fe20affd700 1 -- 172.21.15.7:0/2573105637 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7fe210058140 con 0x7fe21406fb30 2024-09-13T18:53:04.349 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:04.349 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:53:04.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.345+0000 7fe21b47a700 1 -- 172.21.15.7:0/2573105637 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe1f8066730 msgr2=0x7fe1f8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:04.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.345+0000 7fe21b47a700 1 --2- 172.21.15.7:0/2573105637 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe1f8066730 0x7fe1f8068bf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe204006d50 tx=0x7fe2040092e0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.345+0000 7fe21b47a700 1 -- 172.21.15.7:0/2573105637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21406fb30 msgr2=0x7fe21415bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:04.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.345+0000 7fe21b47a700 1 --2- 172.21.15.7:0/2573105637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21406fb30 0x7fe21415bdb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe21000b2c0 tx=0x7fe210004770 comp rx=0 tx=0).stop 2024-09-13T18:53:04.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.345+0000 7fe21b47a700 1 -- 172.21.15.7:0/2573105637 shutdown_connections 2024-09-13T18:53:04.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.345+0000 7fe21b47a700 1 --2- 172.21.15.7:0/2573105637 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe1f8066730 0x7fe1f8068bf0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.345+0000 7fe21b47a700 1 --2- 172.21.15.7:0/2573105637 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe2140b3380 0x7fe2140ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.345+0000 7fe21b47a700 1 --2- 172.21.15.7:0/2573105637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe21406fb30 0x7fe21415bdb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.345+0000 7fe21b47a700 1 --2- 172.21.15.7:0/2573105637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe21406bb90 0x7fe214142240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.345+0000 7fe21b47a700 1 -- 172.21.15.7:0/2573105637 >> 172.21.15.7:0/2573105637 conn(0x7fe2140b0f00 msgr2=0x7fe214158700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:04.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.345+0000 7fe21b47a700 1 -- 172.21.15.7:0/2573105637 shutdown_connections 2024-09-13T18:53:04.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.345+0000 7fe21b47a700 1 -- 172.21.15.7:0/2573105637 wait complete. 2024-09-13T18:53:04.365 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:52:15.554683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:52:16.896879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:53:04.366 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:53:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.473+0000 7f8191d95700 1 Processor -- start 2024-09-13T18:53:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.473+0000 7f8191d95700 1 -- start start 2024-09-13T18:53:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8191d95700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f818c160740 0x7f818c160b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8191d95700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f818c159e00 0x7f818c15a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8191d95700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818c0ae5f0 con 0x7f818c0ad870 2024-09-13T18:53:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8191d95700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818c15ede0 con 0x7f818c160740 2024-09-13T18:53:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8191d95700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818c0ae410 con 0x7f818c159e00 2024-09-13T18:53:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8190d93700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f818c160740 0x7f818c160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8190d93700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f818c160740 0x7f818c160b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59878/0 (socket says 172.21.15.7:59878) 2024-09-13T18:53:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8190d93700 1 -- 172.21.15.7:0/2725663281 learned_addr learned my addr 172.21.15.7:0/2725663281 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:04.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8191594700 1 --2- 172.21.15.7:0/2725663281 >> v2:172.21.15.171:3300/0 conn(0x7f818c159e00 0x7f818c15a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8190d93700 1 -- 172.21.15.7:0/2725663281 >> v2:172.21.15.171:3300/0 conn(0x7f818c159e00 msgr2=0x7f818c15a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8190d93700 1 --2- 172.21.15.7:0/2725663281 >> v2:172.21.15.171:3300/0 conn(0x7f818c159e00 0x7f818c15a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8190d93700 1 -- 172.21.15.7:0/2725663281 >> v1:172.21.15.171:6789/0 conn(0x7f818c0ad870 legacy=0x7f818c0add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8190d93700 1 -- 172.21.15.7:0/2725663281 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c161060 con 0x7f818c160740 2024-09-13T18:53:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8190d93700 1 --2- 172.21.15.7:0/2725663281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f818c160740 0x7f818c160b20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f818c0b5470 tx=0x7f8174009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec902da629155f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f818b7fe700 1 -- 172.21.15.7:0/2725663281 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174011830 con 0x7f818c160740 2024-09-13T18:53:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f818b7fe700 1 -- 172.21.15.7:0/2725663281 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8174011990 con 0x7f818c160740 2024-09-13T18:53:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f818b7fe700 1 -- 172.21.15.7:0/2725663281 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174011d20 con 0x7f818c160740 2024-09-13T18:53:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8191d95700 1 -- 172.21.15.7:0/2725663281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f818c160740 msgr2=0x7f818c160b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8191d95700 1 --2- 172.21.15.7:0/2725663281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f818c160740 0x7f818c160b20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f818c0b5470 tx=0x7f8174009bf0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8191d95700 1 -- 172.21.15.7:0/2725663281 shutdown_connections 2024-09-13T18:53:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8191d95700 1 --2- 172.21.15.7:0/2725663281 >> v2:172.21.15.171:3300/0 conn(0x7f818c159e00 0x7f818c15a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8191d95700 1 --2- 172.21.15.7:0/2725663281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f818c160740 0x7f818c160b20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.477+0000 7f8191d95700 1 -- 172.21.15.7:0/2725663281 >> 172.21.15.7:0/2725663281 conn(0x7f818c0b1cd0 msgr2=0x7f818c0cf910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8191d95700 1 -- 172.21.15.7:0/2725663281 shutdown_connections 2024-09-13T18:53:04.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8191d95700 1 -- 172.21.15.7:0/2725663281 wait complete. 2024-09-13T18:53:04.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8191d95700 1 Processor -- start 2024-09-13T18:53:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8191d95700 1 -- start start 2024-09-13T18:53:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8191d95700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f818c0ad870 0x7f818c158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8191d95700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f818c159e00 0x7f818c06b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8191d95700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f818c160740 0x7f818c15af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8191d95700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c158ba0 con 0x7f818c159e00 2024-09-13T18:53:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8191d95700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c06cea0 con 0x7f818c0ad870 2024-09-13T18:53:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8191d95700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c06be00 con 0x7f818c160740 2024-09-13T18:53:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f818bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f818c159e00 0x7f818c06b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8190d93700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f818c0ad870 0x7f818c158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f818bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f818c159e00 0x7f818c06b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59882/0 (socket says 172.21.15.7:59882) 2024-09-13T18:53:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f818bfff700 1 -- 172.21.15.7:0/3446256958 learned_addr learned my addr 172.21.15.7:0/3446256958 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:04.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8191594700 1 --2- 172.21.15.7:0/3446256958 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f818c160740 0x7f818c15af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8190d93700 1 -- 172.21.15.7:0/3446256958 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f818c160740 msgr2=0x7f818c15af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8190d93700 1 --2- 172.21.15.7:0/3446256958 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f818c160740 0x7f818c15af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8190d93700 1 -- 172.21.15.7:0/3446256958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f818c159e00 msgr2=0x7f818c06b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8190d93700 1 --2- 172.21.15.7:0/3446256958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f818c159e00 0x7f818c06b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8190d93700 1 -- 172.21.15.7:0/3446256958 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8174009570 con 0x7f818c0ad870 2024-09-13T18:53:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8190d93700 1 --2- 172.21.15.7:0/3446256958 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f818c0ad870 0x7f818c158660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8174016000 tx=0x7f8174004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:04.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f81897fa700 1 -- 172.21.15.7:0/3446256958 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174004450 con 0x7f818c0ad870 2024-09-13T18:53:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f81897fa700 1 -- 172.21.15.7:0/3446256958 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817401e070 con 0x7f818c0ad870 2024-09-13T18:53:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f8191d95700 1 -- 172.21.15.7:0/3446256958 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818c06c0e0 con 0x7f818c0ad870 2024-09-13T18:53:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.481+0000 7f81897fa700 1 -- 172.21.15.7:0/3446256958 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174003900 con 0x7f818c0ad870 2024-09-13T18:53:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.485+0000 7f8191d95700 1 -- 172.21.15.7:0/3446256958 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818c06c6a0 con 0x7f818c0ad870 2024-09-13T18:53:04.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.485+0000 7f8191d95700 1 -- 172.21.15.7:0/3446256958 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818c15ab00 con 0x7f818c0ad870 2024-09-13T18:53:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.485+0000 7f81897fa700 1 -- 172.21.15.7:0/3446256958 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8174003a60 con 0x7f818c0ad870 2024-09-13T18:53:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.485+0000 7f81897fa700 1 --2- 172.21.15.7:0/3446256958 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8170066730 0x7f8170068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:04.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.485+0000 7f81897fa700 1 -- 172.21.15.7:0/3446256958 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8174084f90 con 0x7f818c0ad870 2024-09-13T18:53:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.485+0000 7f818bfff700 1 --2- 172.21.15.7:0/3446256958 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8170066730 0x7f8170068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:04.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.489+0000 7f818bfff700 1 --2- 172.21.15.7:0/3446256958 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8170066730 0x7f8170068bf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f818c0cbb60 tx=0x7f817c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:04.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.497+0000 7f81897fa700 1 -- 172.21.15.7:0/3446256958 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f81740b6db0 con 0x7f818c0ad870 2024-09-13T18:53:04.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.669+0000 7f8191d95700 1 -- 172.21.15.7:0/3446256958 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f818c0b4fa0 con 0x7f818c0ad870 2024-09-13T18:53:04.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.673+0000 7f81897fa700 1 -- 172.21.15.7:0/3446256958 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f817405ae60 con 0x7f818c0ad870 2024-09-13T18:53:04.678 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:53:04.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.673+0000 7f8191d95700 1 -- 172.21.15.7:0/3446256958 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8170066730 msgr2=0x7f8170068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:04.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.673+0000 7f8191d95700 1 --2- 172.21.15.7:0/3446256958 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8170066730 0x7f8170068bf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f818c0cbb60 tx=0x7f817c009040 comp rx=0 tx=0).stop 2024-09-13T18:53:04.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.673+0000 7f8191d95700 1 -- 172.21.15.7:0/3446256958 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f818c0ad870 msgr2=0x7f818c158660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:04.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.673+0000 7f8191d95700 1 --2- 172.21.15.7:0/3446256958 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f818c0ad870 0x7f818c158660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8174016000 tx=0x7f8174004770 comp rx=0 tx=0).stop 2024-09-13T18:53:04.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.673+0000 7f8191d95700 1 -- 172.21.15.7:0/3446256958 shutdown_connections 2024-09-13T18:53:04.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.673+0000 7f8191d95700 1 --2- 172.21.15.7:0/3446256958 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8170066730 0x7f8170068bf0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.673+0000 7f8191d95700 1 --2- 172.21.15.7:0/3446256958 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f818c160740 0x7f818c15af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.673+0000 7f8191d95700 1 --2- 172.21.15.7:0/3446256958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f818c159e00 0x7f818c06b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.673+0000 7f8191d95700 1 --2- 172.21.15.7:0/3446256958 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f818c0ad870 0x7f818c158660 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.673+0000 7f8191d95700 1 -- 172.21.15.7:0/3446256958 >> 172.21.15.7:0/3446256958 conn(0x7f818c0b1cd0 msgr2=0x7f818c0b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:04.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.673+0000 7f8191d95700 1 -- 172.21.15.7:0/3446256958 shutdown_connections 2024-09-13T18:53:04.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.673+0000 7f8191d95700 1 -- 172.21.15.7:0/3446256958 wait complete. 2024-09-13T18:53:04.695 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:53:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.801+0000 7f14f441f700 1 Processor -- start 2024-09-13T18:53:04.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.801+0000 7f14f441f700 1 -- start start 2024-09-13T18:53:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.801+0000 7f14f441f700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14ec0cd0e0 0x7f14ec15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.801+0000 7f14f441f700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f14ec0cdf20 0x7f14ec0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.801+0000 7f14f441f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ec0cda90 con 0x7f14ec1599d0 2024-09-13T18:53:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f441f700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ec15afe0 con 0x7f14ec0cdf20 2024-09-13T18:53:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f441f700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec15da40 con 0x7f14ec0cd0e0 2024-09-13T18:53:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f21bb700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f14ec1599d0 0x7f14ec0cc9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47576/0 (socket says 172.21.15.7:47576) 2024-09-13T18:53:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f21bb700 1 -- 172.21.15.7:0/2307558924 learned_addr learned my addr 172.21.15.7:0/2307558924 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f19ba700 1 --2- 172.21.15.7:0/2307558924 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14ec0cd0e0 0x7f14ec15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f29bc700 1 --2- 172.21.15.7:0/2307558924 >> v2:172.21.15.171:3300/0 conn(0x7f14ec0cdf20 0x7f14ec0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:04.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f11b9700 1 -- 172.21.15.7:0/2307558924 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764760437 0 0) 0x7f14ec0cda90 con 0x7f14ec1599d0 2024-09-13T18:53:04.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f11b9700 1 -- 172.21.15.7:0/2307558924 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14d80035d0 con 0x7f14ec1599d0 2024-09-13T18:53:04.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f19ba700 1 -- 172.21.15.7:0/2307558924 >> v2:172.21.15.171:3300/0 conn(0x7f14ec0cdf20 msgr2=0x7f14ec0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:04.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f19ba700 1 --2- 172.21.15.7:0/2307558924 >> v2:172.21.15.171:3300/0 conn(0x7f14ec0cdf20 0x7f14ec0ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f19ba700 1 -- 172.21.15.7:0/2307558924 >> v1:172.21.15.7:6789/0 conn(0x7f14ec1599d0 legacy=0x7f14ec0cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:04.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f19ba700 1 -- 172.21.15.7:0/2307558924 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec15dd20 con 0x7f14ec0cd0e0 2024-09-13T18:53:04.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f29bc700 1 --2- 172.21.15.7:0/2307558924 >> v2:172.21.15.171:3300/0 conn(0x7f14ec0cdf20 0x7f14ec0ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f19ba700 1 --2- 172.21.15.7:0/2307558924 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14ec0cd0e0 0x7f14ec15aaa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009850 tx=0x7f14dc009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35980d894ce20085 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f11b9700 1 -- 172.21.15.7:0/2307558924 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc011830 con 0x7f14ec0cd0e0 2024-09-13T18:53:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f11b9700 1 -- 172.21.15.7:0/2307558924 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc011990 con 0x7f14ec0cd0e0 2024-09-13T18:53:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f11b9700 1 -- 172.21.15.7:0/2307558924 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc011d00 con 0x7f14ec0cd0e0 2024-09-13T18:53:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f441f700 1 -- 172.21.15.7:0/2307558924 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14ec0cd0e0 msgr2=0x7f14ec15aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f441f700 1 --2- 172.21.15.7:0/2307558924 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14ec0cd0e0 0x7f14ec15aaa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f14dc009850 tx=0x7f14dc009b60 comp rx=0 tx=0).stop 2024-09-13T18:53:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f441f700 1 -- 172.21.15.7:0/2307558924 shutdown_connections 2024-09-13T18:53:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f441f700 1 --2- 172.21.15.7:0/2307558924 >> v2:172.21.15.171:3300/0 conn(0x7f14ec0cdf20 0x7f14ec0ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f441f700 1 --2- 172.21.15.7:0/2307558924 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14ec0cd0e0 0x7f14ec15aaa0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f441f700 1 -- 172.21.15.7:0/2307558924 >> 172.21.15.7:0/2307558924 conn(0x7f14ec0b1b00 msgr2=0x7f14ec0b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f441f700 1 -- 172.21.15.7:0/2307558924 shutdown_connections 2024-09-13T18:53:04.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.805+0000 7f14f441f700 1 -- 172.21.15.7:0/2307558924 wait complete. 2024-09-13T18:53:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f441f700 1 Processor -- start 2024-09-13T18:53:04.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f441f700 1 -- start start 2024-09-13T18:53:04.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f441f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14ec0cd0e0 0x7f14ec153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:04.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f441f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14ec0cdf20 0x7f14ec06dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:04.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f441f700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14ec1599d0 0x7f14ec145280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f441f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec06cd00 con 0x7f14ec0cd0e0 2024-09-13T18:53:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f441f700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec06cfe0 con 0x7f14ec0cdf20 2024-09-13T18:53:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f441f700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec06d2c0 con 0x7f14ec1599d0 2024-09-13T18:53:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f21bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14ec0cd0e0 0x7f14ec153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f21bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14ec0cd0e0 0x7f14ec153c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59898/0 (socket says 172.21.15.7:59898) 2024-09-13T18:53:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f21bb700 1 -- 172.21.15.7:0/2117908993 learned_addr learned my addr 172.21.15.7:0/2117908993 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f29bc700 1 --2- 172.21.15.7:0/2117908993 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14ec1599d0 0x7f14ec145280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f19ba700 1 --2- 172.21.15.7:0/2117908993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14ec0cdf20 0x7f14ec06dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f21bb700 1 -- 172.21.15.7:0/2117908993 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14ec1599d0 msgr2=0x7f14ec145280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f21bb700 1 --2- 172.21.15.7:0/2117908993 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14ec1599d0 0x7f14ec145280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f21bb700 1 -- 172.21.15.7:0/2117908993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14ec0cdf20 msgr2=0x7f14ec06dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f21bb700 1 --2- 172.21.15.7:0/2117908993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14ec0cdf20 0x7f14ec06dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:04.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f21bb700 1 -- 172.21.15.7:0/2117908993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e4009a70 con 0x7f14ec0cd0e0 2024-09-13T18:53:04.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f19ba700 1 --2- 172.21.15.7:0/2117908993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14ec0cdf20 0x7f14ec06dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:04.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f21bb700 1 --2- 172.21.15.7:0/2117908993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14ec0cd0e0 0x7f14ec153c90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f14e800c4d0 tx=0x7f14e800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:04.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14e2ffd700 1 -- 172.21.15.7:0/2117908993 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8005d90 con 0x7f14ec0cd0e0 2024-09-13T18:53:04.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f441f700 1 -- 172.21.15.7:0/2117908993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14dc0094b0 con 0x7f14ec0cd0e0 2024-09-13T18:53:04.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14e2ffd700 1 -- 172.21.15.7:0/2117908993 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e80070d0 con 0x7f14ec0cd0e0 2024-09-13T18:53:04.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14e2ffd700 1 -- 172.21.15.7:0/2117908993 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8007440 con 0x7f14ec0cd0e0 2024-09-13T18:53:04.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.809+0000 7f14f441f700 1 -- 172.21.15.7:0/2117908993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ec06d880 con 0x7f14ec0cd0e0 2024-09-13T18:53:04.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.813+0000 7f14f441f700 1 -- 172.21.15.7:0/2117908993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ec15d5f0 con 0x7f14ec0cd0e0 2024-09-13T18:53:04.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.813+0000 7f14e2ffd700 1 -- 172.21.15.7:0/2117908993 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f14e80075a0 con 0x7f14ec0cd0e0 2024-09-13T18:53:04.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.813+0000 7f14e2ffd700 1 --2- 172.21.15.7:0/2117908993 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f14d0066730 0x7f14d0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:04.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.813+0000 7f14e2ffd700 1 -- 172.21.15.7:0/2117908993 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14e8083510 con 0x7f14ec0cd0e0 2024-09-13T18:53:04.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.813+0000 7f14f19ba700 1 --2- 172.21.15.7:0/2117908993 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f14d0066730 0x7f14d0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:04.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.813+0000 7f14f19ba700 1 --2- 172.21.15.7:0/2117908993 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f14d0066730 0x7f14d0068bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0b4ad0 tx=0x7f14dc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.821+0000 7f14e2ffd700 1 -- 172.21.15.7:0/2117908993 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f14e8050810 con 0x7f14ec0cd0e0 2024-09-13T18:53:05.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.997+0000 7f14f441f700 1 -- 172.21.15.7:0/2117908993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f14ec15c330 con 0x7f14ec0cd0e0 2024-09-13T18:53:05.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:04.997+0000 7f14e2ffd700 1 -- 172.21.15.7:0/2117908993 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f14e805a070 con 0x7f14ec0cd0e0 2024-09-13T18:53:05.004 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:53:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.001+0000 7f14f441f700 1 -- 172.21.15.7:0/2117908993 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f14d0066730 msgr2=0x7f14d0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.001+0000 7f14f441f700 1 --2- 172.21.15.7:0/2117908993 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f14d0066730 0x7f14d0068bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0b4ad0 tx=0x7f14dc008030 comp rx=0 tx=0).stop 2024-09-13T18:53:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.001+0000 7f14f441f700 1 -- 172.21.15.7:0/2117908993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14ec0cd0e0 msgr2=0x7f14ec153c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.001+0000 7f14f441f700 1 --2- 172.21.15.7:0/2117908993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14ec0cd0e0 0x7f14ec153c90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f14e800c4d0 tx=0x7f14e800c890 comp rx=0 tx=0).stop 2024-09-13T18:53:05.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.001+0000 7f14f441f700 1 -- 172.21.15.7:0/2117908993 shutdown_connections 2024-09-13T18:53:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.001+0000 7f14f441f700 1 --2- 172.21.15.7:0/2117908993 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f14d0066730 0x7f14d0068bf0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.001+0000 7f14f441f700 1 --2- 172.21.15.7:0/2117908993 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14ec1599d0 0x7f14ec145280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.001+0000 7f14f441f700 1 --2- 172.21.15.7:0/2117908993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14ec0cdf20 0x7f14ec06dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.001+0000 7f14f441f700 1 --2- 172.21.15.7:0/2117908993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14ec0cd0e0 0x7f14ec153c90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.001+0000 7f14f441f700 1 -- 172.21.15.7:0/2117908993 >> 172.21.15.7:0/2117908993 conn(0x7f14ec0b1b00 msgr2=0x7f14ec159f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.001+0000 7f14f441f700 1 -- 172.21.15.7:0/2117908993 shutdown_connections 2024-09-13T18:53:05.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.001+0000 7f14f441f700 1 -- 172.21.15.7:0/2117908993 wait complete. 2024-09-13T18:53:05.018 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:53:05.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26e194b700 1 Processor -- start 2024-09-13T18:53:05.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26e194b700 1 -- start start 2024-09-13T18:53:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26e194b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f26dc15cda0 0x7f26dc0d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26e194b700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f26dc0b5e00 0x7f26dc0b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26e194b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26dc0b6780 0x7f26dc06a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26e194b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26dc0cde50 con 0x7f26dc0b6780 2024-09-13T18:53:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26e194b700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26dc0d0880 con 0x7f26dc0b5e00 2024-09-13T18:53:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26e194b700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc0d16e0 con 0x7f26dc15cda0 2024-09-13T18:53:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26db7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26dc0b6780 0x7f26dc06a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26db7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26dc0b6780 0x7f26dc06a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59902/0 (socket says 172.21.15.7:59902) 2024-09-13T18:53:05.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26db7fe700 1 -- 172.21.15.7:0/2693594586 learned_addr learned my addr 172.21.15.7:0/2693594586 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:05.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26d27fc700 1 --2- 172.21.15.7:0/2693594586 >> v2:172.21.15.171:3300/0 conn(0x7f26dc0b5e00 0x7f26dc0b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26daffd700 1 --2- 172.21.15.7:0/2693594586 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f26dc15cda0 0x7f26dc0d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26db7fe700 1 -- 172.21.15.7:0/2693594586 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f26dc15cda0 msgr2=0x7f26dc0d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:05.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26db7fe700 1 --2- 172.21.15.7:0/2693594586 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f26dc15cda0 0x7f26dc0d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26db7fe700 1 -- 172.21.15.7:0/2693594586 >> v2:172.21.15.171:3300/0 conn(0x7f26dc0b5e00 msgr2=0x7f26dc0b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:05.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26db7fe700 1 --2- 172.21.15.7:0/2693594586 >> v2:172.21.15.171:3300/0 conn(0x7f26dc0b5e00 0x7f26dc0b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26db7fe700 1 -- 172.21.15.7:0/2693594586 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26dc0d0ad0 con 0x7f26dc0b6780 2024-09-13T18:53:05.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26daffd700 1 --2- 172.21.15.7:0/2693594586 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f26dc15cda0 0x7f26dc0d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:05.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.137+0000 7f26db7fe700 1 --2- 172.21.15.7:0/2693594586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc0b6780 0x7f26dc06a5b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009e50 tx=0x7f26d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6879c8aa84e4f432 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:05.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.141+0000 7f26da7fc700 1 -- 172.21.15.7:0/2693594586 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d400e070 con 0x7f26dc0b6780 2024-09-13T18:53:05.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.141+0000 7f26da7fc700 1 -- 172.21.15.7:0/2693594586 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d400b8d0 con 0x7f26dc0b6780 2024-09-13T18:53:05.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.141+0000 7f26da7fc700 1 -- 172.21.15.7:0/2693594586 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d400bc80 con 0x7f26dc0b6780 2024-09-13T18:53:05.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.141+0000 7f26e194b700 1 -- 172.21.15.7:0/2693594586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc0b6780 msgr2=0x7f26dc06a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.141+0000 7f26e194b700 1 --2- 172.21.15.7:0/2693594586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc0b6780 0x7f26dc06a5b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f26d4009e50 tx=0x7f26d400b330 comp rx=0 tx=0).stop 2024-09-13T18:53:05.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.141+0000 7f26e194b700 1 -- 172.21.15.7:0/2693594586 shutdown_connections 2024-09-13T18:53:05.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.141+0000 7f26e194b700 1 --2- 172.21.15.7:0/2693594586 >> v2:172.21.15.171:3300/0 conn(0x7f26dc0b5e00 0x7f26dc0b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.141+0000 7f26e194b700 1 --2- 172.21.15.7:0/2693594586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc0b6780 0x7f26dc06a5b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.141+0000 7f26e194b700 1 --2- 172.21.15.7:0/2693594586 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f26dc15cda0 0x7f26dc0d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.141+0000 7f26e194b700 1 -- 172.21.15.7:0/2693594586 >> 172.21.15.7:0/2693594586 conn(0x7f26dc0d2cb0 msgr2=0x7f26dc0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:05.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.141+0000 7f26e194b700 1 -- 172.21.15.7:0/2693594586 shutdown_connections 2024-09-13T18:53:05.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.141+0000 7f26e194b700 1 -- 172.21.15.7:0/2693594586 wait complete. 2024-09-13T18:53:05.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26e194b700 1 Processor -- start 2024-09-13T18:53:05.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26e194b700 1 -- start start 2024-09-13T18:53:05.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26e194b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f26dc0b5e00 0x7f26dc1598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26e194b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26dc0b6780 0x7f26dc152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26e194b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc15cda0 0x7f26dc1f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26e194b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc1f5cf0 con 0x7f26dc15cda0 2024-09-13T18:53:05.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26e194b700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc159e00 con 0x7f26dc0b6780 2024-09-13T18:53:05.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26e194b700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc1f5f20 con 0x7f26dc0b5e00 2024-09-13T18:53:05.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26db7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc15cda0 0x7f26dc1f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26db7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc15cda0 0x7f26dc1f3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59904/0 (socket says 172.21.15.7:59904) 2024-09-13T18:53:05.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26db7fe700 1 -- 172.21.15.7:0/3089912500 learned_addr learned my addr 172.21.15.7:0/3089912500 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:05.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26d27fc700 1 --2- 172.21.15.7:0/3089912500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26dc0b6780 0x7f26dc152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26daffd700 1 --2- 172.21.15.7:0/3089912500 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f26dc0b5e00 0x7f26dc1598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26db7fe700 1 -- 172.21.15.7:0/3089912500 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f26dc0b5e00 msgr2=0x7f26dc1598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26db7fe700 1 --2- 172.21.15.7:0/3089912500 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f26dc0b5e00 0x7f26dc1598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26db7fe700 1 -- 172.21.15.7:0/3089912500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26dc0b6780 msgr2=0x7f26dc152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26db7fe700 1 --2- 172.21.15.7:0/3089912500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26dc0b6780 0x7f26dc152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26db7fe700 1 -- 172.21.15.7:0/3089912500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d4009b30 con 0x7f26dc15cda0 2024-09-13T18:53:05.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26db7fe700 1 --2- 172.21.15.7:0/3089912500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc15cda0 0x7f26dc1f3380 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f26d4012000 tx=0x7f26d4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:05.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26d8ff9700 1 -- 172.21.15.7:0/3089912500 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4004250 con 0x7f26dc15cda0 2024-09-13T18:53:05.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26d8ff9700 1 -- 172.21.15.7:0/3089912500 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d40043d0 con 0x7f26dc15cda0 2024-09-13T18:53:05.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26e194b700 1 -- 172.21.15.7:0/3089912500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26dc1f6200 con 0x7f26dc15cda0 2024-09-13T18:53:05.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26d8ff9700 1 -- 172.21.15.7:0/3089912500 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4004780 con 0x7f26dc15cda0 2024-09-13T18:53:05.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.145+0000 7f26e194b700 1 -- 172.21.15.7:0/3089912500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26dc1f67c0 con 0x7f26dc15cda0 2024-09-13T18:53:05.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.149+0000 7f26e194b700 1 -- 172.21.15.7:0/3089912500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26dc15f4f0 con 0x7f26dc15cda0 2024-09-13T18:53:05.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.149+0000 7f26d8ff9700 1 -- 172.21.15.7:0/3089912500 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f26d401c440 con 0x7f26dc15cda0 2024-09-13T18:53:05.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.149+0000 7f26d8ff9700 1 --2- 172.21.15.7:0/3089912500 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f26bc066730 0x7f26bc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.149+0000 7f26d8ff9700 1 -- 172.21.15.7:0/3089912500 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26d40856e0 con 0x7f26dc15cda0 2024-09-13T18:53:05.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.149+0000 7f26daffd700 1 --2- 172.21.15.7:0/3089912500 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f26bc066730 0x7f26bc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.149+0000 7f26daffd700 1 --2- 172.21.15.7:0/3089912500 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f26bc066730 0x7f26bc068bf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f26dc15b1d0 tx=0x7f26cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:05.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.153+0000 7f26d8ff9700 1 -- 172.21.15.7:0/3089912500 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f26d4052b60 con 0x7f26dc15cda0 2024-09-13T18:53:05.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.325+0000 7f26e194b700 1 -- 172.21.15.7:0/3089912500 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f26dc0b6240 con 0x7f26dc15cda0 2024-09-13T18:53:05.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.329+0000 7f26d8ff9700 1 -- 172.21.15.7:0/3089912500 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f26d4056180 con 0x7f26dc15cda0 2024-09-13T18:53:05.333 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:53:05.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.329+0000 7f26e194b700 1 -- 172.21.15.7:0/3089912500 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f26bc066730 msgr2=0x7f26bc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.329+0000 7f26e194b700 1 --2- 172.21.15.7:0/3089912500 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f26bc066730 0x7f26bc068bf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f26dc15b1d0 tx=0x7f26cc008040 comp rx=0 tx=0).stop 2024-09-13T18:53:05.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.329+0000 7f26e194b700 1 -- 172.21.15.7:0/3089912500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc15cda0 msgr2=0x7f26dc1f3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.329+0000 7f26e194b700 1 --2- 172.21.15.7:0/3089912500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc15cda0 0x7f26dc1f3380 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f26d4012000 tx=0x7f26d4004af0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.329+0000 7f26e194b700 1 -- 172.21.15.7:0/3089912500 shutdown_connections 2024-09-13T18:53:05.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.329+0000 7f26e194b700 1 --2- 172.21.15.7:0/3089912500 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f26bc066730 0x7f26bc068bf0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.329+0000 7f26e194b700 1 --2- 172.21.15.7:0/3089912500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26dc15cda0 0x7f26dc1f3380 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.329+0000 7f26e194b700 1 --2- 172.21.15.7:0/3089912500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26dc0b6780 0x7f26dc152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.329+0000 7f26e194b700 1 --2- 172.21.15.7:0/3089912500 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f26dc0b5e00 0x7f26dc1598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.329+0000 7f26e194b700 1 -- 172.21.15.7:0/3089912500 >> 172.21.15.7:0/3089912500 conn(0x7f26dc0d2cb0 msgr2=0x7f26dc06b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:05.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.329+0000 7f26e194b700 1 -- 172.21.15.7:0/3089912500 shutdown_connections 2024-09-13T18:53:05.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.329+0000 7f26e194b700 1 -- 172.21.15.7:0/3089912500 wait complete. 2024-09-13T18:53:05.349 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:05.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.457+0000 7fabe61f7700 1 Processor -- start 2024-09-13T18:53:05.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.457+0000 7fabe61f7700 1 -- start start 2024-09-13T18:53:05.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe61f7700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabe015cec0 0x7fabe00cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe61f7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabe006b310 0x7fabe00d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe61f7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe00d21e0 con 0x7fabe015cae0 2024-09-13T18:53:05.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe61f7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabe00d1750 con 0x7fabe006b310 2024-09-13T18:53:05.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe61f7700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe00d1940 con 0x7fabe015cec0 2024-09-13T18:53:05.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe59f6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabe006b310 0x7fabe00d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe51f5700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fabe015cae0 0x7fabe006ac00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47584/0 (socket says 172.21.15.7:47584) 2024-09-13T18:53:05.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe51f5700 1 -- 172.21.15.7:0/1051763634 learned_addr learned my addr 172.21.15.7:0/1051763634 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:05.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe49f4700 1 --2- 172.21.15.7:0/1051763634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabe015cec0 0x7fabe00cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe59f6700 1 -- 172.21.15.7:0/1051763634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabe015cec0 msgr2=0x7fabe00cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:05.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe59f6700 1 --2- 172.21.15.7:0/1051763634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabe015cec0 0x7fabe00cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe59f6700 1 -- 172.21.15.7:0/1051763634 >> v1:172.21.15.7:6789/0 conn(0x7fabe015cae0 legacy=0x7fabe006ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabd7fff700 1 -- 172.21.15.7:0/1051763634 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769526205 0 0) 0x7fabe00d21e0 con 0x7fabe015cae0 2024-09-13T18:53:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe59f6700 1 -- 172.21.15.7:0/1051763634 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe00d1c20 con 0x7fabe006b310 2024-09-13T18:53:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe59f6700 1 --2- 172.21.15.7:0/1051763634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe006b310 0x7fabe00d1210 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009d90 tx=0x7fabd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2fae2a25e821360 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabd7fff700 1 -- 172.21.15.7:0/1051763634 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd800e070 con 0x7fabe006b310 2024-09-13T18:53:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabd7fff700 1 -- 172.21.15.7:0/1051763634 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd800bb20 con 0x7fabe006b310 2024-09-13T18:53:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabd7fff700 1 -- 172.21.15.7:0/1051763634 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd8012430 con 0x7fabe006b310 2024-09-13T18:53:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe61f7700 1 -- 172.21.15.7:0/1051763634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe006b310 msgr2=0x7fabe00d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe61f7700 1 --2- 172.21.15.7:0/1051763634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe006b310 0x7fabe00d1210 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fabd8009d90 tx=0x7fabd800b330 comp rx=0 tx=0).stop 2024-09-13T18:53:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe61f7700 1 -- 172.21.15.7:0/1051763634 shutdown_connections 2024-09-13T18:53:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe61f7700 1 --2- 172.21.15.7:0/1051763634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe006b310 0x7fabe00d1210 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe61f7700 1 --2- 172.21.15.7:0/1051763634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabe015cec0 0x7fabe00cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe61f7700 1 -- 172.21.15.7:0/1051763634 >> 172.21.15.7:0/1051763634 conn(0x7fabe00d2c80 msgr2=0x7fabe00d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:05.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe61f7700 1 -- 172.21.15.7:0/1051763634 shutdown_connections 2024-09-13T18:53:05.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.461+0000 7fabe61f7700 1 -- 172.21.15.7:0/1051763634 wait complete. 2024-09-13T18:53:05.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe61f7700 1 Processor -- start 2024-09-13T18:53:05.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe61f7700 1 -- start start 2024-09-13T18:53:05.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe61f7700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabe006b310 0x7fabe01f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe61f7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe015cae0 0x7fabe015fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe61f7700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabe015cec0 0x7fabe00c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe61f7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe015d3d0 con 0x7fabe015cae0 2024-09-13T18:53:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe61f7700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe00c7d00 con 0x7fabe006b310 2024-09-13T18:53:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe61f7700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe00c7f60 con 0x7fabe015cec0 2024-09-13T18:53:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe49f4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe015cae0 0x7fabe015fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe49f4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe015cae0 0x7fabe015fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59928/0 (socket says 172.21.15.7:59928) 2024-09-13T18:53:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe49f4700 1 -- 172.21.15.7:0/2116421989 learned_addr learned my addr 172.21.15.7:0/2116421989 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe51f5700 1 --2- 172.21.15.7:0/2116421989 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabe006b310 0x7fabe01f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe59f6700 1 --2- 172.21.15.7:0/2116421989 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabe015cec0 0x7fabe00c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe49f4700 1 -- 172.21.15.7:0/2116421989 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabe015cec0 msgr2=0x7fabe00c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe49f4700 1 --2- 172.21.15.7:0/2116421989 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabe015cec0 0x7fabe00c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe49f4700 1 -- 172.21.15.7:0/2116421989 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabe006b310 msgr2=0x7fabe01f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe49f4700 1 --2- 172.21.15.7:0/2116421989 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabe006b310 0x7fabe01f3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe49f4700 1 -- 172.21.15.7:0/2116421989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd8009a70 con 0x7fabe015cae0 2024-09-13T18:53:05.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabe49f4700 1 --2- 172.21.15.7:0/2116421989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe015cae0 0x7fabe015fac0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fabd000e9d0 tx=0x7fabd000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:05.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabd5ffb700 1 -- 172.21.15.7:0/2116421989 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd0009720 con 0x7fabe015cae0 2024-09-13T18:53:05.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabd5ffb700 1 -- 172.21.15.7:0/2116421989 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd000ee70 con 0x7fabe015cae0 2024-09-13T18:53:05.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.465+0000 7fabd5ffb700 1 -- 172.21.15.7:0/2116421989 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd000ca50 con 0x7fabe015cae0 2024-09-13T18:53:05.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.469+0000 7fabe61f7700 1 -- 172.21.15.7:0/2116421989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe00c8240 con 0x7fabe015cae0 2024-09-13T18:53:05.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.469+0000 7fabe61f7700 1 -- 172.21.15.7:0/2116421989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe00c8800 con 0x7fabe015cae0 2024-09-13T18:53:05.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.469+0000 7fabe61f7700 1 -- 172.21.15.7:0/2116421989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe00d1210 con 0x7fabe015cae0 2024-09-13T18:53:05.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.469+0000 7fabd5ffb700 1 -- 172.21.15.7:0/2116421989 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fabd000cbb0 con 0x7fabe015cae0 2024-09-13T18:53:05.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.469+0000 7fabd5ffb700 1 --2- 172.21.15.7:0/2116421989 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fabc4066760 0x7fabc4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.473+0000 7fabd5ffb700 1 -- 172.21.15.7:0/2116421989 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabd0084190 con 0x7fabe015cae0 2024-09-13T18:53:05.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.473+0000 7fabe51f5700 1 --2- 172.21.15.7:0/2116421989 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fabc4066760 0x7fabc4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.473+0000 7fabe51f5700 1 --2- 172.21.15.7:0/2116421989 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fabc4066760 0x7fabc4068c20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fabdc000f60 tx=0x7fabdc007e00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:05.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.481+0000 7fabd5ffb700 1 -- 172.21.15.7:0/2116421989 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fabd0051480 con 0x7fabe015cae0 2024-09-13T18:53:05.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.685+0000 7fabe61f7700 1 -- 172.21.15.7:0/2116421989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabe006ac50 con 0x7fabe015cae0 2024-09-13T18:53:05.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.685+0000 7fabd5ffb700 1 -- 172.21.15.7:0/2116421989 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4725 (secure 0 0 0) 0x7fabd000ce60 con 0x7fabe015cae0 2024-09-13T18:53:05.693 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:05.693 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1309419125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1309419125},{"type":"v1","addr":"172.21.15.7:6835","nonce":1309419125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/951828843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":951828843},{"type":"v1","addr":"172.21.15.7:6833","nonce":951828843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4226627580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4226627580},{"type":"v1","addr":"172.21.15.171:6835","nonce":4226627580}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:52:17.908937+0000","modified":"2024-09-13T18:52:20.930515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6837/3658296803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3658296803},{"type":"v1","addr":"172.21.15.171:6837","nonce":3658296803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.689+0000 7fabe61f7700 1 -- 172.21.15.7:0/2116421989 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fabc4066760 msgr2=0x7fabc4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.689+0000 7fabe61f7700 1 --2- 172.21.15.7:0/2116421989 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fabc4066760 0x7fabc4068c20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fabdc000f60 tx=0x7fabdc007e00 comp rx=0 tx=0).stop 2024-09-13T18:53:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.689+0000 7fabe61f7700 1 -- 172.21.15.7:0/2116421989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe015cae0 msgr2=0x7fabe015fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.689+0000 7fabe61f7700 1 --2- 172.21.15.7:0/2116421989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe015cae0 0x7fabe015fac0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fabd000e9d0 tx=0x7fabd000ece0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.689+0000 7fabe61f7700 1 -- 172.21.15.7:0/2116421989 shutdown_connections 2024-09-13T18:53:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.689+0000 7fabe61f7700 1 --2- 172.21.15.7:0/2116421989 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fabc4066760 0x7fabc4068c20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.689+0000 7fabe61f7700 1 --2- 172.21.15.7:0/2116421989 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fabe015cec0 0x7fabe00c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.689+0000 7fabe61f7700 1 --2- 172.21.15.7:0/2116421989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabe015cae0 0x7fabe015fac0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.689+0000 7fabe61f7700 1 --2- 172.21.15.7:0/2116421989 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fabe006b310 0x7fabe01f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.689+0000 7fabe61f7700 1 -- 172.21.15.7:0/2116421989 >> 172.21.15.7:0/2116421989 conn(0x7fabe00d2c80 msgr2=0x7fabe00ce720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.689+0000 7fabe61f7700 1 -- 172.21.15.7:0/2116421989 shutdown_connections 2024-09-13T18:53:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.689+0000 7fabe61f7700 1 -- 172.21.15.7:0/2116421989 wait complete. 2024-09-13T18:53:05.696 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-13T18:53:05.707 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:53:05.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.817+0000 7fc011af9700 1 Processor -- start 2024-09-13T18:53:05.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.817+0000 7fc011af9700 1 -- start start 2024-09-13T18:53:05.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.817+0000 7fc011af9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc00c15ecb0 0x7fc00c0cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.817+0000 7fc011af9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc00c0ce2a0 con 0x7fc00c0cfd80 2024-09-13T18:53:05.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.817+0000 7fc011af9700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc00c0cefc0 con 0x7fc00c1444b0 2024-09-13T18:53:05.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.817+0000 7fc011af9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00c0cf2a0 con 0x7fc00c15ecb0 2024-09-13T18:53:05.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.821+0000 7fc00bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc00c0cfd80 0x7fc00c15e5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47598/0 (socket says 172.21.15.7:47598) 2024-09-13T18:53:05.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.821+0000 7fc0112f8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc00c15ecb0 0x7fc00c0cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.821+0000 7fc00bfff700 1 -- 172.21.15.7:0/4094878810 learned_addr learned my addr 172.21.15.7:0/4094878810 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.821+0000 7fc0112f8700 1 -- 172.21.15.7:0/4094878810 >> v1:172.21.15.171:6789/0 conn(0x7fc00c1444b0 legacy=0x7fc00c0b67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.821+0000 7fc0112f8700 1 -- 172.21.15.7:0/4094878810 >> v1:172.21.15.7:6789/0 conn(0x7fc00c0cfd80 legacy=0x7fc00c15e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.821+0000 7fc0112f8700 1 -- 172.21.15.7:0/4094878810 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c0cf580 con 0x7fc00c15ecb0 2024-09-13T18:53:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.821+0000 7fc0112f8700 1 --2- 172.21.15.7:0/4094878810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc00c15ecb0 0x7fc00c0cdc10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc004009d90 tx=0x7fc00400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df49eda5dffe1983 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.821+0000 7fc00b7fe700 1 -- 172.21.15.7:0/4094878810 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00400e070 con 0x7fc00c15ecb0 2024-09-13T18:53:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.821+0000 7fc00b7fe700 1 -- 172.21.15.7:0/4094878810 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00400bbd0 con 0x7fc00c15ecb0 2024-09-13T18:53:05.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.821+0000 7fc00b7fe700 1 -- 172.21.15.7:0/4094878810 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004012430 con 0x7fc00c15ecb0 2024-09-13T18:53:05.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.821+0000 7fc011af9700 1 -- 172.21.15.7:0/4094878810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc00c15ecb0 msgr2=0x7fc00c0cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.821+0000 7fc011af9700 1 --2- 172.21.15.7:0/4094878810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc00c15ecb0 0x7fc00c0cdc10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc004009d90 tx=0x7fc00400b330 comp rx=0 tx=0).stop 2024-09-13T18:53:05.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.821+0000 7fc011af9700 1 -- 172.21.15.7:0/4094878810 shutdown_connections 2024-09-13T18:53:05.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.821+0000 7fc011af9700 1 --2- 172.21.15.7:0/4094878810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc00c15ecb0 0x7fc00c0cdc10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.821+0000 7fc011af9700 1 -- 172.21.15.7:0/4094878810 >> 172.21.15.7:0/4094878810 conn(0x7fc00c071d00 msgr2=0x7fc00c0720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:05.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.821+0000 7fc011af9700 1 -- 172.21.15.7:0/4094878810 shutdown_connections 2024-09-13T18:53:05.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.821+0000 7fc011af9700 1 -- 172.21.15.7:0/4094878810 wait complete. 2024-09-13T18:53:05.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc011af9700 1 Processor -- start 2024-09-13T18:53:05.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc011af9700 1 -- start start 2024-09-13T18:53:05.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc011af9700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc00c0cfd80 0x7fc00c159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc011af9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc00c1444b0 0x7fc00c162730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc011af9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc00c15ecb0 0x7fc00c0c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc011af9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c0c74e0 con 0x7fc00c15ecb0 2024-09-13T18:53:05.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc011af9700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c163010 con 0x7fc00c1444b0 2024-09-13T18:53:05.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc011af9700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c1632f0 con 0x7fc00c0cfd80 2024-09-13T18:53:05.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc0112f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc00c15ecb0 0x7fc00c0c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc0112f8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc00c15ecb0 0x7fc00c0c98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59950/0 (socket says 172.21.15.7:59950) 2024-09-13T18:53:05.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc0112f8700 1 -- 172.21.15.7:0/2150038594 learned_addr learned my addr 172.21.15.7:0/2150038594 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:05.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc00bfff700 1 --2- 172.21.15.7:0/2150038594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc00c1444b0 0x7fc00c162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc010af7700 1 --2- 172.21.15.7:0/2150038594 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc00c0cfd80 0x7fc00c159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc0112f8700 1 -- 172.21.15.7:0/2150038594 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc00c0cfd80 msgr2=0x7fc00c159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc0112f8700 1 --2- 172.21.15.7:0/2150038594 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc00c0cfd80 0x7fc00c159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc0112f8700 1 -- 172.21.15.7:0/2150038594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc00c1444b0 msgr2=0x7fc00c162730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc0112f8700 1 --2- 172.21.15.7:0/2150038594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc00c1444b0 0x7fc00c162730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc0112f8700 1 -- 172.21.15.7:0/2150038594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc004009a70 con 0x7fc00c15ecb0 2024-09-13T18:53:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc00bfff700 1 --2- 172.21.15.7:0/2150038594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc00c1444b0 0x7fc00c162730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc0112f8700 1 --2- 172.21.15.7:0/2150038594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc00c15ecb0 0x7fc00c0c98a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc004014040 tx=0x7fc004004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc0097fa700 1 -- 172.21.15.7:0/2150038594 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004012430 con 0x7fc00c15ecb0 2024-09-13T18:53:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc0097fa700 1 -- 172.21.15.7:0/2150038594 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc004020070 con 0x7fc00c15ecb0 2024-09-13T18:53:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc011af9700 1 -- 172.21.15.7:0/2150038594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00c1635d0 con 0x7fc00c15ecb0 2024-09-13T18:53:05.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc0097fa700 1 -- 172.21.15.7:0/2150038594 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004004280 con 0x7fc00c15ecb0 2024-09-13T18:53:05.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.825+0000 7fc011af9700 1 -- 172.21.15.7:0/2150038594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00c162c70 con 0x7fc00c15ecb0 2024-09-13T18:53:05.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.829+0000 7fc011af9700 1 -- 172.21.15.7:0/2150038594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00c0c7650 con 0x7fc00c15ecb0 2024-09-13T18:53:05.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.829+0000 7fc0097fa700 1 -- 172.21.15.7:0/2150038594 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc0040043e0 con 0x7fc00c15ecb0 2024-09-13T18:53:05.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.829+0000 7fc0097fa700 1 --2- 172.21.15.7:0/2150038594 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbfec066730 0x7fbfec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:05.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.829+0000 7fc010af7700 1 --2- 172.21.15.7:0/2150038594 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbfec066730 0x7fbfec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:05.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.829+0000 7fc0097fa700 1 -- 172.21.15.7:0/2150038594 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc004022070 con 0x7fc00c15ecb0 2024-09-13T18:53:05.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.829+0000 7fc010af7700 1 --2- 172.21.15.7:0/2150038594 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbfec066730 0x7fbfec068bf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc00c15a6b0 tx=0x7fbffc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:05.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:05.837+0000 7fc0097fa700 1 -- 172.21.15.7:0/2150038594 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc0040537f0 con 0x7fc00c15ecb0 2024-09-13T18:53:06.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.017+0000 7fc011af9700 1 -- 172.21.15.7:0/2150038594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc00c1637c0 con 0x7fc00c15ecb0 2024-09-13T18:53:06.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.021+0000 7fc0097fa700 1 -- 172.21.15.7:0/2150038594 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16457 (secure 0 0 0) 0x7fc004056e10 con 0x7fc00c15ecb0 2024-09-13T18:53:06.028 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:06.028 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:52:18.912374+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:53:06.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.025+0000 7fc011af9700 1 -- 172.21.15.7:0/2150038594 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbfec066730 msgr2=0x7fbfec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.025+0000 7fc011af9700 1 --2- 172.21.15.7:0/2150038594 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbfec066730 0x7fbfec068bf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc00c15a6b0 tx=0x7fbffc009040 comp rx=0 tx=0).stop 2024-09-13T18:53:06.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.025+0000 7fc011af9700 1 -- 172.21.15.7:0/2150038594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc00c15ecb0 msgr2=0x7fc00c0c98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.025+0000 7fc011af9700 1 --2- 172.21.15.7:0/2150038594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc00c15ecb0 0x7fc00c0c98a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc004014040 tx=0x7fc004004af0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.025+0000 7fc011af9700 1 -- 172.21.15.7:0/2150038594 shutdown_connections 2024-09-13T18:53:06.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.025+0000 7fc011af9700 1 --2- 172.21.15.7:0/2150038594 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbfec066730 0x7fbfec068bf0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.025+0000 7fc011af9700 1 --2- 172.21.15.7:0/2150038594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc00c15ecb0 0x7fc00c0c98a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.025+0000 7fc011af9700 1 --2- 172.21.15.7:0/2150038594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc00c1444b0 0x7fc00c162730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.025+0000 7fc011af9700 1 --2- 172.21.15.7:0/2150038594 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc00c0cfd80 0x7fc00c159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.025+0000 7fc011af9700 1 -- 172.21.15.7:0/2150038594 >> 172.21.15.7:0/2150038594 conn(0x7fc00c071d00 msgr2=0x7fc00c15d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:06.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.025+0000 7fc011af9700 1 -- 172.21.15.7:0/2150038594 shutdown_connections 2024-09-13T18:53:06.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.025+0000 7fc011af9700 1 -- 172.21.15.7:0/2150038594 wait complete. 2024-09-13T18:53:06.043 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T18:53:06.044 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:06.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.149+0000 7f715b59e700 1 Processor -- start 2024-09-13T18:53:06.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.149+0000 7f715b59e700 1 -- start start 2024-09-13T18:53:06.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.149+0000 7f715b59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f715c06fb30 0x7f715c0adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.149+0000 7f715b59e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f715c06bb90 0x7f715c0cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.149+0000 7f715b59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f715c0b3380 0x7f715c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.149+0000 7f715b59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715c06cd30 con 0x7f715c0b3380 2024-09-13T18:53:06.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.149+0000 7f715b59e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715c06a780 con 0x7f715c06bb90 2024-09-13T18:53:06.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.149+0000 7f715b59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715c159470 con 0x7f715c06fb30 2024-09-13T18:53:06.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.149+0000 7f715ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f715c0b3380 0x7f715c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.149+0000 7f715ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f715c0b3380 0x7f715c06d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59960/0 (socket says 172.21.15.7:59960) 2024-09-13T18:53:06.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.149+0000 7f715ad9d700 1 -- 172.21.15.7:0/760322913 learned_addr learned my addr 172.21.15.7:0/760322913 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:06.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.149+0000 7f7159d9b700 1 --2- 172.21.15.7:0/760322913 >> v2:172.21.15.171:3300/0 conn(0x7f715c06bb90 0x7f715c0cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715a59c700 1 --2- 172.21.15.7:0/760322913 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f715c06fb30 0x7f715c0adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715ad9d700 1 -- 172.21.15.7:0/760322913 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f715c06fb30 msgr2=0x7f715c0adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:06.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715ad9d700 1 --2- 172.21.15.7:0/760322913 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f715c06fb30 0x7f715c0adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715ad9d700 1 -- 172.21.15.7:0/760322913 >> v2:172.21.15.171:3300/0 conn(0x7f715c06bb90 msgr2=0x7f715c0cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:06.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715ad9d700 1 --2- 172.21.15.7:0/760322913 >> v2:172.21.15.171:3300/0 conn(0x7f715c06bb90 0x7f715c0cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715ad9d700 1 -- 172.21.15.7:0/760322913 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c06c0b0 con 0x7f715c0b3380 2024-09-13T18:53:06.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715a59c700 1 --2- 172.21.15.7:0/760322913 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f715c06fb30 0x7f715c0adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:06.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715ad9d700 1 --2- 172.21.15.7:0/760322913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c0b3380 0x7f715c06d150 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f715c071800 tx=0x7f7154009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0a6bd7ee346e95 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:06.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715959a700 1 -- 172.21.15.7:0/760322913 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154011830 con 0x7f715c0b3380 2024-09-13T18:53:06.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715959a700 1 -- 172.21.15.7:0/760322913 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7154011990 con 0x7f715c0b3380 2024-09-13T18:53:06.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715959a700 1 -- 172.21.15.7:0/760322913 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154011d20 con 0x7f715c0b3380 2024-09-13T18:53:06.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715b59e700 1 -- 172.21.15.7:0/760322913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c0b3380 msgr2=0x7f715c06d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715b59e700 1 --2- 172.21.15.7:0/760322913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c0b3380 0x7f715c06d150 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f715c071800 tx=0x7f7154009bf0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715b59e700 1 -- 172.21.15.7:0/760322913 shutdown_connections 2024-09-13T18:53:06.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715b59e700 1 --2- 172.21.15.7:0/760322913 >> v2:172.21.15.171:3300/0 conn(0x7f715c06bb90 0x7f715c0cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715b59e700 1 --2- 172.21.15.7:0/760322913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c0b3380 0x7f715c06d150 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715b59e700 1 --2- 172.21.15.7:0/760322913 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f715c06fb30 0x7f715c0adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715b59e700 1 -- 172.21.15.7:0/760322913 >> 172.21.15.7:0/760322913 conn(0x7f715c0b0f00 msgr2=0x7f715c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:06.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715b59e700 1 -- 172.21.15.7:0/760322913 shutdown_connections 2024-09-13T18:53:06.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.153+0000 7f715b59e700 1 -- 172.21.15.7:0/760322913 wait complete. 2024-09-13T18:53:06.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f715b59e700 1 Processor -- start 2024-09-13T18:53:06.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f715b59e700 1 -- start start 2024-09-13T18:53:06.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f715b59e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f715c06bb90 0x7f715c1448b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f715b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c06fb30 0x7f715c142770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f715b59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f715c0b3380 0x7f715c142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f715b59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c145320 con 0x7f715c06fb30 2024-09-13T18:53:06.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f715b59e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715c145600 con 0x7f715c06bb90 2024-09-13T18:53:06.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f715b59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715c14dad0 con 0x7f715c0b3380 2024-09-13T18:53:06.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f7159d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c06fb30 0x7f715c142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f7159d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c06fb30 0x7f715c142770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59964/0 (socket says 172.21.15.7:59964) 2024-09-13T18:53:06.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f7159d9b700 1 -- 172.21.15.7:0/2969573140 learned_addr learned my addr 172.21.15.7:0/2969573140 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:06.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f715ad9d700 1 --2- 172.21.15.7:0/2969573140 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f715c0b3380 0x7f715c142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f715a59c700 1 --2- 172.21.15.7:0/2969573140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f715c06bb90 0x7f715c1448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f7159d9b700 1 -- 172.21.15.7:0/2969573140 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f715c0b3380 msgr2=0x7f715c142cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f7159d9b700 1 --2- 172.21.15.7:0/2969573140 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f715c0b3380 0x7f715c142cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f7159d9b700 1 -- 172.21.15.7:0/2969573140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f715c06bb90 msgr2=0x7f715c1448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f7159d9b700 1 --2- 172.21.15.7:0/2969573140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f715c06bb90 0x7f715c1448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f7159d9b700 1 -- 172.21.15.7:0/2969573140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7154009570 con 0x7f715c06fb30 2024-09-13T18:53:06.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f715ad9d700 1 --2- 172.21.15.7:0/2969573140 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f715c0b3380 0x7f715c142cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:06.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.157+0000 7f7159d9b700 1 --2- 172.21.15.7:0/2969573140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c06fb30 0x7f715c142770 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f714c00eed0 tx=0x7f714c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:06.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.161+0000 7f714affd700 1 -- 172.21.15.7:0/2969573140 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c010040 con 0x7f715c06fb30 2024-09-13T18:53:06.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.161+0000 7f714affd700 1 -- 172.21.15.7:0/2969573140 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714c00bdb0 con 0x7f715c06fb30 2024-09-13T18:53:06.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.161+0000 7f715b59e700 1 -- 172.21.15.7:0/2969573140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715c14ddb0 con 0x7f715c06fb30 2024-09-13T18:53:06.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.161+0000 7f714affd700 1 -- 172.21.15.7:0/2969573140 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c014620 con 0x7f715c06fb30 2024-09-13T18:53:06.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.161+0000 7f715b59e700 1 -- 172.21.15.7:0/2969573140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715c144df0 con 0x7f715c06fb30 2024-09-13T18:53:06.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.161+0000 7f715b59e700 1 -- 172.21.15.7:0/2969573140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715c06a910 con 0x7f715c06fb30 2024-09-13T18:53:06.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.161+0000 7f714affd700 1 -- 172.21.15.7:0/2969573140 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f714c014780 con 0x7f715c06fb30 2024-09-13T18:53:06.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.161+0000 7f714affd700 1 --2- 172.21.15.7:0/2969573140 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f713c066730 0x7f713c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.161+0000 7f714affd700 1 -- 172.21.15.7:0/2969573140 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f714c0845f0 con 0x7f715c06fb30 2024-09-13T18:53:06.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.161+0000 7f715a59c700 1 --2- 172.21.15.7:0/2969573140 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f713c066730 0x7f713c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.165+0000 7f715a59c700 1 --2- 172.21.15.7:0/2969573140 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f713c066730 0x7f713c068bf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f715c145890 tx=0x7f7150008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:06.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.169+0000 7f714affd700 1 -- 172.21.15.7:0/2969573140 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f714c0519f0 con 0x7f715c06fb30 2024-09-13T18:53:06.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.385+0000 7f715b59e700 1 -- 172.21.15.7:0/2969573140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f715c06a5e0 con 0x7f715c06fb30 2024-09-13T18:53:06.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.385+0000 7f714affd700 1 -- 172.21.15.7:0/2969573140 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4725 (secure 0 0 0) 0x7f715c06a5e0 con 0x7f715c06fb30 2024-09-13T18:53:06.391 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:06.391 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1309419125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1309419125},{"type":"v1","addr":"172.21.15.7:6835","nonce":1309419125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/951828843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":951828843},{"type":"v1","addr":"172.21.15.7:6833","nonce":951828843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4226627580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4226627580},{"type":"v1","addr":"172.21.15.171:6835","nonce":4226627580}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:52:17.908937+0000","modified":"2024-09-13T18:52:20.930515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6837/3658296803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3658296803},{"type":"v1","addr":"172.21.15.171:6837","nonce":3658296803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:06.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.385+0000 7f715b59e700 1 -- 172.21.15.7:0/2969573140 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f713c066730 msgr2=0x7f713c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.385+0000 7f715b59e700 1 --2- 172.21.15.7:0/2969573140 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f713c066730 0x7f713c068bf0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f715c145890 tx=0x7f7150008040 comp rx=0 tx=0).stop 2024-09-13T18:53:06.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.385+0000 7f715b59e700 1 -- 172.21.15.7:0/2969573140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c06fb30 msgr2=0x7f715c142770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.385+0000 7f715b59e700 1 --2- 172.21.15.7:0/2969573140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c06fb30 0x7f715c142770 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f714c00eed0 tx=0x7f714c00b390 comp rx=0 tx=0).stop 2024-09-13T18:53:06.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.385+0000 7f715b59e700 1 -- 172.21.15.7:0/2969573140 shutdown_connections 2024-09-13T18:53:06.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.385+0000 7f715b59e700 1 --2- 172.21.15.7:0/2969573140 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f713c066730 0x7f713c068bf0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.385+0000 7f715b59e700 1 --2- 172.21.15.7:0/2969573140 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f715c0b3380 0x7f715c142cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.385+0000 7f715b59e700 1 --2- 172.21.15.7:0/2969573140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f715c06fb30 0x7f715c142770 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.385+0000 7f715b59e700 1 --2- 172.21.15.7:0/2969573140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f715c06bb90 0x7f715c1448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.385+0000 7f715b59e700 1 -- 172.21.15.7:0/2969573140 >> 172.21.15.7:0/2969573140 conn(0x7f715c0b0f00 msgr2=0x7f715c070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:06.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.389+0000 7f715b59e700 1 -- 172.21.15.7:0/2969573140 shutdown_connections 2024-09-13T18:53:06.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.389+0000 7f715b59e700 1 -- 172.21.15.7:0/2969573140 wait complete. 2024-09-13T18:53:06.395 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-13T18:53:06.406 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:06.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835359e700 1 Processor -- start 2024-09-13T18:53:06.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835359e700 1 -- start start 2024-09-13T18:53:06.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835359e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f835406bbf0 0x7f83540b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835359e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f835406acc0 0x7f835415e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835359e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f835413dcb0 con 0x7f83540b3910 2024-09-13T18:53:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835359e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f835415be60 con 0x7f835406bbf0 2024-09-13T18:53:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835359e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835413ffd0 con 0x7f835406acc0 2024-09-13T18:53:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f8351d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f83540b3910 0x7f835406a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47602/0 (socket says 172.21.15.7:47602) 2024-09-13T18:53:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f8351d9b700 1 -- 172.21.15.7:0/4127418497 learned_addr learned my addr 172.21.15.7:0/4127418497 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835259c700 1 --2- 172.21.15.7:0/4127418497 >> v2:172.21.15.7:3300/0 conn(0x7f835406bbf0 0x7f83540b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f8352d9d700 1 --2- 172.21.15.7:0/4127418497 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f835406acc0 0x7f835415e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835159a700 1 -- 172.21.15.7:0/4127418497 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568322213 0 0) 0x7f835413dcb0 con 0x7f83540b3910 2024-09-13T18:53:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835159a700 1 -- 172.21.15.7:0/4127418497 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83340035d0 con 0x7f83540b3910 2024-09-13T18:53:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835159a700 1 -- 172.21.15.7:0/4127418497 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 443579630 0 0) 0x7f83340035d0 con 0x7f83540b3910 2024-09-13T18:53:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835259c700 1 -- 172.21.15.7:0/4127418497 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f835406acc0 msgr2=0x7f835415e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835259c700 1 --2- 172.21.15.7:0/4127418497 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f835406acc0 0x7f835415e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835259c700 1 -- 172.21.15.7:0/4127418497 >> v1:172.21.15.7:6789/0 conn(0x7f83540b3910 legacy=0x7f835406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835259c700 1 -- 172.21.15.7:0/4127418497 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83540b3d70 con 0x7f835406bbf0 2024-09-13T18:53:06.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f8352d9d700 1 --2- 172.21.15.7:0/4127418497 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f835406acc0 0x7f835415e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835259c700 1 --2- 172.21.15.7:0/4127418497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f835406bbf0 0x7f83540b33d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8344009d90 tx=0x7f834400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76c7a5de7fd38b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835159a700 1 -- 172.21.15.7:0/4127418497 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834400e070 con 0x7f835406bbf0 2024-09-13T18:53:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835159a700 1 -- 172.21.15.7:0/4127418497 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8344003a40 con 0x7f835406bbf0 2024-09-13T18:53:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.513+0000 7f835159a700 1 -- 172.21.15.7:0/4127418497 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c001670 con 0x7f835406bbf0 2024-09-13T18:53:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835359e700 1 -- 172.21.15.7:0/4127418497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f835406bbf0 msgr2=0x7f83540b33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835359e700 1 --2- 172.21.15.7:0/4127418497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f835406bbf0 0x7f83540b33d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8344009d90 tx=0x7f834400b330 comp rx=0 tx=0).stop 2024-09-13T18:53:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835359e700 1 -- 172.21.15.7:0/4127418497 shutdown_connections 2024-09-13T18:53:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835359e700 1 --2- 172.21.15.7:0/4127418497 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f835406acc0 0x7f835415e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835359e700 1 --2- 172.21.15.7:0/4127418497 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f835406bbf0 0x7f83540b33d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835359e700 1 -- 172.21.15.7:0/4127418497 >> 172.21.15.7:0/4127418497 conn(0x7f83540b0f00 msgr2=0x7f83540b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835359e700 1 -- 172.21.15.7:0/4127418497 shutdown_connections 2024-09-13T18:53:06.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835359e700 1 -- 172.21.15.7:0/4127418497 wait complete. 2024-09-13T18:53:06.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835359e700 1 Processor -- start 2024-09-13T18:53:06.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835359e700 1 -- start start 2024-09-13T18:53:06.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835359e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f835406acc0 0x7f83541363f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835359e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f835406bbf0 0x7f8354159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83540b3910 0x7f835415a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835359e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83541392d0 con 0x7f83540b3910 2024-09-13T18:53:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835359e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83541386c0 con 0x7f835406acc0 2024-09-13T18:53:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835359e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83541389a0 con 0x7f835406bbf0 2024-09-13T18:53:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f8352d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83540b3910 0x7f835415a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f8352d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83540b3910 0x7f835415a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59976/0 (socket says 172.21.15.7:59976) 2024-09-13T18:53:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f8352d9d700 1 -- 172.21.15.7:0/2131611205 learned_addr learned my addr 172.21.15.7:0/2131611205 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835259c700 1 --2- 172.21.15.7:0/2131611205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f835406acc0 0x7f83541363f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f8351d9b700 1 --2- 172.21.15.7:0/2131611205 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f835406bbf0 0x7f8354159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f8352d9d700 1 -- 172.21.15.7:0/2131611205 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f835406bbf0 msgr2=0x7f8354159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f8352d9d700 1 --2- 172.21.15.7:0/2131611205 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f835406bbf0 0x7f8354159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f8352d9d700 1 -- 172.21.15.7:0/2131611205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f835406acc0 msgr2=0x7f83541363f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f8352d9d700 1 --2- 172.21.15.7:0/2131611205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f835406acc0 0x7f83541363f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f8352d9d700 1 -- 172.21.15.7:0/2131611205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8344009a70 con 0x7f83540b3910 2024-09-13T18:53:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f8352d9d700 1 --2- 172.21.15.7:0/2131611205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83540b3910 0x7f835415a000 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f834c00e810 tx=0x7f834c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f834affd700 1 -- 172.21.15.7:0/2131611205 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c004220 con 0x7f83540b3910 2024-09-13T18:53:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f834affd700 1 -- 172.21.15.7:0/2131611205 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834c0043a0 con 0x7f83540b3910 2024-09-13T18:53:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835359e700 1 -- 172.21.15.7:0/2131611205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8354138c80 con 0x7f83540b3910 2024-09-13T18:53:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f834affd700 1 -- 172.21.15.7:0/2131611205 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c0036c0 con 0x7f83540b3910 2024-09-13T18:53:06.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.517+0000 7f835359e700 1 -- 172.21.15.7:0/2131611205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8354136930 con 0x7f83540b3910 2024-09-13T18:53:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.521+0000 7f835359e700 1 -- 172.21.15.7:0/2131611205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8354139480 con 0x7f83540b3910 2024-09-13T18:53:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.521+0000 7f834affd700 1 -- 172.21.15.7:0/2131611205 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f834c003840 con 0x7f83540b3910 2024-09-13T18:53:06.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.521+0000 7f834affd700 1 --2- 172.21.15.7:0/2131611205 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8338066760 0x7f8338068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.521+0000 7f834affd700 1 -- 172.21.15.7:0/2131611205 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f834c083c10 con 0x7f83540b3910 2024-09-13T18:53:06.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.521+0000 7f835259c700 1 --2- 172.21.15.7:0/2131611205 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8338066760 0x7f8338068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.525+0000 7f835259c700 1 --2- 172.21.15.7:0/2131611205 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8338066760 0x7f8338068c20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f834400b370 tx=0x7f8344011000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:06.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.529+0000 7f834affd700 1 -- 172.21.15.7:0/2131611205 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8354139480 con 0x7f83540b3910 2024-09-13T18:53:06.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.737+0000 7f835359e700 1 -- 172.21.15.7:0/2131611205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83540b3530 con 0x7f83540b3910 2024-09-13T18:53:06.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.737+0000 7f834affd700 1 -- 172.21.15.7:0/2131611205 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4725 (secure 0 0 0) 0x7f834c020020 con 0x7f83540b3910 2024-09-13T18:53:06.743 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:06.743 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/1309419125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":1309419125},{"type":"v1","addr":"172.21.15.7:6835","nonce":1309419125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/951828843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":951828843},{"type":"v1","addr":"172.21.15.7:6833","nonce":951828843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/4226627580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":4226627580},{"type":"v1","addr":"172.21.15.171:6835","nonce":4226627580}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:52:17.908937+0000","modified":"2024-09-13T18:52:20.930515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.171:6837/3658296803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3658296803},{"type":"v1","addr":"172.21.15.171:6837","nonce":3658296803}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T18:53:06.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.741+0000 7f835359e700 1 -- 172.21.15.7:0/2131611205 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8338066760 msgr2=0x7f8338068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.741+0000 7f835359e700 1 --2- 172.21.15.7:0/2131611205 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8338066760 0x7f8338068c20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f834400b370 tx=0x7f8344011000 comp rx=0 tx=0).stop 2024-09-13T18:53:06.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.741+0000 7f835359e700 1 -- 172.21.15.7:0/2131611205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83540b3910 msgr2=0x7f835415a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.741+0000 7f835359e700 1 --2- 172.21.15.7:0/2131611205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83540b3910 0x7f835415a000 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f834c00e810 tx=0x7f834c00ebd0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.741+0000 7f835359e700 1 -- 172.21.15.7:0/2131611205 shutdown_connections 2024-09-13T18:53:06.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.741+0000 7f835359e700 1 --2- 172.21.15.7:0/2131611205 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8338066760 0x7f8338068c20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.741+0000 7f835359e700 1 --2- 172.21.15.7:0/2131611205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83540b3910 0x7f835415a000 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.741+0000 7f835359e700 1 --2- 172.21.15.7:0/2131611205 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f835406bbf0 0x7f8354159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.741+0000 7f835359e700 1 --2- 172.21.15.7:0/2131611205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f835406acc0 0x7f83541363f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.741+0000 7f835359e700 1 -- 172.21.15.7:0/2131611205 >> 172.21.15.7:0/2131611205 conn(0x7f83540b0f00 msgr2=0x7f83540afa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:06.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.741+0000 7f835359e700 1 -- 172.21.15.7:0/2131611205 shutdown_connections 2024-09-13T18:53:06.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.741+0000 7f835359e700 1 -- 172.21.15.7:0/2131611205 wait complete. 2024-09-13T18:53:06.748 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-13T18:53:06.759 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:53:06.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.869+0000 7f3f4ffff700 1 Processor -- start 2024-09-13T18:53:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4ffff700 1 -- start start 2024-09-13T18:53:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3f50071990 0x7f3f5015d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4ffff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3f5015ab00 0x7f3f5015d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4ffff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5015dea0 con 0x7f3f5006aad0 2024-09-13T18:53:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4ffff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f50159e30 con 0x7f3f50071990 2024-09-13T18:53:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4ffff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5015e3f0 con 0x7f3f5015ab00 2024-09-13T18:53:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4effd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3f5006aad0 0x7f3f50071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47606/0 (socket says 172.21.15.7:47606) 2024-09-13T18:53:06.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4effd700 1 -- 172.21.15.7:0/3910577620 learned_addr learned my addr 172.21.15.7:0/3910577620 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:06.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4e7fc700 1 --2- 172.21.15.7:0/3910577620 >> v2:172.21.15.7:3300/0 conn(0x7f3f50071990 0x7f3f5015d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4f7fe700 1 --2- 172.21.15.7:0/3910577620 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3f5015ab00 0x7f3f5015d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4dffb700 1 -- 172.21.15.7:0/3910577620 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147532432 0 0) 0x7f3f5015dea0 con 0x7f3f5006aad0 2024-09-13T18:53:06.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4dffb700 1 -- 172.21.15.7:0/3910577620 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f380035d0 con 0x7f3f5006aad0 2024-09-13T18:53:06.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4e7fc700 1 -- 172.21.15.7:0/3910577620 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3f5015ab00 msgr2=0x7f3f5015d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:06.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4e7fc700 1 --2- 172.21.15.7:0/3910577620 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3f5015ab00 0x7f3f5015d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4e7fc700 1 -- 172.21.15.7:0/3910577620 >> v1:172.21.15.7:6789/0 conn(0x7f3f5006aad0 legacy=0x7f3f50071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4e7fc700 1 -- 172.21.15.7:0/3910577620 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5015e110 con 0x7f3f50071990 2024-09-13T18:53:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4dffb700 1 -- 172.21.15.7:0/3910577620 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2654862562 0 0) 0x7f3f380035d0 con 0x7f3f5006aad0 2024-09-13T18:53:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4e7fc700 1 --2- 172.21.15.7:0/3910577620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f50071990 0x7f3f5015d420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009850 tx=0x7f3f40009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d849d4044cd90526 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4dffb700 1 -- 172.21.15.7:0/3910577620 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44001690 con 0x7f3f50071990 2024-09-13T18:53:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4dffb700 1 -- 172.21.15.7:0/3910577620 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f40004030 con 0x7f3f50071990 2024-09-13T18:53:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4dffb700 1 -- 172.21.15.7:0/3910577620 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f400043e0 con 0x7f3f50071990 2024-09-13T18:53:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4ffff700 1 -- 172.21.15.7:0/3910577620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f50071990 msgr2=0x7f3f5015d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4ffff700 1 --2- 172.21.15.7:0/3910577620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f50071990 0x7f3f5015d420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009850 tx=0x7f3f40009b60 comp rx=0 tx=0).stop 2024-09-13T18:53:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4ffff700 1 -- 172.21.15.7:0/3910577620 shutdown_connections 2024-09-13T18:53:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4ffff700 1 --2- 172.21.15.7:0/3910577620 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3f5015ab00 0x7f3f5015d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4ffff700 1 --2- 172.21.15.7:0/3910577620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f50071990 0x7f3f5015d420 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4ffff700 1 -- 172.21.15.7:0/3910577620 >> 172.21.15.7:0/3910577620 conn(0x7f3f500b0d30 msgr2=0x7f3f5006fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4ffff700 1 -- 172.21.15.7:0/3910577620 shutdown_connections 2024-09-13T18:53:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.873+0000 7f3f4ffff700 1 -- 172.21.15.7:0/3910577620 wait complete. 2024-09-13T18:53:06.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4ffff700 1 Processor -- start 2024-09-13T18:53:06.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4ffff700 1 -- start start 2024-09-13T18:53:06.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4ffff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f5006aad0 0x7f3f501f2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4ffff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3f50071990 0x7f3f5014a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f5015ab00 0x7f3f5015b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4ffff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f501471e0 con 0x7f3f5015ab00 2024-09-13T18:53:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4ffff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f50147480 con 0x7f3f5006aad0 2024-09-13T18:53:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4ffff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f50147760 con 0x7f3f50071990 2024-09-13T18:53:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f5015ab00 0x7f3f5015b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f5015ab00 0x7f3f5015b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59984/0 (socket says 172.21.15.7:59984) 2024-09-13T18:53:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4f7fe700 1 -- 172.21.15.7:0/66824913 learned_addr learned my addr 172.21.15.7:0/66824913 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4e7fc700 1 --2- 172.21.15.7:0/66824913 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3f50071990 0x7f3f5014a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4effd700 1 --2- 172.21.15.7:0/66824913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f5006aad0 0x7f3f501f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4f7fe700 1 -- 172.21.15.7:0/66824913 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3f50071990 msgr2=0x7f3f5014a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4f7fe700 1 --2- 172.21.15.7:0/66824913 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3f50071990 0x7f3f5014a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4f7fe700 1 -- 172.21.15.7:0/66824913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f5006aad0 msgr2=0x7f3f501f2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4f7fe700 1 --2- 172.21.15.7:0/66824913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f5006aad0 0x7f3f501f2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4f7fe700 1 -- 172.21.15.7:0/66824913 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f400094b0 con 0x7f3f5015ab00 2024-09-13T18:53:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4f7fe700 1 --2- 172.21.15.7:0/66824913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f5015ab00 0x7f3f5015b8c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800efc0 tx=0x7f3f4800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f377fe700 1 -- 172.21.15.7:0/66824913 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48009ea0 con 0x7f3f5015ab00 2024-09-13T18:53:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f377fe700 1 -- 172.21.15.7:0/66824913 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4800f040 con 0x7f3f5015ab00 2024-09-13T18:53:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4ffff700 1 -- 172.21.15.7:0/66824913 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f50147a40 con 0x7f3f5015ab00 2024-09-13T18:53:06.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f377fe700 1 -- 172.21.15.7:0/66824913 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4800ced0 con 0x7f3f5015ab00 2024-09-13T18:53:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.877+0000 7f3f4ffff700 1 -- 172.21.15.7:0/66824913 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f5014a960 con 0x7f3f5015ab00 2024-09-13T18:53:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.881+0000 7f3f4ffff700 1 -- 172.21.15.7:0/66824913 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f50071280 con 0x7f3f5015ab00 2024-09-13T18:53:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.881+0000 7f3f377fe700 1 -- 172.21.15.7:0/66824913 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3f48014480 con 0x7f3f5015ab00 2024-09-13T18:53:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.881+0000 7f3f377fe700 1 --2- 172.21.15.7:0/66824913 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3f3c066730 0x7f3f3c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.881+0000 7f3f377fe700 1 -- 172.21.15.7:0/66824913 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3f48085120 con 0x7f3f5015ab00 2024-09-13T18:53:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.881+0000 7f3f4effd700 1 --2- 172.21.15.7:0/66824913 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3f3c066730 0x7f3f3c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.881+0000 7f3f4effd700 1 --2- 172.21.15.7:0/66824913 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3f3c066730 0x7f3f3c068bf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3f44001200 tx=0x7f3f44007ba0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:06.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:06.889+0000 7f3f377fe700 1 -- 172.21.15.7:0/66824913 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3f48052460 con 0x7f3f5015ab00 2024-09-13T18:53:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.061+0000 7f3f4ffff700 1 -- 172.21.15.7:0/66824913 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f50149280 con 0x7f3f5015ab00 2024-09-13T18:53:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.065+0000 7f3f377fe700 1 -- 172.21.15.7:0/66824913 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16707 (secure 0 0 0) 0x7f3f48018070 con 0x7f3f5015ab00 2024-09-13T18:53:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:07.071 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":23,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:53:06.403335+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:52:15.554683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T18:52:16.896879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:53:07.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.069+0000 7f3f4ffff700 1 -- 172.21.15.7:0/66824913 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3f3c066730 msgr2=0x7f3f3c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.069+0000 7f3f4ffff700 1 --2- 172.21.15.7:0/66824913 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3f3c066730 0x7f3f3c068bf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3f44001200 tx=0x7f3f44007ba0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.069+0000 7f3f4ffff700 1 -- 172.21.15.7:0/66824913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f5015ab00 msgr2=0x7f3f5015b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.069+0000 7f3f4ffff700 1 --2- 172.21.15.7:0/66824913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f5015ab00 0x7f3f5015b8c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800efc0 tx=0x7f3f4800c3a0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.069+0000 7f3f4ffff700 1 -- 172.21.15.7:0/66824913 shutdown_connections 2024-09-13T18:53:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.069+0000 7f3f4ffff700 1 --2- 172.21.15.7:0/66824913 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3f3c066730 0x7f3f3c068bf0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.069+0000 7f3f4ffff700 1 --2- 172.21.15.7:0/66824913 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f5015ab00 0x7f3f5015b8c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.069+0000 7f3f4ffff700 1 --2- 172.21.15.7:0/66824913 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3f50071990 0x7f3f5014a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.069+0000 7f3f4ffff700 1 --2- 172.21.15.7:0/66824913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3f5006aad0 0x7f3f501f2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.069+0000 7f3f4ffff700 1 -- 172.21.15.7:0/66824913 >> 172.21.15.7:0/66824913 conn(0x7f3f500b0d30 msgr2=0x7f3f500b1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.069+0000 7f3f4ffff700 1 -- 172.21.15.7:0/66824913 shutdown_connections 2024-09-13T18:53:07.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.069+0000 7f3f4ffff700 1 -- 172.21.15.7:0/66824913 wait complete. 2024-09-13T18:53:07.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T18:53:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.201+0000 7fe7c3fff700 1 Processor -- start 2024-09-13T18:53:07.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.201+0000 7fe7c3fff700 1 -- start start 2024-09-13T18:53:07.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7c3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe7c415a930 0x7fe7c4071d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:07.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7c3fff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fe7c40722b0 0x7fe7c40cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7c3fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c415ad10 con 0x7fe7c40cd960 2024-09-13T18:53:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7c3fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c40cd0a0 con 0x7fe7c415a930 2024-09-13T18:53:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7c3fff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c40cdd90 con 0x7fe7c40722b0 2024-09-13T18:53:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7c2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe7c415a930 0x7fe7c4071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7c37fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe7c40cd960 0x7fe7c406aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47618/0 (socket says 172.21.15.7:47618) 2024-09-13T18:53:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7c37fe700 1 -- 172.21.15.7:0/1744380599 learned_addr learned my addr 172.21.15.7:0/1744380599 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7ba7fc700 1 --2- 172.21.15.7:0/1744380599 >> v2:172.21.15.171:3300/0 conn(0x7fe7c40722b0 0x7fe7c40cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7c27fc700 1 -- 172.21.15.7:0/1744380599 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199038134 0 0) 0x7fe7c415ad10 con 0x7fe7c40cd960 2024-09-13T18:53:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7c27fc700 1 -- 172.21.15.7:0/1744380599 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7b00035d0 con 0x7fe7c40cd960 2024-09-13T18:53:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7ba7fc700 1 -- 172.21.15.7:0/1744380599 >> v2:172.21.15.7:3300/0 conn(0x7fe7c415a930 msgr2=0x7fe7c4071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7ba7fc700 1 --2- 172.21.15.7:0/1744380599 >> v2:172.21.15.7:3300/0 conn(0x7fe7c415a930 0x7fe7c4071d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7ba7fc700 1 -- 172.21.15.7:0/1744380599 >> v1:172.21.15.7:6789/0 conn(0x7fe7c40cd960 legacy=0x7fe7c406aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:07.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7ba7fc700 1 -- 172.21.15.7:0/1744380599 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c40718e0 con 0x7fe7c40722b0 2024-09-13T18:53:07.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7c2ffd700 1 --2- 172.21.15.7:0/1744380599 >> v2:172.21.15.7:3300/0 conn(0x7fe7c415a930 0x7fe7c4071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:07.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7ba7fc700 1 --2- 172.21.15.7:0/1744380599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7c40722b0 0x7fe7c40cd420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4072870 tx=0x7fe7ac009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aecf6f792e03e47a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:07.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7c27fc700 1 -- 172.21.15.7:0/1744380599 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac011830 con 0x7fe7c40722b0 2024-09-13T18:53:07.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7c27fc700 1 -- 172.21.15.7:0/1744380599 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ac011990 con 0x7fe7c40722b0 2024-09-13T18:53:07.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.205+0000 7fe7c27fc700 1 -- 172.21.15.7:0/1744380599 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac011d00 con 0x7fe7c40722b0 2024-09-13T18:53:07.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.209+0000 7fe7c3fff700 1 -- 172.21.15.7:0/1744380599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7c40722b0 msgr2=0x7fe7c40cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:07.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.209+0000 7fe7c3fff700 1 --2- 172.21.15.7:0/1744380599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7c40722b0 0x7fe7c40cd420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4072870 tx=0x7fe7ac009b30 comp rx=0 tx=0).stop 2024-09-13T18:53:07.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.209+0000 7fe7c3fff700 1 -- 172.21.15.7:0/1744380599 shutdown_connections 2024-09-13T18:53:07.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.209+0000 7fe7c3fff700 1 --2- 172.21.15.7:0/1744380599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7c40722b0 0x7fe7c40cd420 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.209+0000 7fe7c3fff700 1 --2- 172.21.15.7:0/1744380599 >> v2:172.21.15.7:3300/0 conn(0x7fe7c415a930 0x7fe7c4071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.209+0000 7fe7c3fff700 1 -- 172.21.15.7:0/1744380599 >> 172.21.15.7:0/1744380599 conn(0x7fe7c40b1810 msgr2=0x7fe7c40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:07.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.209+0000 7fe7c3fff700 1 -- 172.21.15.7:0/1744380599 shutdown_connections 2024-09-13T18:53:07.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.209+0000 7fe7c3fff700 1 -- 172.21.15.7:0/1744380599 wait complete. 2024-09-13T18:53:07.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.209+0000 7fe7c3fff700 1 Processor -- start 2024-09-13T18:53:07.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.209+0000 7fe7c3fff700 1 -- start start 2024-09-13T18:53:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7c3fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe7c40722b0 0x7fe7c41e4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7c40cd960 0x7fe7c414fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7c3fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7c415a930 0x7fe7c41502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7c3fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c41e4cd0 con 0x7fe7c40cd960 2024-09-13T18:53:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7c3fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c41e4f00 con 0x7fe7c415a930 2024-09-13T18:53:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7c3fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c41e51e0 con 0x7fe7c40722b0 2024-09-13T18:53:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7ba7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7c40cd960 0x7fe7c414fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7ba7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7c40cd960 0x7fe7c414fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60006/0 (socket says 172.21.15.7:60006) 2024-09-13T18:53:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7ba7fc700 1 -- 172.21.15.7:0/2060909025 learned_addr learned my addr 172.21.15.7:0/2060909025 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7c37fe700 1 --2- 172.21.15.7:0/2060909025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7c415a930 0x7fe7c41502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7c2ffd700 1 --2- 172.21.15.7:0/2060909025 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe7c40722b0 0x7fe7c41e4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7ba7fc700 1 -- 172.21.15.7:0/2060909025 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe7c40722b0 msgr2=0x7fe7c41e4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7ba7fc700 1 --2- 172.21.15.7:0/2060909025 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe7c40722b0 0x7fe7c41e4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7ba7fc700 1 -- 172.21.15.7:0/2060909025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7c415a930 msgr2=0x7fe7c41502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7ba7fc700 1 --2- 172.21.15.7:0/2060909025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7c415a930 0x7fe7c41502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7ba7fc700 1 -- 172.21.15.7:0/2060909025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b40094b0 con 0x7fe7c40cd960 2024-09-13T18:53:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7ba7fc700 1 --2- 172.21.15.7:0/2060909025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7c40cd960 0x7fe7c414fdb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0090b0 tx=0x7fe7ac004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7c0ff9700 1 -- 172.21.15.7:0/2060909025 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac0044d0 con 0x7fe7c40cd960 2024-09-13T18:53:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7c0ff9700 1 -- 172.21.15.7:0/2060909025 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ac01e070 con 0x7fe7c40cd960 2024-09-13T18:53:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7c3fff700 1 -- 172.21.15.7:0/2060909025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ac0094b0 con 0x7fe7c40cd960 2024-09-13T18:53:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7c0ff9700 1 -- 172.21.15.7:0/2060909025 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac003900 con 0x7fe7c40cd960 2024-09-13T18:53:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.213+0000 7fe7c3fff700 1 -- 172.21.15.7:0/2060909025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c41e57a0 con 0x7fe7c40cd960 2024-09-13T18:53:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.217+0000 7fe7c3fff700 1 -- 172.21.15.7:0/2060909025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c40cd420 con 0x7fe7c40cd960 2024-09-13T18:53:07.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.217+0000 7fe7c0ff9700 1 -- 172.21.15.7:0/2060909025 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe7ac003a60 con 0x7fe7c40cd960 2024-09-13T18:53:07.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.217+0000 7fe7c0ff9700 1 --2- 172.21.15.7:0/2060909025 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe7a4066730 0x7fe7a4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:07.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.217+0000 7fe7c0ff9700 1 -- 172.21.15.7:0/2060909025 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe7ac084ec0 con 0x7fe7c40cd960 2024-09-13T18:53:07.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.217+0000 7fe7c2ffd700 1 --2- 172.21.15.7:0/2060909025 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe7a4066730 0x7fe7a4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:07.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.217+0000 7fe7c2ffd700 1 --2- 172.21.15.7:0/2060909025 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe7a4066730 0x7fe7a4068bf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4074710 tx=0x7fe7b4009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.229+0000 7fe7c0ff9700 1 -- 172.21.15.7:0/2060909025 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe7ac0521a0 con 0x7fe7c40cd960 2024-09-13T18:53:07.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.441+0000 7fe7c3fff700 1 -- 172.21.15.7:0/2060909025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe7c415dd40 con 0x7fe7c40cd960 2024-09-13T18:53:07.447 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-13T18:53:07.441+0000 7f28f0ff8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T18:53:07.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.453+0000 7fe7c0ff9700 1 -- 172.21.15.7:0/2060909025 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe7ac0557c0 con 0x7fe7c40cd960 2024-09-13T18:53:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.457+0000 7fe7c3fff700 1 -- 172.21.15.7:0/2060909025 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe7a4066730 msgr2=0x7fe7a4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.457+0000 7fe7c3fff700 1 --2- 172.21.15.7:0/2060909025 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe7a4066730 0x7fe7a4068bf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4074710 tx=0x7fe7b4009240 comp rx=0 tx=0).stop 2024-09-13T18:53:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.457+0000 7fe7c3fff700 1 -- 172.21.15.7:0/2060909025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7c40cd960 msgr2=0x7fe7c414fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:07.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.457+0000 7fe7c3fff700 1 --2- 172.21.15.7:0/2060909025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7c40cd960 0x7fe7c414fdb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac0090b0 tx=0x7fe7ac004770 comp rx=0 tx=0).stop 2024-09-13T18:53:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.457+0000 7fe7c3fff700 1 -- 172.21.15.7:0/2060909025 shutdown_connections 2024-09-13T18:53:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.457+0000 7fe7c3fff700 1 --2- 172.21.15.7:0/2060909025 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe7a4066730 0x7fe7a4068bf0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.457+0000 7fe7c3fff700 1 --2- 172.21.15.7:0/2060909025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7c415a930 0x7fe7c41502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.457+0000 7fe7c3fff700 1 --2- 172.21.15.7:0/2060909025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7c40cd960 0x7fe7c414fdb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.457+0000 7fe7c3fff700 1 --2- 172.21.15.7:0/2060909025 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe7c40722b0 0x7fe7c41e4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.457+0000 7fe7c3fff700 1 -- 172.21.15.7:0/2060909025 >> 172.21.15.7:0/2060909025 conn(0x7fe7c40b1810 msgr2=0x7fe7c406e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.457+0000 7fe7c3fff700 1 -- 172.21.15.7:0/2060909025 shutdown_connections 2024-09-13T18:53:07.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.457+0000 7fe7c3fff700 1 -- 172.21.15.7:0/2060909025 wait complete. 2024-09-13T18:53:07.464 INFO:tasks.ceph.mds.b.smithi171.stderr: -2980> 2024-09-13T18:52:22.210+0000 7f9587f7c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T18:53:07.464 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T18:53:07.474 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T18:53:07.497 INFO:tasks.ceph.mds.b.smithi171.stdout:starting mds.b at 2024-09-13T18:53:07.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.581+0000 7f2046076700 1 Processor -- start 2024-09-13T18:53:07.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.581+0000 7f2046076700 1 -- start start 2024-09-13T18:53:07.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.581+0000 7f2046076700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f20400cd7c0 0x7f20400cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:07.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.581+0000 7f2046076700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f20400d0ce0 0x7f20400d1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:07.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.581+0000 7f2046076700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20400ce460 con 0x7f204006ac90 2024-09-13T18:53:07.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.581+0000 7f2046076700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20400b5b10 con 0x7f20400d0ce0 2024-09-13T18:53:07.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.581+0000 7f2046076700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20400ce740 con 0x7f20400cd7c0 2024-09-13T18:53:07.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.581+0000 7f2044873700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f20400cd7c0 0x7f20400cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:07.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.581+0000 7f2045875700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f20400d0ce0 0x7f20400d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:07.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.581+0000 7f2044873700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f20400cd7c0 0x7f20400cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:33028/0 (socket says 172.21.15.7:33028) 2024-09-13T18:53:07.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.581+0000 7f2044873700 1 -- 172.21.15.7:0/2654588460 learned_addr learned my addr 172.21.15.7:0/2654588460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:07.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.585+0000 7f2044873700 1 -- 172.21.15.7:0/2654588460 >> v2:172.21.15.171:3300/0 conn(0x7f20400d0ce0 msgr2=0x7f20400d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:07.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.585+0000 7f2044873700 1 --2- 172.21.15.7:0/2654588460 >> v2:172.21.15.171:3300/0 conn(0x7f20400d0ce0 0x7f20400d1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.585+0000 7f2044873700 1 -- 172.21.15.7:0/2654588460 >> v1:172.21.15.171:6789/0 conn(0x7f204006ac90 legacy=0x7f204006b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:07.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.585+0000 7f2044873700 1 -- 172.21.15.7:0/2654588460 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20400cea20 con 0x7f20400cd7c0 2024-09-13T18:53:07.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.585+0000 7f2044873700 1 --2- 172.21.15.7:0/2654588460 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f20400cd7c0 0x7f20400cdc10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f203c009910 tx=0x7f203c009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19e8d65e2300af0c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:07.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.585+0000 7f2037fff700 1 -- 172.21.15.7:0/2654588460 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c011830 con 0x7f20400cd7c0 2024-09-13T18:53:07.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.585+0000 7f2037fff700 1 -- 172.21.15.7:0/2654588460 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203c011990 con 0x7f20400cd7c0 2024-09-13T18:53:07.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.585+0000 7f2037fff700 1 -- 172.21.15.7:0/2654588460 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c011d40 con 0x7f20400cd7c0 2024-09-13T18:53:07.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.585+0000 7f2046076700 1 -- 172.21.15.7:0/2654588460 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f20400cd7c0 msgr2=0x7f20400cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:07.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.585+0000 7f2046076700 1 --2- 172.21.15.7:0/2654588460 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f20400cd7c0 0x7f20400cdc10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f203c009910 tx=0x7f203c009c20 comp rx=0 tx=0).stop 2024-09-13T18:53:07.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.585+0000 7f2046076700 1 -- 172.21.15.7:0/2654588460 shutdown_connections 2024-09-13T18:53:07.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.585+0000 7f2046076700 1 --2- 172.21.15.7:0/2654588460 >> v2:172.21.15.171:3300/0 conn(0x7f20400d0ce0 0x7f20400d1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.585+0000 7f2046076700 1 --2- 172.21.15.7:0/2654588460 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f20400cd7c0 0x7f20400cdc10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.585+0000 7f2046076700 1 -- 172.21.15.7:0/2654588460 >> 172.21.15.7:0/2654588460 conn(0x7f20400d2bc0 msgr2=0x7f20400d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:07.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.585+0000 7f2046076700 1 -- 172.21.15.7:0/2654588460 shutdown_connections 2024-09-13T18:53:07.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.585+0000 7f2046076700 1 -- 172.21.15.7:0/2654588460 wait complete. 2024-09-13T18:53:07.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.589+0000 7f2046076700 1 Processor -- start 2024-09-13T18:53:07.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.589+0000 7f2046076700 1 -- start start 2024-09-13T18:53:07.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.589+0000 7f2046076700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f204006ac90 0x7f2040154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:07.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.589+0000 7f2046076700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20400cd7c0 0x7f20400cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:07.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.589+0000 7f2046076700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20400d0ce0 0x7f204015cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:07.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.589+0000 7f2046076700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2040155410 con 0x7f20400d0ce0 2024-09-13T18:53:07.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.589+0000 7f2046076700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20401e65a0 con 0x7f20400cd7c0 2024-09-13T18:53:07.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.589+0000 7f2046076700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20401e6840 con 0x7f204006ac90 2024-09-13T18:53:07.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.589+0000 7f2045875700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20400d0ce0 0x7f204015cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:07.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.593+0000 7f2045875700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20400d0ce0 0x7f204015cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60018/0 (socket says 172.21.15.7:60018) 2024-09-13T18:53:07.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.593+0000 7f2045875700 1 -- 172.21.15.7:0/2795719916 learned_addr learned my addr 172.21.15.7:0/2795719916 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:07.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.593+0000 7f2044873700 1 --2- 172.21.15.7:0/2795719916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20400cd7c0 0x7f20400cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:07.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.593+0000 7f2045074700 1 --2- 172.21.15.7:0/2795719916 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f204006ac90 0x7f2040154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:07.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.593+0000 7f2045875700 1 -- 172.21.15.7:0/2795719916 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f204006ac90 msgr2=0x7f2040154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.593+0000 7f2045875700 1 --2- 172.21.15.7:0/2795719916 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f204006ac90 0x7f2040154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.593+0000 7f2045875700 1 -- 172.21.15.7:0/2795719916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20400cd7c0 msgr2=0x7f20400cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.593+0000 7f2045875700 1 --2- 172.21.15.7:0/2795719916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20400cd7c0 0x7f20400cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.593+0000 7f2045875700 1 -- 172.21.15.7:0/2795719916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c009570 con 0x7f20400d0ce0 2024-09-13T18:53:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.593+0000 7f2045074700 1 --2- 172.21.15.7:0/2795719916 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f204006ac90 0x7f2040154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.593+0000 7f2045875700 1 --2- 172.21.15.7:0/2795719916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20400d0ce0 0x7f204015cb80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f203800ec10 tx=0x7f203800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.593+0000 7f2035ffb700 1 -- 172.21.15.7:0/2795719916 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203800cab0 con 0x7f20400d0ce0 2024-09-13T18:53:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.593+0000 7f2035ffb700 1 -- 172.21.15.7:0/2795719916 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203800f040 con 0x7f20400d0ce0 2024-09-13T18:53:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.593+0000 7f2046076700 1 -- 172.21.15.7:0/2795719916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20401e6b20 con 0x7f20400d0ce0 2024-09-13T18:53:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.593+0000 7f2035ffb700 1 -- 172.21.15.7:0/2795719916 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203800ce50 con 0x7f20400d0ce0 2024-09-13T18:53:07.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.593+0000 7f2046076700 1 -- 172.21.15.7:0/2795719916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20401e70e0 con 0x7f20400d0ce0 2024-09-13T18:53:07.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.593+0000 7f2046076700 1 -- 172.21.15.7:0/2795719916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20400cdc10 con 0x7f20400d0ce0 2024-09-13T18:53:07.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.597+0000 7f2035ffb700 1 -- 172.21.15.7:0/2795719916 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2038014480 con 0x7f20400d0ce0 2024-09-13T18:53:07.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.597+0000 7f2035ffb700 1 --2- 172.21.15.7:0/2795719916 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2024066730 0x7f2024068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:07.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.597+0000 7f2035ffb700 1 -- 172.21.15.7:0/2795719916 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f20380849c0 con 0x7f20400d0ce0 2024-09-13T18:53:07.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.597+0000 7f2045074700 1 --2- 172.21.15.7:0/2795719916 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2024066730 0x7f2024068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:07.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.597+0000 7f2045074700 1 --2- 172.21.15.7:0/2795719916 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2024066730 0x7f2024068bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2030000f60 tx=0x7f2030009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:07.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.605+0000 7f2035ffb700 1 -- 172.21.15.7:0/2795719916 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2038051c90 con 0x7f20400d0ce0 2024-09-13T18:53:07.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:07.813+0000 7f2046076700 1 -- 172.21.15.7:0/2795719916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f20400b8b40 con 0x7f20400d0ce0 2024-09-13T18:53:08.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.460+0000 7f2035ffb700 1 -- 172.21.15.7:0/2795719916 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f20380180a0 con 0x7f20400d0ce0 2024-09-13T18:53:08.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.468+0000 7f2046076700 1 -- 172.21.15.7:0/2795719916 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2024066730 msgr2=0x7f2024068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:08.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.468+0000 7f2046076700 1 --2- 172.21.15.7:0/2795719916 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2024066730 0x7f2024068bf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2030000f60 tx=0x7f2030009040 comp rx=0 tx=0).stop 2024-09-13T18:53:08.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.468+0000 7f2046076700 1 -- 172.21.15.7:0/2795719916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20400d0ce0 msgr2=0x7f204015cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:08.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.468+0000 7f2046076700 1 --2- 172.21.15.7:0/2795719916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20400d0ce0 0x7f204015cb80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f203800ec10 tx=0x7f203800ef20 comp rx=0 tx=0).stop 2024-09-13T18:53:08.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.468+0000 7f2046076700 1 -- 172.21.15.7:0/2795719916 shutdown_connections 2024-09-13T18:53:08.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.468+0000 7f2046076700 1 --2- 172.21.15.7:0/2795719916 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2024066730 0x7f2024068bf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:08.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.468+0000 7f2046076700 1 --2- 172.21.15.7:0/2795719916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20400d0ce0 0x7f204015cb80 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:08.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.468+0000 7f2046076700 1 --2- 172.21.15.7:0/2795719916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20400cd7c0 0x7f20400cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:08.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.468+0000 7f2046076700 1 --2- 172.21.15.7:0/2795719916 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f204006ac90 0x7f2040154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:08.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.468+0000 7f2046076700 1 -- 172.21.15.7:0/2795719916 >> 172.21.15.7:0/2795719916 conn(0x7f20400d2bc0 msgr2=0x7f20400d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:08.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.468+0000 7f2046076700 1 -- 172.21.15.7:0/2795719916 shutdown_connections 2024-09-13T18:53:08.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.468+0000 7f2046076700 1 -- 172.21.15.7:0/2795719916 wait complete. 2024-09-13T18:53:08.493 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T18:53:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.612+0000 7f119f59e700 1 Processor -- start 2024-09-13T18:53:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.612+0000 7f119f59e700 1 -- start start 2024-09-13T18:53:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119f59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f11a00b44c0 0x7f11a00b4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119f59e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f11a015d3e0 0x7f11a015d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119f59e700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a00b3420 con 0x7f11a01599e0 2024-09-13T18:53:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119f59e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a015ddb0 con 0x7f11a015d3e0 2024-09-13T18:53:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119f59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00b3700 con 0x7f11a00b44c0 2024-09-13T18:53:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119ed9d700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f11a015d3e0 0x7f11a015d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119dd9b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f11a00b44c0 0x7f11a00b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119e59c700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f11a01599e0 0x7f11a00b29a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:58976/0 (socket says 172.21.15.7:58976) 2024-09-13T18:53:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119dd9b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f11a00b44c0 0x7f11a00b4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:33046/0 (socket says 172.21.15.7:33046) 2024-09-13T18:53:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119ed9d700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f11a015d3e0 0x7f11a015d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:48208/0 (socket says 172.21.15.7:48208) 2024-09-13T18:53:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119e59c700 1 -- 172.21.15.7:0/3315285243 learned_addr learned my addr 172.21.15.7:0/3315285243 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119dd9b700 1 -- 172.21.15.7:0/3315285243 >> v2:172.21.15.171:3300/0 conn(0x7f11a015d3e0 msgr2=0x7f11a015d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119dd9b700 1 --2- 172.21.15.7:0/3315285243 >> v2:172.21.15.171:3300/0 conn(0x7f11a015d3e0 0x7f11a015d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119dd9b700 1 -- 172.21.15.7:0/3315285243 >> v1:172.21.15.171:6789/0 conn(0x7f11a01599e0 legacy=0x7f11a00b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119dd9b700 1 -- 172.21.15.7:0/3315285243 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a00b39e0 con 0x7f11a00b44c0 2024-09-13T18:53:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119dd9b700 1 --2- 172.21.15.7:0/3315285243 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f11a00b44c0 0x7f11a00b4900 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1188009850 tx=0x7f1188009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4649657960258eab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119d59a700 1 -- 172.21.15.7:0/3315285243 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188011830 con 0x7f11a00b44c0 2024-09-13T18:53:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119d59a700 1 -- 172.21.15.7:0/3315285243 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1188011990 con 0x7f11a00b44c0 2024-09-13T18:53:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119d59a700 1 -- 172.21.15.7:0/3315285243 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1188011d40 con 0x7f11a00b44c0 2024-09-13T18:53:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119f59e700 1 -- 172.21.15.7:0/3315285243 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f11a00b44c0 msgr2=0x7f11a00b4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119f59e700 1 --2- 172.21.15.7:0/3315285243 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f11a00b44c0 0x7f11a00b4900 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1188009850 tx=0x7f1188009b60 comp rx=0 tx=0).stop 2024-09-13T18:53:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119f59e700 1 -- 172.21.15.7:0/3315285243 shutdown_connections 2024-09-13T18:53:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119f59e700 1 --2- 172.21.15.7:0/3315285243 >> v2:172.21.15.171:3300/0 conn(0x7f11a015d3e0 0x7f11a015d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119f59e700 1 --2- 172.21.15.7:0/3315285243 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f11a00b44c0 0x7f11a00b4900 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119f59e700 1 -- 172.21.15.7:0/3315285243 >> 172.21.15.7:0/3315285243 conn(0x7f11a00cf8f0 msgr2=0x7f11a00cfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119f59e700 1 -- 172.21.15.7:0/3315285243 shutdown_connections 2024-09-13T18:53:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.616+0000 7f119f59e700 1 -- 172.21.15.7:0/3315285243 wait complete. 2024-09-13T18:53:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.620+0000 7f119f59e700 1 Processor -- start 2024-09-13T18:53:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.620+0000 7f119f59e700 1 -- start start 2024-09-13T18:53:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.620+0000 7f119f59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f11a00b44c0 0x7f11a0156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.620+0000 7f119f59e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11a01599e0 0x7f11a014d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.620+0000 7f119f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11a015d3e0 0x7f11a014da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f119f59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00cdda0 con 0x7f11a015d3e0 2024-09-13T18:53:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f119f59e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00cf090 con 0x7f11a01599e0 2024-09-13T18:53:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f119f59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00cdff0 con 0x7f11a00b44c0 2024-09-13T18:53:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f119ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11a015d3e0 0x7f11a014da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f119ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11a015d3e0 0x7f11a014da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60030/0 (socket says 172.21.15.7:60030) 2024-09-13T18:53:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f119dd9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11a01599e0 0x7f11a014d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f119ed9d700 1 -- 172.21.15.7:0/2015709893 learned_addr learned my addr 172.21.15.7:0/2015709893 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f119e59c700 1 --2- 172.21.15.7:0/2015709893 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f11a00b44c0 0x7f11a0156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:08.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f119ed9d700 1 -- 172.21.15.7:0/2015709893 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f11a00b44c0 msgr2=0x7f11a0156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:08.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f119ed9d700 1 --2- 172.21.15.7:0/2015709893 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f11a00b44c0 0x7f11a0156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f119ed9d700 1 -- 172.21.15.7:0/2015709893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11a01599e0 msgr2=0x7f11a014d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f119ed9d700 1 --2- 172.21.15.7:0/2015709893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11a01599e0 0x7f11a014d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f119ed9d700 1 -- 172.21.15.7:0/2015709893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11880094b0 con 0x7f11a015d3e0 2024-09-13T18:53:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f119ed9d700 1 --2- 172.21.15.7:0/2015709893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11a015d3e0 0x7f11a014da20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1198010fc0 tx=0x7f119800e3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f1196ffd700 1 -- 172.21.15.7:0/2015709893 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119800e940 con 0x7f11a015d3e0 2024-09-13T18:53:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f1196ffd700 1 -- 172.21.15.7:0/2015709893 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1198011040 con 0x7f11a015d3e0 2024-09-13T18:53:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f1196ffd700 1 -- 172.21.15.7:0/2015709893 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119800ece0 con 0x7f11a015d3e0 2024-09-13T18:53:08.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f119f59e700 1 -- 172.21.15.7:0/2015709893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a00ce2d0 con 0x7f11a015d3e0 2024-09-13T18:53:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f119f59e700 1 -- 172.21.15.7:0/2015709893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a00ce890 con 0x7f11a015d3e0 2024-09-13T18:53:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.624+0000 7f119f59e700 1 -- 172.21.15.7:0/2015709893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a015d870 con 0x7f11a015d3e0 2024-09-13T18:53:08.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.628+0000 7f1196ffd700 1 -- 172.21.15.7:0/2015709893 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1198009540 con 0x7f11a015d3e0 2024-09-13T18:53:08.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.628+0000 7f1196ffd700 1 --2- 172.21.15.7:0/2015709893 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1180066730 0x7f1180068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:08.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.628+0000 7f1196ffd700 1 -- 172.21.15.7:0/2015709893 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f11980867b0 con 0x7f11a015d3e0 2024-09-13T18:53:08.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.628+0000 7f119e59c700 1 --2- 172.21.15.7:0/2015709893 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1180066730 0x7f1180068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:08.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.628+0000 7f119e59c700 1 --2- 172.21.15.7:0/2015709893 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1180066730 0x7f1180068bf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1190000f60 tx=0x7f1190007f30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.632+0000 7f1196ffd700 1 -- 172.21.15.7:0/2015709893 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1198053a50 con 0x7f11a015d3e0 2024-09-13T18:53:08.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:08.808+0000 7f119f59e700 1 -- 172.21.15.7:0/2015709893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f11a00cbba0 con 0x7f11a015d3e0 2024-09-13T18:53:09.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.468+0000 7f1196ffd700 1 -- 172.21.15.7:0/2015709893 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1198057070 con 0x7f11a015d3e0 2024-09-13T18:53:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.476+0000 7f119f59e700 1 -- 172.21.15.7:0/2015709893 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1180066730 msgr2=0x7f1180068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.476+0000 7f119f59e700 1 --2- 172.21.15.7:0/2015709893 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1180066730 0x7f1180068bf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1190000f60 tx=0x7f1190007f30 comp rx=0 tx=0).stop 2024-09-13T18:53:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.476+0000 7f119f59e700 1 -- 172.21.15.7:0/2015709893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11a015d3e0 msgr2=0x7f11a014da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.476+0000 7f119f59e700 1 --2- 172.21.15.7:0/2015709893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11a015d3e0 0x7f11a014da20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1198010fc0 tx=0x7f119800e3a0 comp rx=0 tx=0).stop 2024-09-13T18:53:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.476+0000 7f119f59e700 1 -- 172.21.15.7:0/2015709893 shutdown_connections 2024-09-13T18:53:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.476+0000 7f119f59e700 1 --2- 172.21.15.7:0/2015709893 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1180066730 0x7f1180068bf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.476+0000 7f119f59e700 1 --2- 172.21.15.7:0/2015709893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11a015d3e0 0x7f11a014da20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.476+0000 7f119f59e700 1 --2- 172.21.15.7:0/2015709893 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f11a01599e0 0x7f11a014d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.476+0000 7f119f59e700 1 --2- 172.21.15.7:0/2015709893 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f11a00b44c0 0x7f11a0156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.476+0000 7f119f59e700 1 -- 172.21.15.7:0/2015709893 >> 172.21.15.7:0/2015709893 conn(0x7f11a00cf8f0 msgr2=0x7f11a006e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.476+0000 7f119f59e700 1 -- 172.21.15.7:0/2015709893 shutdown_connections 2024-09-13T18:53:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.476+0000 7f119f59e700 1 -- 172.21.15.7:0/2015709893 wait complete. 2024-09-13T18:53:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-09-13T18:53:09.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T18:53:09.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.608+0000 7fd63d6af700 1 Processor -- start 2024-09-13T18:53:09.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.608+0000 7fd63d6af700 1 -- start start 2024-09-13T18:53:09.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.608+0000 7fd63d6af700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd6380ce710 0x7fd6380b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:09.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.608+0000 7fd63d6af700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd638160d90 0x7fd6381611c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:09.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.608+0000 7fd63d6af700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63806aa30 con 0x7fd6380ceaf0 2024-09-13T18:53:09.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.608+0000 7fd63d6af700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6380d0660 con 0x7fd638160d90 2024-09-13T18:53:09.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.608+0000 7fd63d6af700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6380b1e60 con 0x7fd6380ce710 2024-09-13T18:53:09.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.608+0000 7fd6377fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd6380ceaf0 0x7fd6380cff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47620/0 (socket says 172.21.15.7:47620) 2024-09-13T18:53:09.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.608+0000 7fd6377fe700 1 -- 172.21.15.7:0/302559346 learned_addr learned my addr 172.21.15.7:0/302559346 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:09.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.608+0000 7fd63ceae700 1 --2- 172.21.15.7:0/302559346 >> v2:172.21.15.171:3300/0 conn(0x7fd638160d90 0x7fd6381611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:09.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.608+0000 7fd637fff700 1 --2- 172.21.15.7:0/302559346 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd6380ce710 0x7fd6380b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:09.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.608+0000 7fd636ffd700 1 -- 172.21.15.7:0/302559346 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329826683 0 0) 0x7fd63806aa30 con 0x7fd6380ceaf0 2024-09-13T18:53:09.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.608+0000 7fd636ffd700 1 -- 172.21.15.7:0/302559346 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6200035d0 con 0x7fd6380ceaf0 2024-09-13T18:53:09.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.608+0000 7fd637fff700 1 -- 172.21.15.7:0/302559346 >> v2:172.21.15.171:3300/0 conn(0x7fd638160d90 msgr2=0x7fd6381611c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:09.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.608+0000 7fd637fff700 1 --2- 172.21.15.7:0/302559346 >> v2:172.21.15.171:3300/0 conn(0x7fd638160d90 0x7fd6381611c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:09.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.608+0000 7fd637fff700 1 -- 172.21.15.7:0/302559346 >> v1:172.21.15.7:6789/0 conn(0x7fd6380ceaf0 legacy=0x7fd6380cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:09.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.608+0000 7fd637fff700 1 -- 172.21.15.7:0/302559346 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6380b18a0 con 0x7fd6380ce710 2024-09-13T18:53:09.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd63ceae700 1 --2- 172.21.15.7:0/302559346 >> v2:172.21.15.171:3300/0 conn(0x7fd638160d90 0x7fd6381611c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:09.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd637fff700 1 --2- 172.21.15.7:0/302559346 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd6380ce710 0x7fd6380b5da0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd628009850 tx=0x7fd628009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f84d79016b548fc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:09.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd636ffd700 1 -- 172.21.15.7:0/302559346 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628011830 con 0x7fd6380ce710 2024-09-13T18:53:09.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd636ffd700 1 -- 172.21.15.7:0/302559346 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd628011990 con 0x7fd6380ce710 2024-09-13T18:53:09.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd636ffd700 1 -- 172.21.15.7:0/302559346 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628011d00 con 0x7fd6380ce710 2024-09-13T18:53:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd63d6af700 1 -- 172.21.15.7:0/302559346 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd6380ce710 msgr2=0x7fd6380b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd63d6af700 1 --2- 172.21.15.7:0/302559346 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd6380ce710 0x7fd6380b5da0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd628009850 tx=0x7fd628009b60 comp rx=0 tx=0).stop 2024-09-13T18:53:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd63d6af700 1 -- 172.21.15.7:0/302559346 shutdown_connections 2024-09-13T18:53:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd63d6af700 1 --2- 172.21.15.7:0/302559346 >> v2:172.21.15.171:3300/0 conn(0x7fd638160d90 0x7fd6381611c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd63d6af700 1 --2- 172.21.15.7:0/302559346 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd6380ce710 0x7fd6380b5da0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd63d6af700 1 -- 172.21.15.7:0/302559346 >> 172.21.15.7:0/302559346 conn(0x7fd6380d3260 msgr2=0x7fd6380cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd63d6af700 1 -- 172.21.15.7:0/302559346 shutdown_connections 2024-09-13T18:53:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd63d6af700 1 -- 172.21.15.7:0/302559346 wait complete. 2024-09-13T18:53:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd63d6af700 1 Processor -- start 2024-09-13T18:53:09.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd63d6af700 1 -- start start 2024-09-13T18:53:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd63d6af700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6380ce710 0x7fd6380ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd63d6af700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6380ceaf0 0x7fd638159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd63d6af700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd638160d90 0x7fd6381e6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd63d6af700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6380cac50 con 0x7fd6380ceaf0 2024-09-13T18:53:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd63d6af700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6380cb850 con 0x7fd6380ce710 2024-09-13T18:53:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.612+0000 7fd63d6af700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6380cbb30 con 0x7fd638160d90 2024-09-13T18:53:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd6377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6380ceaf0 0x7fd638159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd6377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6380ceaf0 0x7fd638159910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60040/0 (socket says 172.21.15.7:60040) 2024-09-13T18:53:09.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd6377fe700 1 -- 172.21.15.7:0/2780087993 learned_addr learned my addr 172.21.15.7:0/2780087993 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd637fff700 1 --2- 172.21.15.7:0/2780087993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6380ce710 0x7fd6380ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd63ceae700 1 --2- 172.21.15.7:0/2780087993 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd638160d90 0x7fd6381e6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd6377fe700 1 -- 172.21.15.7:0/2780087993 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd638160d90 msgr2=0x7fd6381e6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd6377fe700 1 --2- 172.21.15.7:0/2780087993 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd638160d90 0x7fd6381e6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd6377fe700 1 -- 172.21.15.7:0/2780087993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6380ce710 msgr2=0x7fd6380ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd6377fe700 1 --2- 172.21.15.7:0/2780087993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6380ce710 0x7fd6380ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd6377fe700 1 -- 172.21.15.7:0/2780087993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd630009a70 con 0x7fd6380ceaf0 2024-09-13T18:53:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd63ceae700 1 --2- 172.21.15.7:0/2780087993 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd638160d90 0x7fd6381e6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd637fff700 1 --2- 172.21.15.7:0/2780087993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6380ce710 0x7fd6380ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:09.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd6377fe700 1 --2- 172.21.15.7:0/2780087993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6380ceaf0 0x7fd638159910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00c4f0 tx=0x7fd62c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd634ff9700 1 -- 172.21.15.7:0/2780087993 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c005d90 con 0x7fd6380ceaf0 2024-09-13T18:53:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd634ff9700 1 -- 172.21.15.7:0/2780087993 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62c0070d0 con 0x7fd6380ceaf0 2024-09-13T18:53:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd63d6af700 1 -- 172.21.15.7:0/2780087993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6280094b0 con 0x7fd6380ceaf0 2024-09-13T18:53:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd634ff9700 1 -- 172.21.15.7:0/2780087993 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c007440 con 0x7fd6380ceaf0 2024-09-13T18:53:09.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd63d6af700 1 -- 172.21.15.7:0/2780087993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6380cc0f0 con 0x7fd6380ceaf0 2024-09-13T18:53:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.616+0000 7fd63d6af700 1 -- 172.21.15.7:0/2780087993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6380b5da0 con 0x7fd6380ceaf0 2024-09-13T18:53:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.620+0000 7fd634ff9700 1 -- 172.21.15.7:0/2780087993 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd62c0075a0 con 0x7fd6380ceaf0 2024-09-13T18:53:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.620+0000 7fd634ff9700 1 --2- 172.21.15.7:0/2780087993 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd624066730 0x7fd624068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:09.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.620+0000 7fd634ff9700 1 -- 172.21.15.7:0/2780087993 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5473+0+0 (secure 0 0 0) 0x7fd62c0834b0 con 0x7fd6380ceaf0 2024-09-13T18:53:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.620+0000 7fd637fff700 1 --2- 172.21.15.7:0/2780087993 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd624066730 0x7fd624068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:09.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.620+0000 7fd637fff700 1 --2- 172.21.15.7:0/2780087993 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd624066730 0x7fd624068bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd6280090b0 tx=0x7fd628008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:09.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.624+0000 7fd634ff9700 1 -- 172.21.15.7:0/2780087993 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd62c0509b0 con 0x7fd6380ceaf0 2024-09-13T18:53:09.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:09.796+0000 7fd63d6af700 1 -- 172.21.15.7:0/2780087993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd63806c1f0 con 0x7fd6380ceaf0 2024-09-13T18:53:10.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.476+0000 7fd634ff9700 1 -- 172.21.15.7:0/2780087993 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd62c053fd0 con 0x7fd6380ceaf0 2024-09-13T18:53:10.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.484+0000 7fd63d6af700 1 -- 172.21.15.7:0/2780087993 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd624066730 msgr2=0x7fd624068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:10.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.484+0000 7fd63d6af700 1 --2- 172.21.15.7:0/2780087993 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd624066730 0x7fd624068bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd6280090b0 tx=0x7fd628008030 comp rx=0 tx=0).stop 2024-09-13T18:53:10.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.484+0000 7fd63d6af700 1 -- 172.21.15.7:0/2780087993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6380ceaf0 msgr2=0x7fd638159910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:10.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.484+0000 7fd63d6af700 1 --2- 172.21.15.7:0/2780087993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6380ceaf0 0x7fd638159910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00c4f0 tx=0x7fd62c00c8b0 comp rx=0 tx=0).stop 2024-09-13T18:53:10.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.484+0000 7fd63d6af700 1 -- 172.21.15.7:0/2780087993 shutdown_connections 2024-09-13T18:53:10.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.484+0000 7fd63d6af700 1 --2- 172.21.15.7:0/2780087993 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd624066730 0x7fd624068bf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:10.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.484+0000 7fd63d6af700 1 --2- 172.21.15.7:0/2780087993 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd638160d90 0x7fd6381e6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:10.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.484+0000 7fd63d6af700 1 --2- 172.21.15.7:0/2780087993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6380ceaf0 0x7fd638159910 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:10.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.484+0000 7fd63d6af700 1 --2- 172.21.15.7:0/2780087993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6380ce710 0x7fd6380ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:10.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.484+0000 7fd63d6af700 1 -- 172.21.15.7:0/2780087993 >> 172.21.15.7:0/2780087993 conn(0x7fd6380d3260 msgr2=0x7fd6381e8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:10.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.484+0000 7fd63d6af700 1 -- 172.21.15.7:0/2780087993 shutdown_connections 2024-09-13T18:53:10.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.484+0000 7fd63d6af700 1 -- 172.21.15.7:0/2780087993 wait complete. 2024-09-13T18:53:10.492 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-09-13T18:53:10.505 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T18:53:10.505 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T18:53:10.505 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T18:53:10.505 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:53:10.505 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T18:53:10.505 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-13T18:53:10.506 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T18:53:10.506 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:53:10.506 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T18:53:10.506 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T18:53:10.506 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T18:53:10.506 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:53:10.506 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T18:53:10.506 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-13T18:53:10.507 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T18:53:10.507 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:53:10.518 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T18:53:10.518 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T18:53:10.519 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T18:53:10.519 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T18:53:10.520 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T18:53:10.520 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T18:53:10.522 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T18:53:10.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T18:53:10.523 INFO:tasks.ceph.mds.a:Started 2024-09-13T18:53:10.524 DEBUG:teuthology.parallel:result is None 2024-09-13T18:53:10.524 INFO:tasks.ceph.mds.c:Started 2024-09-13T18:53:10.525 DEBUG:teuthology.parallel:result is None 2024-09-13T18:53:10.562 INFO:tasks.ceph.mds.d:Started 2024-09-13T18:53:10.563 DEBUG:teuthology.parallel:result is None 2024-09-13T18:53:10.563 INFO:tasks.ceph.mds.b:Started 2024-09-13T18:53:10.563 DEBUG:teuthology.parallel:result is None 2024-09-13T18:53:10.564 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T18:53:10.599 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-13T18:53:10.613 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-09-13T18:53:10.653 INFO:tasks.ceph.mds.d.smithi171.stdout:starting mds.d at 2024-09-13T18:53:10.660 INFO:tasks.ceph.mds.b.smithi171.stdout:starting mds.b at 2024-09-13T18:53:10.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.676+0000 7f8adb59e700 1 Processor -- start 2024-09-13T18:53:10.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.676+0000 7f8adb59e700 1 -- start start 2024-09-13T18:53:10.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.676+0000 7f8adb59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc159950 0x7f8adc0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:10.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.676+0000 7f8adb59e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f8adc0cc3c0 0x7f8adc071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:10.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.676+0000 7f8adb59e700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8adc0ce000 con 0x7f8adc071970 2024-09-13T18:53:10.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.676+0000 7f8adb59e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8adc071e20 con 0x7f8adc0cc3c0 2024-09-13T18:53:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.676+0000 7f8adb59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc0cd3f0 con 0x7f8adc159950 2024-09-13T18:53:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.676+0000 7f8ad9d9b700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f8adc0cc3c0 0x7f8adc071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.676+0000 7f8ad9d9b700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f8adc0cc3c0 0x7f8adc071430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:48256/0 (socket says 172.21.15.7:48256) 2024-09-13T18:53:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.676+0000 7f8ad9d9b700 1 -- 172.21.15.7:0/127072297 learned_addr learned my addr 172.21.15.7:0/127072297 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.676+0000 7f8adad9d700 1 --1- 172.21.15.7:0/127072297 >> v1:172.21.15.171:6789/0 conn(0x7f8adc071970 0x7f8adc0cb2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:58994/0 (socket says 172.21.15.7:58994) 2024-09-13T18:53:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.676+0000 7f8ada59c700 1 --2- 172.21.15.7:0/127072297 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc159950 0x7f8adc0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.680+0000 7f8ad9d9b700 1 -- 172.21.15.7:0/127072297 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc159950 msgr2=0x7f8adc0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.680+0000 7f8ad9d9b700 1 --2- 172.21.15.7:0/127072297 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc159950 0x7f8adc0ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.680+0000 7f8ad9d9b700 1 -- 172.21.15.7:0/127072297 >> v1:172.21.15.171:6789/0 conn(0x7f8adc071970 legacy=0x7f8adc0cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.680+0000 7f8ad9d9b700 1 -- 172.21.15.7:0/127072297 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc0cd6d0 con 0x7f8adc0cc3c0 2024-09-13T18:53:10.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.680+0000 7f8ad9d9b700 1 --2- 172.21.15.7:0/127072297 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8adc0cc3c0 0x7f8adc071430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000bbb0 tx=0x7f8ad000bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ae9e98fba23f9f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:10.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.680+0000 7f8ad959a700 1 -- 172.21.15.7:0/127072297 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad000dd30 con 0x7f8adc0cc3c0 2024-09-13T18:53:10.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.680+0000 7f8ad959a700 1 -- 172.21.15.7:0/127072297 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad000de90 con 0x7f8adc0cc3c0 2024-09-13T18:53:10.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.680+0000 7f8ad959a700 1 -- 172.21.15.7:0/127072297 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0010690 con 0x7f8adc0cc3c0 2024-09-13T18:53:10.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.684+0000 7f8adb59e700 1 -- 172.21.15.7:0/127072297 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8adc0cc3c0 msgr2=0x7f8adc071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:10.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.684+0000 7f8adb59e700 1 --2- 172.21.15.7:0/127072297 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8adc0cc3c0 0x7f8adc071430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000bbb0 tx=0x7f8ad000bec0 comp rx=0 tx=0).stop 2024-09-13T18:53:10.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.688+0000 7f8adb59e700 1 -- 172.21.15.7:0/127072297 shutdown_connections 2024-09-13T18:53:10.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.688+0000 7f8adb59e700 1 --2- 172.21.15.7:0/127072297 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8adc0cc3c0 0x7f8adc071430 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:10.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.688+0000 7f8adb59e700 1 --2- 172.21.15.7:0/127072297 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc159950 0x7f8adc0ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:10.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.688+0000 7f8adb59e700 1 -- 172.21.15.7:0/127072297 >> 172.21.15.7:0/127072297 conn(0x7f8adc0ae590 msgr2=0x7f8adc0ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:10.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.688+0000 7f8adb59e700 1 -- 172.21.15.7:0/127072297 shutdown_connections 2024-09-13T18:53:10.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.688+0000 7f8adb59e700 1 -- 172.21.15.7:0/127072297 wait complete. 2024-09-13T18:53:10.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8adb59e700 1 Processor -- start 2024-09-13T18:53:10.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8adb59e700 1 -- start start 2024-09-13T18:53:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8adb59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc071970 0x7f8adc13bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8adb59e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8adc0cc3c0 0x7f8adc1e2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8adb59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc159950 0x7f8adc0ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8adb59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc13bff0 con 0x7f8adc071970 2024-09-13T18:53:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8adb59e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc13e720 con 0x7f8adc0cc3c0 2024-09-13T18:53:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8adb59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc13ea00 con 0x7f8adc159950 2024-09-13T18:53:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8ada59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc071970 0x7f8adc13bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:10.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8ada59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc071970 0x7f8adc13bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60076/0 (socket says 172.21.15.7:60076) 2024-09-13T18:53:10.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8ada59c700 1 -- 172.21.15.7:0/822735404 learned_addr learned my addr 172.21.15.7:0/822735404 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:10.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8ad9d9b700 1 --2- 172.21.15.7:0/822735404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8adc0cc3c0 0x7f8adc1e2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:10.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8adad9d700 1 --2- 172.21.15.7:0/822735404 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc159950 0x7f8adc0ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:10.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8ada59c700 1 -- 172.21.15.7:0/822735404 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc159950 msgr2=0x7f8adc0ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:10.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8ada59c700 1 --2- 172.21.15.7:0/822735404 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc159950 0x7f8adc0ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:10.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8ada59c700 1 -- 172.21.15.7:0/822735404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8adc0cc3c0 msgr2=0x7f8adc1e2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:10.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8ada59c700 1 --2- 172.21.15.7:0/822735404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8adc0cc3c0 0x7f8adc1e2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:10.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8ada59c700 1 -- 172.21.15.7:0/822735404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad000b810 con 0x7f8adc071970 2024-09-13T18:53:10.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8ada59c700 1 --2- 172.21.15.7:0/822735404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc071970 0x7f8adc13bab0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00e9d0 tx=0x7f8acc00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:10.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8acaffd700 1 -- 172.21.15.7:0/822735404 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc009720 con 0x7f8adc071970 2024-09-13T18:53:10.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8acaffd700 1 -- 172.21.15.7:0/822735404 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc00ee70 con 0x7f8adc071970 2024-09-13T18:53:10.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8adb59e700 1 -- 172.21.15.7:0/822735404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8adc13ece0 con 0x7f8adc071970 2024-09-13T18:53:10.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8acaffd700 1 -- 172.21.15.7:0/822735404 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc00ca50 con 0x7f8adc071970 2024-09-13T18:53:10.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.692+0000 7f8adb59e700 1 -- 172.21.15.7:0/822735404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc13f2a0 con 0x7f8adc071970 2024-09-13T18:53:10.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.696+0000 7f8adb59e700 1 -- 172.21.15.7:0/822735404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8adc0ce490 con 0x7f8adc071970 2024-09-13T18:53:10.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.696+0000 7f8acaffd700 1 -- 172.21.15.7:0/822735404 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8acc00cbb0 con 0x7f8adc071970 2024-09-13T18:53:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.696+0000 7f8acaffd700 1 --2- 172.21.15.7:0/822735404 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8abc066730 0x7f8abc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.696+0000 7f8acaffd700 1 -- 172.21.15.7:0/822735404 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8acc083e60 con 0x7f8adc071970 2024-09-13T18:53:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.696+0000 7f8ad9d9b700 1 --2- 172.21.15.7:0/822735404 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8abc066730 0x7f8abc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.696+0000 7f8ad9d9b700 1 --2- 172.21.15.7:0/822735404 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8abc066730 0x7f8abc068bf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000bbb0 tx=0x7f8ad001b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:10.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.700+0000 7f8acaffd700 1 -- 172.21.15.7:0/822735404 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8acc0514d0 con 0x7f8adc071970 2024-09-13T18:53:10.724 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-13T18:53:10.724 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-13T18:53:10.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:10.880+0000 7f8adb59e700 1 -- 172.21.15.7:0/822735404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8adc0af180 con 0x7f8adc071970 2024-09-13T18:53:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.684+0000 7f8acaffd700 1 -- 172.21.15.7:0/822735404 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8acc00ce60 con 0x7f8adc071970 2024-09-13T18:53:11.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.688+0000 7f8adb59e700 1 -- 172.21.15.7:0/822735404 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8abc066730 msgr2=0x7f8abc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:11.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.688+0000 7f8adb59e700 1 --2- 172.21.15.7:0/822735404 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8abc066730 0x7f8abc068bf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000bbb0 tx=0x7f8ad001b040 comp rx=0 tx=0).stop 2024-09-13T18:53:11.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.688+0000 7f8adb59e700 1 -- 172.21.15.7:0/822735404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc071970 msgr2=0x7f8adc13bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:11.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.688+0000 7f8adb59e700 1 --2- 172.21.15.7:0/822735404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc071970 0x7f8adc13bab0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00e9d0 tx=0x7f8acc00ece0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.688+0000 7f8adb59e700 1 -- 172.21.15.7:0/822735404 shutdown_connections 2024-09-13T18:53:11.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.688+0000 7f8adb59e700 1 --2- 172.21.15.7:0/822735404 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8abc066730 0x7f8abc068bf0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.688+0000 7f8adb59e700 1 --2- 172.21.15.7:0/822735404 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc159950 0x7f8adc0ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.688+0000 7f8adb59e700 1 --2- 172.21.15.7:0/822735404 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8adc0cc3c0 0x7f8adc1e2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.688+0000 7f8adb59e700 1 --2- 172.21.15.7:0/822735404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc071970 0x7f8adc13bab0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.688+0000 7f8adb59e700 1 -- 172.21.15.7:0/822735404 >> 172.21.15.7:0/822735404 conn(0x7f8adc0ae590 msgr2=0x7f8adc0af980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.688+0000 7f8adb59e700 1 -- 172.21.15.7:0/822735404 shutdown_connections 2024-09-13T18:53:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.688+0000 7f8adb59e700 1 -- 172.21.15.7:0/822735404 wait complete. 2024-09-13T18:53:11.699 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-09-13T18:53:11.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T18:53:11.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175fce2700 1 Processor -- start 2024-09-13T18:53:11.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175fce2700 1 -- start start 2024-09-13T18:53:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175fce2700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f175815b890 0x7f1758162220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175fce2700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17580cdcc0 0x7f17580b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175fce2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17580b2930 0x7f175806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175fce2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17580b2dc0 con 0x7f17580b2930 2024-09-13T18:53:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175fce2700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1758162760 con 0x7f175815b890 2024-09-13T18:53:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175fce2700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175815bc70 con 0x7f17580cdcc0 2024-09-13T18:53:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175e27f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17580b2930 0x7f175806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175e27f700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17580b2930 0x7f175806ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46126/0 (socket says 172.21.15.7:46126) 2024-09-13T18:53:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175e27f700 1 -- 172.21.15.7:0/4162071096 learned_addr learned my addr 172.21.15.7:0/4162071096 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175da7e700 1 --2- 172.21.15.7:0/4162071096 >> v2:172.21.15.171:3300/0 conn(0x7f175815b890 0x7f1758162220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175d27d700 1 --2- 172.21.15.7:0/4162071096 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17580cdcc0 0x7f17580b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175e27f700 1 -- 172.21.15.7:0/4162071096 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17580cdcc0 msgr2=0x7f17580b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175e27f700 1 --2- 172.21.15.7:0/4162071096 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17580cdcc0 0x7f17580b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175e27f700 1 -- 172.21.15.7:0/4162071096 >> v2:172.21.15.171:3300/0 conn(0x7f175815b890 msgr2=0x7f1758162220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175e27f700 1 --2- 172.21.15.7:0/4162071096 >> v2:172.21.15.171:3300/0 conn(0x7f175815b890 0x7f1758162220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175e27f700 1 -- 172.21.15.7:0/4162071096 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17580ccc20 con 0x7f17580b2930 2024-09-13T18:53:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175e27f700 1 --2- 172.21.15.7:0/4162071096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17580b2930 0x7f175806ffe0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f17580b1750 tx=0x7f175000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fab42144a530028 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175ca7c700 1 -- 172.21.15.7:0/4162071096 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175000e070 con 0x7f17580b2930 2024-09-13T18:53:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175ca7c700 1 -- 172.21.15.7:0/4162071096 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175000b910 con 0x7f17580b2930 2024-09-13T18:53:11.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175ca7c700 1 -- 172.21.15.7:0/4162071096 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175000bcc0 con 0x7f17580b2930 2024-09-13T18:53:11.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175fce2700 1 -- 172.21.15.7:0/4162071096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17580b2930 msgr2=0x7f175806ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:11.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175fce2700 1 --2- 172.21.15.7:0/4162071096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17580b2930 0x7f175806ffe0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f17580b1750 tx=0x7f175000b330 comp rx=0 tx=0).stop 2024-09-13T18:53:11.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175fce2700 1 -- 172.21.15.7:0/4162071096 shutdown_connections 2024-09-13T18:53:11.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175fce2700 1 --2- 172.21.15.7:0/4162071096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17580b2930 0x7f175806ffe0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175fce2700 1 --2- 172.21.15.7:0/4162071096 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17580cdcc0 0x7f17580b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175fce2700 1 --2- 172.21.15.7:0/4162071096 >> v2:172.21.15.171:3300/0 conn(0x7f175815b890 0x7f1758162220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.876+0000 7f175fce2700 1 -- 172.21.15.7:0/4162071096 >> 172.21.15.7:0/4162071096 conn(0x7f17580ce980 msgr2=0x7f17580ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:11.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.880+0000 7f175fce2700 1 -- 172.21.15.7:0/4162071096 shutdown_connections 2024-09-13T18:53:11.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.880+0000 7f175fce2700 1 -- 172.21.15.7:0/4162071096 wait complete. 2024-09-13T18:53:11.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.880+0000 7f175fce2700 1 Processor -- start 2024-09-13T18:53:11.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.880+0000 7f175fce2700 1 -- start start 2024-09-13T18:53:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175fce2700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17580b2930 0x7f1758152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175fce2700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17580cdcc0 0x7f1758158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175fce2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f175815b890 0x7f1758148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175fce2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1758159350 con 0x7f175815b890 2024-09-13T18:53:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175fce2700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175815a190 con 0x7f17580b2930 2024-09-13T18:53:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175fce2700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1758155740 con 0x7f17580cdcc0 2024-09-13T18:53:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175e27f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f175815b890 0x7f1758148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175e27f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f175815b890 0x7f1758148aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46136/0 (socket says 172.21.15.7:46136) 2024-09-13T18:53:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175da7e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17580b2930 0x7f1758152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175e27f700 1 -- 172.21.15.7:0/434817804 learned_addr learned my addr 172.21.15.7:0/434817804 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175d27d700 1 --2- 172.21.15.7:0/434817804 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17580cdcc0 0x7f1758158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175e27f700 1 -- 172.21.15.7:0/434817804 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17580cdcc0 msgr2=0x7f1758158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175e27f700 1 --2- 172.21.15.7:0/434817804 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17580cdcc0 0x7f1758158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175e27f700 1 -- 172.21.15.7:0/434817804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17580b2930 msgr2=0x7f1758152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175e27f700 1 --2- 172.21.15.7:0/434817804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17580b2930 0x7f1758152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175e27f700 1 -- 172.21.15.7:0/434817804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1750009b30 con 0x7f175815b890 2024-09-13T18:53:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175e27f700 1 --2- 172.21.15.7:0/434817804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f175815b890 0x7f1758148aa0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f17580cc560 tx=0x7f1750004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f174e7fc700 1 -- 172.21.15.7:0/434817804 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17500099b0 con 0x7f175815b890 2024-09-13T18:53:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f174e7fc700 1 -- 172.21.15.7:0/434817804 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175000ce80 con 0x7f175815b890 2024-09-13T18:53:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175fce2700 1 -- 172.21.15.7:0/434817804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1758159580 con 0x7f175815b890 2024-09-13T18:53:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f174e7fc700 1 -- 172.21.15.7:0/434817804 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1750004490 con 0x7f175815b890 2024-09-13T18:53:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.884+0000 7f175fce2700 1 -- 172.21.15.7:0/434817804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1758159b40 con 0x7f175815b890 2024-09-13T18:53:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.888+0000 7f175fce2700 1 -- 172.21.15.7:0/434817804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175815a300 con 0x7f175815b890 2024-09-13T18:53:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.888+0000 7f174e7fc700 1 -- 172.21.15.7:0/434817804 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f17500045f0 con 0x7f175815b890 2024-09-13T18:53:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.888+0000 7f174e7fc700 1 --2- 172.21.15.7:0/434817804 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f173c066730 0x7f173c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.888+0000 7f174e7fc700 1 -- 172.21.15.7:0/434817804 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1750085340 con 0x7f175815b890 2024-09-13T18:53:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.888+0000 7f175da7e700 1 --2- 172.21.15.7:0/434817804 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f173c066730 0x7f173c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.888+0000 7f175da7e700 1 --2- 172.21.15.7:0/434817804 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f173c066730 0x7f173c068bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f17580cb080 tx=0x7f1754009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:11.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:11.896+0000 7f174e7fc700 1 -- 172.21.15.7:0/434817804 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1750052b90 con 0x7f175815b890 2024-09-13T18:53:12.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.128+0000 7f175fce2700 1 -- 172.21.15.7:0/434817804 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f17580b23a0 con 0x7f175815b890 2024-09-13T18:53:12.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.128+0000 7f174e7fc700 1 -- 172.21.15.7:0/434817804 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f17500048a0 con 0x7f175815b890 2024-09-13T18:53:12.136 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:12.137 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:53:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-09-13T18:53:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-09-13T18:53:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRmluqnExAAsUgfqXyo3d0MsBU4pTUFKA==", 2024-09-13T18:53:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-13T18:53:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-13T18:53:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-13T18:53:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:53:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-09-13T18:53:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRmse2YGxAA1xevwPXbqkiFfH1c51CyUQ==", 2024-09-13T18:53:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-13T18:53:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-13T18:53:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-13T18:53:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:53:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.138 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-09-13T18:53:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRmovm+FxAAACx3SjaGO+6TwKcii8ZfcQ==", 2024-09-13T18:53:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-13T18:53:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-13T18:53:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-13T18:53:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:53:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.139 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-09-13T18:53:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRms9RQHxAAWQjRb5IFrsEkHgk1fFl1gA==", 2024-09-13T18:53:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-13T18:53:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-13T18:53:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-13T18:53:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:53:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-09-13T18:53:12.140 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCzieRmoweYHRAAJq4W0iwb4bNJtNoojDmYvA==", 2024-09-13T18:53:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-13T18:53:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-13T18:53:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:53:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-09-13T18:53:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQC2ieRmMlp6IBAA6UPBa8XtvToSEXOH7jRp2Q==", 2024-09-13T18:53:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-13T18:53:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-13T18:53:12.141 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:53:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-09-13T18:53:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQC5ieRm8PRwJxAAJ6u6O2CY9AB8ADXGUq3tHw==", 2024-09-13T18:53:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-13T18:53:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-13T18:53:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:53:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-09-13T18:53:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQC8ieRmpelUKxAAZkfxHmxH4AQs2+ZmYEMI5g==", 2024-09-13T18:53:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-13T18:53:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-13T18:53:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:53:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-09-13T18:53:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDCieRmt6JLERAAOSeTXcLflQO4UStq8/r4Kg==", 2024-09-13T18:53:12.143 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-13T18:53:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-13T18:53:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:53:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-09-13T18:53:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDFieRmnrQzFhAAbpEXq+QqxOuUEon2UfwRww==", 2024-09-13T18:53:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-13T18:53:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-13T18:53:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:53:12.144 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-09-13T18:53:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDIieRmbacMGhAAXeApS17wKjLK0KlAvOFMCw==", 2024-09-13T18:53:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-13T18:53:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-13T18:53:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:53:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-09-13T18:53:12.145 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDLieRmcxLrIRAAnXXU0UeSpdKkMzVuT2U9tA==", 2024-09-13T18:53:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-13T18:53:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-13T18:53:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:53:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-09-13T18:53:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRmcmOmIhAAoiL5c3tp8skQ5VcKF/6zVw==", 2024-09-13T18:53:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-13T18:53:12.146 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-13T18:53:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-13T18:53:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-13T18:53:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-09-13T18:53:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRmcNsbLBAAj36vmzv7emQ+ueYkP/2HtA==", 2024-09-13T18:53:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.147 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-13T18:53:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-13T18:53:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-13T18:53:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-13T18:53:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-09-13T18:53:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRmvonEKBAAefHhRIV+mux4L2yvJieukQ==", 2024-09-13T18:53:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-13T18:53:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-13T18:53:12.148 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-13T18:53:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-13T18:53:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-09-13T18:53:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRmh8G8MRAAeqibcXujnGSy9zs4XPf2hA==", 2024-09-13T18:53:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-13T18:53:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-13T18:53:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-13T18:53:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-13T18:53:12.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-09-13T18:53:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRm7nRBARAAInH7XqMLMDnVtQEezwkVhg==", 2024-09-13T18:53:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-13T18:53:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-13T18:53:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-13T18:53:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:53:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.150 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-mds", 2024-09-13T18:53:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDQieRmh9XLJhAANsXs6QAHaiqS1lYCCs8y0g==", 2024-09-13T18:53:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-mds" 2024-09-13T18:53:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-mgr", 2024-09-13T18:53:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDQieRmZ97LJhAAQ3s2qSLMk/Zd9wIjAW9jTw==", 2024-09-13T18:53:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-13T18:53:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-osd", 2024-09-13T18:53:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDQieRmV+bLJhAABOLR0HLXcV4R5PV8fAB8Gg==", 2024-09-13T18:53:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-osd" 2024-09-13T18:53:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rbd", 2024-09-13T18:53:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDQieRmLO/LJhAAx2NxsvE664UnZ09wV7IWHw==", 2024-09-13T18:53:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-13T18:53:12.152 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-13T18:53:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDQieRmsPbLJhAAvVNqPd/snEHkVxHNK7lQQg==", 2024-09-13T18:53:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-13T18:53:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rgw", 2024-09-13T18:53:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDQieRmWP7LJhAARRtP6hNBXBmYxVNmL9z2Hg==", 2024-09-13T18:53:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-13T18:53:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-09-13T18:53:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRm67fCDhAAm9sMp6tlCuJ3vwp57kczzA==", 2024-09-13T18:53:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-13T18:53:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-13T18:53:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:53:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-09-13T18:53:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRmJQ94CxAAIusXMD6x4ey4OLJO9t7n+w==", 2024-09-13T18:53:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:53:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-13T18:53:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-13T18:53:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:53:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-13T18:53:12.155 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:53:12.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.132+0000 7f175fce2700 1 -- 172.21.15.7:0/434817804 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f173c066730 msgr2=0x7f173c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:12.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.132+0000 7f175fce2700 1 --2- 172.21.15.7:0/434817804 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f173c066730 0x7f173c068bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f17580cb080 tx=0x7f1754009240 comp rx=0 tx=0).stop 2024-09-13T18:53:12.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.132+0000 7f175fce2700 1 -- 172.21.15.7:0/434817804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f175815b890 msgr2=0x7f1758148aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:12.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.132+0000 7f175fce2700 1 --2- 172.21.15.7:0/434817804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f175815b890 0x7f1758148aa0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f17580cc560 tx=0x7f1750004af0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.132+0000 7f175fce2700 1 -- 172.21.15.7:0/434817804 shutdown_connections 2024-09-13T18:53:12.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.132+0000 7f175fce2700 1 --2- 172.21.15.7:0/434817804 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f173c066730 0x7f173c068bf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.132+0000 7f175fce2700 1 --2- 172.21.15.7:0/434817804 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f175815b890 0x7f1758148aa0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.132+0000 7f175fce2700 1 --2- 172.21.15.7:0/434817804 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17580cdcc0 0x7f1758158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.132+0000 7f175fce2700 1 --2- 172.21.15.7:0/434817804 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17580b2930 0x7f1758152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.132+0000 7f175fce2700 1 -- 172.21.15.7:0/434817804 >> 172.21.15.7:0/434817804 conn(0x7f17580ce980 msgr2=0x7f1758156550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:12.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.132+0000 7f175fce2700 1 -- 172.21.15.7:0/434817804 shutdown_connections 2024-09-13T18:53:12.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.132+0000 7f175fce2700 1 -- 172.21.15.7:0/434817804 wait complete. 2024-09-13T18:53:12.158 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-13T18:53:12.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.264+0000 7fcb47a81700 1 Processor -- start 2024-09-13T18:53:12.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.264+0000 7fcb47a81700 1 -- start start 2024-09-13T18:53:12.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb47a81700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fcb4015a940 0x7fcb4006d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb47a81700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb4006c950 con 0x7fcb401433e0 2024-09-13T18:53:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb47a81700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb4006cc30 con 0x7fcb4006a310 2024-09-13T18:53:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb47a81700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb4015add0 con 0x7fcb4015a940 2024-09-13T18:53:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb4501c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcb401433e0 0x7fcb4015b290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47316/0 (socket says 172.21.15.7:47316) 2024-09-13T18:53:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb4501c700 1 -- 172.21.15.7:0/525941013 learned_addr learned my addr 172.21.15.7:0/525941013 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb4601e700 1 --2- 172.21.15.7:0/525941013 >> v2:172.21.15.171:3300/0 conn(0x7fcb4015a940 0x7fcb4006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb4481b700 1 -- 172.21.15.7:0/525941013 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479905262 0 0) 0x7fcb4006c950 con 0x7fcb401433e0 2024-09-13T18:53:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb4481b700 1 -- 172.21.15.7:0/525941013 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb2c0035d0 con 0x7fcb401433e0 2024-09-13T18:53:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb4481b700 1 -- 172.21.15.7:0/525941013 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512381856 0 0) 0x7fcb4006cc30 con 0x7fcb4006a310 2024-09-13T18:53:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb4481b700 1 -- 172.21.15.7:0/525941013 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb4006c950 con 0x7fcb4006a310 2024-09-13T18:53:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb4601e700 1 -- 172.21.15.7:0/525941013 >> v1:172.21.15.171:6789/0 conn(0x7fcb4006a310 legacy=0x7fcb4006a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb4601e700 1 -- 172.21.15.7:0/525941013 >> v1:172.21.15.7:6789/0 conn(0x7fcb401433e0 legacy=0x7fcb4015b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb4601e700 1 -- 172.21.15.7:0/525941013 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4006cf10 con 0x7fcb4015a940 2024-09-13T18:53:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb4601e700 1 --2- 172.21.15.7:0/525941013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb4015a940 0x7fcb4006d710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009d90 tx=0x7fcb3800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2617a7fb9dcceb92 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb4481b700 1 -- 172.21.15.7:0/525941013 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3800e070 con 0x7fcb4015a940 2024-09-13T18:53:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb4481b700 1 -- 172.21.15.7:0/525941013 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3800bb20 con 0x7fcb4015a940 2024-09-13T18:53:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb4481b700 1 -- 172.21.15.7:0/525941013 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3800be90 con 0x7fcb4015a940 2024-09-13T18:53:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb47a81700 1 -- 172.21.15.7:0/525941013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb4015a940 msgr2=0x7fcb4006d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.268+0000 7fcb47a81700 1 --2- 172.21.15.7:0/525941013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb4015a940 0x7fcb4006d710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009d90 tx=0x7fcb3800b330 comp rx=0 tx=0).stop 2024-09-13T18:53:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb47a81700 1 -- 172.21.15.7:0/525941013 shutdown_connections 2024-09-13T18:53:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb47a81700 1 --2- 172.21.15.7:0/525941013 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb4015a940 0x7fcb4006d710 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb47a81700 1 -- 172.21.15.7:0/525941013 >> 172.21.15.7:0/525941013 conn(0x7fcb400af620 msgr2=0x7fcb400afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb47a81700 1 -- 172.21.15.7:0/525941013 shutdown_connections 2024-09-13T18:53:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb47a81700 1 -- 172.21.15.7:0/525941013 wait complete. 2024-09-13T18:53:12.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb47a81700 1 Processor -- start 2024-09-13T18:53:12.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb47a81700 1 -- start start 2024-09-13T18:53:12.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb47a81700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4006a310 0x7fcb40157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb47a81700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb401433e0 0x7fcb401578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb47a81700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb4015a940 0x7fcb401389b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb47a81700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4015cdd0 con 0x7fcb4006a310 2024-09-13T18:53:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb47a81700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4015c1c0 con 0x7fcb4015a940 2024-09-13T18:53:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb47a81700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4015c4a0 con 0x7fcb401433e0 2024-09-13T18:53:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb4581d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4006a310 0x7fcb40157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb4581d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4006a310 0x7fcb40157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46152/0 (socket says 172.21.15.7:46152) 2024-09-13T18:53:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb4581d700 1 -- 172.21.15.7:0/106977389 learned_addr learned my addr 172.21.15.7:0/106977389 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb4601e700 1 --2- 172.21.15.7:0/106977389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb4015a940 0x7fcb401389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:12.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb4501c700 1 --2- 172.21.15.7:0/106977389 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb401433e0 0x7fcb401578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:12.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb4581d700 1 -- 172.21.15.7:0/106977389 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb401433e0 msgr2=0x7fcb401578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:12.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb4581d700 1 --2- 172.21.15.7:0/106977389 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb401433e0 0x7fcb401578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb4581d700 1 -- 172.21.15.7:0/106977389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb4015a940 msgr2=0x7fcb401389b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb4581d700 1 --2- 172.21.15.7:0/106977389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb4015a940 0x7fcb401389b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.272+0000 7fcb4581d700 1 -- 172.21.15.7:0/106977389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb38009a70 con 0x7fcb4006a310 2024-09-13T18:53:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.276+0000 7fcb4581d700 1 --2- 172.21.15.7:0/106977389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4006a310 0x7fcb40157370 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000c520 tx=0x7fcb3000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.276+0000 7fcb367fc700 1 -- 172.21.15.7:0/106977389 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb300052d0 con 0x7fcb4006a310 2024-09-13T18:53:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.276+0000 7fcb367fc700 1 -- 172.21.15.7:0/106977389 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb30005450 con 0x7fcb4006a310 2024-09-13T18:53:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.276+0000 7fcb47a81700 1 -- 172.21.15.7:0/106977389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4015c780 con 0x7fcb4006a310 2024-09-13T18:53:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.276+0000 7fcb367fc700 1 -- 172.21.15.7:0/106977389 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb300038e0 con 0x7fcb4006a310 2024-09-13T18:53:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.276+0000 7fcb47a81700 1 -- 172.21.15.7:0/106977389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb400ce330 con 0x7fcb4006a310 2024-09-13T18:53:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.276+0000 7fcb47a81700 1 -- 172.21.15.7:0/106977389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb4006cc30 con 0x7fcb4006a310 2024-09-13T18:53:12.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.276+0000 7fcb367fc700 1 -- 172.21.15.7:0/106977389 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcb30003a40 con 0x7fcb4006a310 2024-09-13T18:53:12.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.280+0000 7fcb367fc700 1 --2- 172.21.15.7:0/106977389 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb24066730 0x7fcb24068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:12.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.280+0000 7fcb367fc700 1 -- 172.21.15.7:0/106977389 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcb30083170 con 0x7fcb4006a310 2024-09-13T18:53:12.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.280+0000 7fcb4501c700 1 --2- 172.21.15.7:0/106977389 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb24066730 0x7fcb24068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:12.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.280+0000 7fcb4501c700 1 --2- 172.21.15.7:0/106977389 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb24066730 0x7fcb24068bf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcb4015b180 tx=0x7fcb3c007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.284+0000 7fcb367fc700 1 -- 172.21.15.7:0/106977389 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcb300508c0 con 0x7fcb4006a310 2024-09-13T18:53:12.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.524+0000 7fcb47a81700 1 -- 172.21.15.7:0/106977389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fcb400b4970 con 0x7fcb4006a310 2024-09-13T18:53:12.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.532+0000 7fcb367fc700 1 -- 172.21.15.7:0/106977389 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fcb30053ee0 con 0x7fcb4006a310 2024-09-13T18:53:12.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.532+0000 7fcb47a81700 1 -- 172.21.15.7:0/106977389 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb24066730 msgr2=0x7fcb24068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.532+0000 7fcb47a81700 1 --2- 172.21.15.7:0/106977389 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb24066730 0x7fcb24068bf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcb4015b180 tx=0x7fcb3c007de0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.532+0000 7fcb47a81700 1 -- 172.21.15.7:0/106977389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4006a310 msgr2=0x7fcb40157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.532+0000 7fcb47a81700 1 --2- 172.21.15.7:0/106977389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4006a310 0x7fcb40157370 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000c520 tx=0x7fcb3000c830 comp rx=0 tx=0).stop 2024-09-13T18:53:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.532+0000 7fcb47a81700 1 -- 172.21.15.7:0/106977389 shutdown_connections 2024-09-13T18:53:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.532+0000 7fcb47a81700 1 --2- 172.21.15.7:0/106977389 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb24066730 0x7fcb24068bf0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.532+0000 7fcb47a81700 1 --2- 172.21.15.7:0/106977389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb4015a940 0x7fcb401389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.532+0000 7fcb47a81700 1 --2- 172.21.15.7:0/106977389 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcb401433e0 0x7fcb401578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.532+0000 7fcb47a81700 1 --2- 172.21.15.7:0/106977389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4006a310 0x7fcb40157370 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.532+0000 7fcb47a81700 1 -- 172.21.15.7:0/106977389 >> 172.21.15.7:0/106977389 conn(0x7fcb400af620 msgr2=0x7fcb401405c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.536+0000 7fcb47a81700 1 -- 172.21.15.7:0/106977389 shutdown_connections 2024-09-13T18:53:12.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.536+0000 7fcb47a81700 1 -- 172.21.15.7:0/106977389 wait complete. 2024-09-13T18:53:12.555 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-13T18:53:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.664+0000 7f1cccaf6700 1 Processor -- start 2024-09-13T18:53:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.664+0000 7f1cccaf6700 1 -- start start 2024-09-13T18:53:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cccaf6700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cc806bdf0 0x7f1cc806b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cccaf6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc806d0e0 con 0x7f1cc80b51b0 2024-09-13T18:53:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cccaf6700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc806e1d0 con 0x7f1cc806dd10 2024-09-13T18:53:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cccaf6700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc806e4b0 con 0x7f1cc806bdf0 2024-09-13T18:53:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc7fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1cc80b51b0 0x7f1cc8163390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47320/0 (socket says 172.21.15.7:47320) 2024-09-13T18:53:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc7fff700 1 -- 172.21.15.7:0/3072076357 learned_addr learned my addr 172.21.15.7:0/3072076357 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc77fe700 1 --2- 172.21.15.7:0/3072076357 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cc806bdf0 0x7f1cc806b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 -- 172.21.15.7:0/3072076357 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 209075454 0 0) 0x7f1cc806d0e0 con 0x7f1cc80b51b0 2024-09-13T18:53:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 -- 172.21.15.7:0/3072076357 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cb40035d0 con 0x7f1cc80b51b0 2024-09-13T18:53:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 -- 172.21.15.7:0/3072076357 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 547103384 0 0) 0x7f1cc806e1d0 con 0x7f1cc806dd10 2024-09-13T18:53:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 -- 172.21.15.7:0/3072076357 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cc806d0e0 con 0x7f1cc806dd10 2024-09-13T18:53:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 -- 172.21.15.7:0/3072076357 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2683772194 0 0) 0x7f1cb40035d0 con 0x7f1cc80b51b0 2024-09-13T18:53:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 -- 172.21.15.7:0/3072076357 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cc806bdf0 msgr2=0x7f1cc806b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 --2- 172.21.15.7:0/3072076357 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cc806bdf0 0x7f1cc806b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 -- 172.21.15.7:0/3072076357 >> v1:172.21.15.171:6789/0 conn(0x7f1cc806dd10 legacy=0x7f1cc806c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 -- 172.21.15.7:0/3072076357 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc80b46d0 con 0x7f1cc80b51b0 2024-09-13T18:53:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 -- 172.21.15.7:0/3072076357 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7f1cc00019f0 con 0x7f1cc80b51b0 2024-09-13T18:53:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 -- 172.21.15.7:0/3072076357 >> v1:172.21.15.7:6789/0 conn(0x7f1cc80b51b0 legacy=0x7f1cc8163390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 --2- 172.21.15.7:0/3072076357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cb4003be0 0x7f1cb4003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 --2- 172.21.15.7:0/3072076357 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cc806bdf0 0x7f1cb4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 --2- 172.21.15.7:0/3072076357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb4004db0 0x7f1cb4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 -- 172.21.15.7:0/3072076357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc806e4b0 con 0x7f1cb4004db0 2024-09-13T18:53:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 -- 172.21.15.7:0/3072076357 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb400dcf0 con 0x7f1cb4003be0 2024-09-13T18:53:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 -- 172.21.15.7:0/3072076357 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb400deb0 con 0x7f1cc806bdf0 2024-09-13T18:53:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc7fff700 1 --2- 172.21.15.7:0/3072076357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb4004db0 0x7f1cb4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc77fe700 1 --2- 172.21.15.7:0/3072076357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cb4003be0 0x7f1cb4003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cbeffd700 1 --2- 172.21.15.7:0/3072076357 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cc806bdf0 0x7f1cb4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc7fff700 1 -- 172.21.15.7:0/3072076357 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cc806bdf0 msgr2=0x7f1cb4004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc7fff700 1 --2- 172.21.15.7:0/3072076357 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cc806bdf0 0x7f1cb4004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc7fff700 1 -- 172.21.15.7:0/3072076357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cb4003be0 msgr2=0x7f1cb4003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc7fff700 1 --2- 172.21.15.7:0/3072076357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cb4003be0 0x7f1cb4003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc7fff700 1 -- 172.21.15.7:0/3072076357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb40035d0 con 0x7f1cb4004db0 2024-09-13T18:53:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cbeffd700 1 --2- 172.21.15.7:0/3072076357 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cc806bdf0 0x7f1cb4004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc77fe700 1 --2- 172.21.15.7:0/3072076357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cb4003be0 0x7f1cb4003fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc7fff700 1 --2- 172.21.15.7:0/3072076357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb4004db0 0x7f1cb4005230 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000a700 tx=0x7f1cb40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd13f7c31fc8382 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 -- 172.21.15.7:0/3072076357 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc00070d0 con 0x7f1cb4004db0 2024-09-13T18:53:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 -- 172.21.15.7:0/3072076357 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc0007250 con 0x7f1cb4004db0 2024-09-13T18:53:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.668+0000 7f1cc6ffd700 1 -- 172.21.15.7:0/3072076357 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0007600 con 0x7f1cb4004db0 2024-09-13T18:53:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cccaf6700 1 -- 172.21.15.7:0/3072076357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb4004db0 msgr2=0x7f1cb4005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cccaf6700 1 --2- 172.21.15.7:0/3072076357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb4004db0 0x7f1cb4005230 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1cc000a700 tx=0x7f1cb40058a0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cc7fff700 1 -- 172.21.15.7:0/3072076357 reap_dead start 2024-09-13T18:53:12.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cccaf6700 1 -- 172.21.15.7:0/3072076357 shutdown_connections 2024-09-13T18:53:12.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cccaf6700 1 -- 172.21.15.7:0/3072076357 >> 172.21.15.7:0/3072076357 conn(0x7f1cc8070ba0 msgr2=0x7f1cc8070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:12.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cccaf6700 1 -- 172.21.15.7:0/3072076357 shutdown_connections 2024-09-13T18:53:12.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cccaf6700 1 -- 172.21.15.7:0/3072076357 wait complete. 2024-09-13T18:53:12.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cccaf6700 1 Processor -- start 2024-09-13T18:53:12.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cccaf6700 1 -- start start 2024-09-13T18:53:12.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cccaf6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cc8159160 0x7f1cc81340b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:12.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cccaf6700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cc81348f0 0x7f1cc8132650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:12.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cccaf6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc8132b90 0x7f1cc81ee910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:12.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cccaf6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc81345f0 con 0x7f1cc8132b90 2024-09-13T18:53:12.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cccaf6700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8133010 con 0x7f1cc8159160 2024-09-13T18:53:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cccaf6700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc81332f0 con 0x7f1cc81348f0 2024-09-13T18:53:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cc7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc8132b90 0x7f1cc81ee910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cc7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc8132b90 0x7f1cc81ee910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46162/0 (socket says 172.21.15.7:46162) 2024-09-13T18:53:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cc7fff700 1 -- 172.21.15.7:0/1458508381 learned_addr learned my addr 172.21.15.7:0/1458508381 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cc77fe700 1 --2- 172.21.15.7:0/1458508381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cc8159160 0x7f1cc81340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cbeffd700 1 --2- 172.21.15.7:0/1458508381 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cc81348f0 0x7f1cc8132650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cc7fff700 1 -- 172.21.15.7:0/1458508381 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cc81348f0 msgr2=0x7f1cc8132650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cc7fff700 1 --2- 172.21.15.7:0/1458508381 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cc81348f0 0x7f1cc8132650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cc7fff700 1 -- 172.21.15.7:0/1458508381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cc8159160 msgr2=0x7f1cc81340b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cc7fff700 1 --2- 172.21.15.7:0/1458508381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cc8159160 0x7f1cc81340b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cc7fff700 1 -- 172.21.15.7:0/1458508381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc80cc8a0 con 0x7f1cc8132b90 2024-09-13T18:53:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cc7fff700 1 --2- 172.21.15.7:0/1458508381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc8132b90 0x7f1cc81ee910 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0011000 tx=0x7f1cc0001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cc4ff9700 1 -- 172.21.15.7:0/1458508381 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc00070d0 con 0x7f1cc8132b90 2024-09-13T18:53:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.672+0000 7f1cc4ff9700 1 -- 172.21.15.7:0/1458508381 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc0021c70 con 0x7f1cc8132b90 2024-09-13T18:53:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.676+0000 7f1cccaf6700 1 -- 172.21.15.7:0/1458508381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc81335d0 con 0x7f1cc8132b90 2024-09-13T18:53:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.676+0000 7f1cc4ff9700 1 -- 172.21.15.7:0/1458508381 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc00073c0 con 0x7f1cc8132b90 2024-09-13T18:53:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.676+0000 7f1cccaf6700 1 -- 172.21.15.7:0/1458508381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc8133b90 con 0x7f1cc8132b90 2024-09-13T18:53:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.676+0000 7f1cccaf6700 1 -- 172.21.15.7:0/1458508381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc8133d80 con 0x7f1cc8132b90 2024-09-13T18:53:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.676+0000 7f1cc4ff9700 1 -- 172.21.15.7:0/1458508381 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1cc001a070 con 0x7f1cc8132b90 2024-09-13T18:53:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.676+0000 7f1cc4ff9700 1 --2- 172.21.15.7:0/1458508381 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1ca8066730 0x7f1ca8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.676+0000 7f1cc4ff9700 1 -- 172.21.15.7:0/1458508381 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1cc0088e50 con 0x7f1cc8132b90 2024-09-13T18:53:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.676+0000 7f1cc77fe700 1 --2- 172.21.15.7:0/1458508381 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1ca8066730 0x7f1ca8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.680+0000 7f1cc77fe700 1 --2- 172.21.15.7:0/1458508381 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1ca8066730 0x7f1ca8068bf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009b50 tx=0x7f1cb8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.688+0000 7f1cc4ff9700 1 -- 172.21.15.7:0/1458508381 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1cc0056770 con 0x7f1cc8132b90 2024-09-13T18:53:12.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.920+0000 7f1cccaf6700 1 -- 172.21.15.7:0/1458508381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f1cc80b3a00 con 0x7f1cc8132b90 2024-09-13T18:53:12.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.928+0000 7f1cc4ff9700 1 -- 172.21.15.7:0/1458508381 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1cc002f020 con 0x7f1cc8132b90 2024-09-13T18:53:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.932+0000 7f1cccaf6700 1 -- 172.21.15.7:0/1458508381 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1ca8066730 msgr2=0x7f1ca8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.932+0000 7f1cccaf6700 1 --2- 172.21.15.7:0/1458508381 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1ca8066730 0x7f1ca8068bf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009b50 tx=0x7f1cb8008040 comp rx=0 tx=0).stop 2024-09-13T18:53:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.932+0000 7f1cccaf6700 1 -- 172.21.15.7:0/1458508381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc8132b90 msgr2=0x7f1cc81ee910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.932+0000 7f1cccaf6700 1 --2- 172.21.15.7:0/1458508381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc8132b90 0x7f1cc81ee910 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0011000 tx=0x7f1cc0001cb0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.932+0000 7f1cccaf6700 1 -- 172.21.15.7:0/1458508381 shutdown_connections 2024-09-13T18:53:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.932+0000 7f1cccaf6700 1 --2- 172.21.15.7:0/1458508381 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1ca8066730 0x7f1ca8068bf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.932+0000 7f1cccaf6700 1 --2- 172.21.15.7:0/1458508381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc8132b90 0x7f1cc81ee910 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.932+0000 7f1cccaf6700 1 --2- 172.21.15.7:0/1458508381 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cc81348f0 0x7f1cc8132650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.932+0000 7f1cccaf6700 1 --2- 172.21.15.7:0/1458508381 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cc8159160 0x7f1cc81340b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:12.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.932+0000 7f1cccaf6700 1 -- 172.21.15.7:0/1458508381 >> 172.21.15.7:0/1458508381 conn(0x7f1cc8070ba0 msgr2=0x7f1cc806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:12.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.932+0000 7f1cccaf6700 1 -- 172.21.15.7:0/1458508381 shutdown_connections 2024-09-13T18:53:12.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:12.932+0000 7f1cccaf6700 1 -- 172.21.15.7:0/1458508381 wait complete. 2024-09-13T18:53:12.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-13T18:53:13.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.060+0000 7f8515cf4700 1 Processor -- start 2024-09-13T18:53:13.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.060+0000 7f8515cf4700 1 -- start start 2024-09-13T18:53:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f8515cf4700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f851006d8c0 0x7f8510164a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f8515cf4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8510070bd0 0x7f8510163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f8515cf4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8510162570 con 0x7f8510164f90 2024-09-13T18:53:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f8515cf4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8510162e60 con 0x7f8510070bd0 2024-09-13T18:53:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f8515cf4700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8510071240 con 0x7f851006d8c0 2024-09-13T18:53:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f85154f3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8510070bd0 0x7f8510163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f850ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8510164f90 0x7f85100704c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47326/0 (socket says 172.21.15.7:47326) 2024-09-13T18:53:13.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f850ffff700 1 -- 172.21.15.7:0/4154102571 learned_addr learned my addr 172.21.15.7:0/4154102571 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f8514cf2700 1 --2- 172.21.15.7:0/4154102571 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f851006d8c0 0x7f8510164a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f85154f3700 1 -- 172.21.15.7:0/4154102571 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f851006d8c0 msgr2=0x7f8510164a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f85154f3700 1 --2- 172.21.15.7:0/4154102571 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f851006d8c0 0x7f8510164a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f85154f3700 1 -- 172.21.15.7:0/4154102571 >> v1:172.21.15.7:6789/0 conn(0x7f8510164f90 legacy=0x7f85100704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f85154f3700 1 -- 172.21.15.7:0/4154102571 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85101653f0 con 0x7f8510070bd0 2024-09-13T18:53:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f85154f3700 1 --2- 172.21.15.7:0/4154102571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8510070bd0 0x7f8510163280 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8508009d90 tx=0x7f850800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9170bb2cbe0eed server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f850f7fe700 1 -- 172.21.15.7:0/4154102571 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850800e070 con 0x7f8510070bd0 2024-09-13T18:53:13.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f850f7fe700 1 -- 172.21.15.7:0/4154102571 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850800bbd0 con 0x7f8510070bd0 2024-09-13T18:53:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f850f7fe700 1 -- 172.21.15.7:0/4154102571 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508012430 con 0x7f8510070bd0 2024-09-13T18:53:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f8515cf4700 1 -- 172.21.15.7:0/4154102571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8510070bd0 msgr2=0x7f8510163280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f8515cf4700 1 --2- 172.21.15.7:0/4154102571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8510070bd0 0x7f8510163280 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8508009d90 tx=0x7f850800b330 comp rx=0 tx=0).stop 2024-09-13T18:53:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f8515cf4700 1 -- 172.21.15.7:0/4154102571 shutdown_connections 2024-09-13T18:53:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f8515cf4700 1 --2- 172.21.15.7:0/4154102571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8510070bd0 0x7f8510163280 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f8515cf4700 1 --2- 172.21.15.7:0/4154102571 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f851006d8c0 0x7f8510164a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f8515cf4700 1 -- 172.21.15.7:0/4154102571 >> 172.21.15.7:0/4154102571 conn(0x7f85100b0e10 msgr2=0x7f85100b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f8515cf4700 1 -- 172.21.15.7:0/4154102571 shutdown_connections 2024-09-13T18:53:13.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.064+0000 7f8515cf4700 1 -- 172.21.15.7:0/4154102571 wait complete. 2024-09-13T18:53:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8515cf4700 1 Processor -- start 2024-09-13T18:53:13.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8515cf4700 1 -- start start 2024-09-13T18:53:13.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8515cf4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f851006d8c0 0x7f851015e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8515cf4700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8510070bd0 0x7f851015abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8515cf4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8510164f90 0x7f851015cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8515cf4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851015e7c0 con 0x7f851006d8c0 2024-09-13T18:53:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8515cf4700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851015ea00 con 0x7f8510164f90 2024-09-13T18:53:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8515cf4700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851015a0e0 con 0x7f8510070bd0 2024-09-13T18:53:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8514cf2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f851006d8c0 0x7f851015e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8514cf2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f851006d8c0 0x7f851015e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46180/0 (socket says 172.21.15.7:46180) 2024-09-13T18:53:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8514cf2700 1 -- 172.21.15.7:0/2700656247 learned_addr learned my addr 172.21.15.7:0/2700656247 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f85154f3700 1 --2- 172.21.15.7:0/2700656247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8510164f90 0x7f851015cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f850ffff700 1 --2- 172.21.15.7:0/2700656247 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8510070bd0 0x7f851015abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8514cf2700 1 -- 172.21.15.7:0/2700656247 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8510070bd0 msgr2=0x7f851015abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8514cf2700 1 --2- 172.21.15.7:0/2700656247 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8510070bd0 0x7f851015abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8514cf2700 1 -- 172.21.15.7:0/2700656247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8510164f90 msgr2=0x7f851015cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8514cf2700 1 --2- 172.21.15.7:0/2700656247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8510164f90 0x7f851015cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8514cf2700 1 -- 172.21.15.7:0/2700656247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8508009a70 con 0x7f851006d8c0 2024-09-13T18:53:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8514cf2700 1 --2- 172.21.15.7:0/2700656247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f851006d8c0 0x7f851015e280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f84f800e760 tx=0x7f84f800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f850d7fa700 1 -- 172.21.15.7:0/2700656247 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f800c070 con 0x7f851006d8c0 2024-09-13T18:53:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f850d7fa700 1 -- 172.21.15.7:0/2700656247 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f8013610 con 0x7f851006d8c0 2024-09-13T18:53:13.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8515cf4700 1 -- 172.21.15.7:0/2700656247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851015a3c0 con 0x7f851006d8c0 2024-09-13T18:53:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f850d7fa700 1 -- 172.21.15.7:0/2700656247 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8013980 con 0x7f851006d8c0 2024-09-13T18:53:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.068+0000 7f8515cf4700 1 -- 172.21.15.7:0/2700656247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8510159dd0 con 0x7f851006d8c0 2024-09-13T18:53:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.072+0000 7f8515cf4700 1 -- 172.21.15.7:0/2700656247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f851015f770 con 0x7f851006d8c0 2024-09-13T18:53:13.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.072+0000 7f850d7fa700 1 -- 172.21.15.7:0/2700656247 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f84f8013ae0 con 0x7f851006d8c0 2024-09-13T18:53:13.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.072+0000 7f850d7fa700 1 --2- 172.21.15.7:0/2700656247 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f84f0066730 0x7f84f0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.072+0000 7f850d7fa700 1 -- 172.21.15.7:0/2700656247 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f84f8083ca0 con 0x7f851006d8c0 2024-09-13T18:53:13.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.072+0000 7f850ffff700 1 --2- 172.21.15.7:0/2700656247 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f84f0066730 0x7f84f0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.076+0000 7f850ffff700 1 --2- 172.21.15.7:0/2700656247 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f84f0066730 0x7f84f0068bf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8510156b30 tx=0x7f8500009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:13.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.080+0000 7f850d7fa700 1 -- 172.21.15.7:0/2700656247 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f84f8051240 con 0x7f851006d8c0 2024-09-13T18:53:13.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.312+0000 7f8515cf4700 1 -- 172.21.15.7:0/2700656247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f85101641e0 con 0x7f851006d8c0 2024-09-13T18:53:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.320+0000 7f850d7fa700 1 -- 172.21.15.7:0/2700656247 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f84f8054860 con 0x7f851006d8c0 2024-09-13T18:53:13.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.324+0000 7f8515cf4700 1 -- 172.21.15.7:0/2700656247 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f84f0066730 msgr2=0x7f84f0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.324+0000 7f8515cf4700 1 --2- 172.21.15.7:0/2700656247 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f84f0066730 0x7f84f0068bf0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8510156b30 tx=0x7f8500009040 comp rx=0 tx=0).stop 2024-09-13T18:53:13.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.324+0000 7f8515cf4700 1 -- 172.21.15.7:0/2700656247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f851006d8c0 msgr2=0x7f851015e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.324+0000 7f8515cf4700 1 --2- 172.21.15.7:0/2700656247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f851006d8c0 0x7f851015e280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f84f800e760 tx=0x7f84f800ea70 comp rx=0 tx=0).stop 2024-09-13T18:53:13.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.324+0000 7f8515cf4700 1 -- 172.21.15.7:0/2700656247 shutdown_connections 2024-09-13T18:53:13.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.324+0000 7f8515cf4700 1 --2- 172.21.15.7:0/2700656247 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f84f0066730 0x7f84f0068bf0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.324+0000 7f8515cf4700 1 --2- 172.21.15.7:0/2700656247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8510164f90 0x7f851015cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.324+0000 7f8515cf4700 1 --2- 172.21.15.7:0/2700656247 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8510070bd0 0x7f851015abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.324+0000 7f8515cf4700 1 --2- 172.21.15.7:0/2700656247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f851006d8c0 0x7f851015e280 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.324+0000 7f8515cf4700 1 -- 172.21.15.7:0/2700656247 >> 172.21.15.7:0/2700656247 conn(0x7f85100b0e10 msgr2=0x7f851015f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:13.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.324+0000 7f8515cf4700 1 -- 172.21.15.7:0/2700656247 shutdown_connections 2024-09-13T18:53:13.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.324+0000 7f8515cf4700 1 -- 172.21.15.7:0/2700656247 wait complete. 2024-09-13T18:53:13.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-13T18:53:13.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.456+0000 7f63d8e63700 1 Processor -- start 2024-09-13T18:53:13.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.456+0000 7f63d8e63700 1 -- start start 2024-09-13T18:53:13.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.456+0000 7f63d8e63700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f63d406d3f0 0x7f63d415a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.456+0000 7f63d8e63700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d415ac50 con 0x7f63d41573e0 2024-09-13T18:53:13.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.456+0000 7f63d8e63700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d40cb3f0 con 0x7f63d40712b0 2024-09-13T18:53:13.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.456+0000 7f63d8e63700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d40cc1b0 con 0x7f63d406d3f0 2024-09-13T18:53:13.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.456+0000 7f63d3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f63d406d3f0 0x7f63d415a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.456+0000 7f63d3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f63d406d3f0 0x7f63d415a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46192/0 (socket says 172.21.15.7:46192) 2024-09-13T18:53:13.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.456+0000 7f63d3fff700 1 -- 172.21.15.7:0/1713443276 learned_addr learned my addr 172.21.15.7:0/1713443276 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:13.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.456+0000 7f63d3fff700 1 -- 172.21.15.7:0/1713443276 >> v1:172.21.15.171:6789/0 conn(0x7f63d40712b0 legacy=0x7f63d4159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.456+0000 7f63d3fff700 1 -- 172.21.15.7:0/1713443276 >> v1:172.21.15.7:6789/0 conn(0x7f63d41573e0 legacy=0x7f63d4070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.456+0000 7f63d3fff700 1 -- 172.21.15.7:0/1713443276 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d40cb6d0 con 0x7f63d406d3f0 2024-09-13T18:53:13.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.456+0000 7f63d3fff700 1 --2- 172.21.15.7:0/1713443276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d406d3f0 0x7f63d415a630 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009910 tx=0x7f63cc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af99d3ed79ec8c77 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:13.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.456+0000 7f63d27fc700 1 -- 172.21.15.7:0/1713443276 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc00ba60 con 0x7f63d406d3f0 2024-09-13T18:53:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.456+0000 7f63d27fc700 1 -- 172.21.15.7:0/1713443276 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63cc00bbc0 con 0x7f63d406d3f0 2024-09-13T18:53:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.456+0000 7f63d27fc700 1 -- 172.21.15.7:0/1713443276 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc009e60 con 0x7f63d406d3f0 2024-09-13T18:53:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.460+0000 7f63d8e63700 1 -- 172.21.15.7:0/1713443276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d406d3f0 msgr2=0x7f63d415a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.460+0000 7f63d8e63700 1 --2- 172.21.15.7:0/1713443276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d406d3f0 0x7f63d415a630 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f63cc009910 tx=0x7f63cc009c20 comp rx=0 tx=0).stop 2024-09-13T18:53:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.460+0000 7f63d8e63700 1 -- 172.21.15.7:0/1713443276 shutdown_connections 2024-09-13T18:53:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.460+0000 7f63d8e63700 1 --2- 172.21.15.7:0/1713443276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d406d3f0 0x7f63d415a630 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.460+0000 7f63d8e63700 1 -- 172.21.15.7:0/1713443276 >> 172.21.15.7:0/1713443276 conn(0x7f63d40b0e10 msgr2=0x7f63d40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.460+0000 7f63d8e63700 1 -- 172.21.15.7:0/1713443276 shutdown_connections 2024-09-13T18:53:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.460+0000 7f63d8e63700 1 -- 172.21.15.7:0/1713443276 wait complete. 2024-09-13T18:53:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.460+0000 7f63d8e63700 1 Processor -- start 2024-09-13T18:53:13.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.460+0000 7f63d8e63700 1 -- start start 2024-09-13T18:53:13.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d8e63700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f63d406d3f0 0x7f63d40cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d8e63700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63d40712b0 0x7f63d4153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d8e63700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d41573e0 0x7f63d414a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d8e63700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d40cd410 con 0x7f63d41573e0 2024-09-13T18:53:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d8e63700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d40cd6f0 con 0x7f63d40712b0 2024-09-13T18:53:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d8e63700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d40cd9d0 con 0x7f63d406d3f0 2024-09-13T18:53:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d41573e0 0x7f63d414a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d41573e0 0x7f63d414a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46206/0 (socket says 172.21.15.7:46206) 2024-09-13T18:53:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d3fff700 1 -- 172.21.15.7:0/2295316478 learned_addr learned my addr 172.21.15.7:0/2295316478 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d2ffd700 1 --2- 172.21.15.7:0/2295316478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63d40712b0 0x7f63d4153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d37fe700 1 --2- 172.21.15.7:0/2295316478 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f63d406d3f0 0x7f63d40cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d3fff700 1 -- 172.21.15.7:0/2295316478 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f63d406d3f0 msgr2=0x7f63d40cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d3fff700 1 --2- 172.21.15.7:0/2295316478 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f63d406d3f0 0x7f63d40cce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d3fff700 1 -- 172.21.15.7:0/2295316478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63d40712b0 msgr2=0x7f63d4153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d3fff700 1 --2- 172.21.15.7:0/2295316478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63d40712b0 0x7f63d4153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d3fff700 1 -- 172.21.15.7:0/2295316478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc009570 con 0x7f63d41573e0 2024-09-13T18:53:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d3fff700 1 --2- 172.21.15.7:0/2295316478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d41573e0 0x7f63d414a5a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f63d4157850 tx=0x7f63cc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63bbfff700 1 -- 172.21.15.7:0/2295316478 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc01e070 con 0x7f63d41573e0 2024-09-13T18:53:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63bbfff700 1 -- 172.21.15.7:0/2295316478 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63cc0036c0 con 0x7f63d41573e0 2024-09-13T18:53:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d8e63700 1 -- 172.21.15.7:0/2295316478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d40cdcb0 con 0x7f63d41573e0 2024-09-13T18:53:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63bbfff700 1 -- 172.21.15.7:0/2295316478 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63cc003a50 con 0x7f63d41573e0 2024-09-13T18:53:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d8e63700 1 -- 172.21.15.7:0/2295316478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d41538d0 con 0x7f63d41573e0 2024-09-13T18:53:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.464+0000 7f63d8e63700 1 -- 172.21.15.7:0/2295316478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d40ce1d0 con 0x7f63d41573e0 2024-09-13T18:53:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.468+0000 7f63bbfff700 1 -- 172.21.15.7:0/2295316478 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f63cc003bb0 con 0x7f63d41573e0 2024-09-13T18:53:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.468+0000 7f63bbfff700 1 --2- 172.21.15.7:0/2295316478 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f63c0066760 0x7f63c0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.468+0000 7f63bbfff700 1 -- 172.21.15.7:0/2295316478 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f63cc086cd0 con 0x7f63d41573e0 2024-09-13T18:53:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.468+0000 7f63d37fe700 1 --2- 172.21.15.7:0/2295316478 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f63c0066760 0x7f63c0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.468+0000 7f63d37fe700 1 --2- 172.21.15.7:0/2295316478 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f63c0066760 0x7f63c0068c20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f63c8000f60 tx=0x7f63c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.476+0000 7f63bbfff700 1 -- 172.21.15.7:0/2295316478 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f63d40ce1d0 con 0x7f63d41573e0 2024-09-13T18:53:13.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.708+0000 7f63d8e63700 1 -- 172.21.15.7:0/2295316478 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f63d40ce1d0 con 0x7f63d41573e0 2024-09-13T18:53:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.712+0000 7f63bbfff700 1 -- 172.21.15.7:0/2295316478 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f63d40ce1d0 con 0x7f63d41573e0 2024-09-13T18:53:13.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.716+0000 7f63d8e63700 1 -- 172.21.15.7:0/2295316478 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f63c0066760 msgr2=0x7f63c0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.716+0000 7f63d8e63700 1 --2- 172.21.15.7:0/2295316478 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f63c0066760 0x7f63c0068c20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f63c8000f60 tx=0x7f63c8009040 comp rx=0 tx=0).stop 2024-09-13T18:53:13.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.716+0000 7f63d8e63700 1 -- 172.21.15.7:0/2295316478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d41573e0 msgr2=0x7f63d414a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.716+0000 7f63d8e63700 1 --2- 172.21.15.7:0/2295316478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d41573e0 0x7f63d414a5a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f63d4157850 tx=0x7f63cc004770 comp rx=0 tx=0).stop 2024-09-13T18:53:13.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.716+0000 7f63d8e63700 1 -- 172.21.15.7:0/2295316478 shutdown_connections 2024-09-13T18:53:13.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.716+0000 7f63d8e63700 1 --2- 172.21.15.7:0/2295316478 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f63c0066760 0x7f63c0068c20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.716+0000 7f63d8e63700 1 --2- 172.21.15.7:0/2295316478 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63d41573e0 0x7f63d414a5a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.716+0000 7f63d8e63700 1 --2- 172.21.15.7:0/2295316478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f63d40712b0 0x7f63d4153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.716+0000 7f63d8e63700 1 --2- 172.21.15.7:0/2295316478 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f63d406d3f0 0x7f63d40cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.716+0000 7f63d8e63700 1 -- 172.21.15.7:0/2295316478 >> 172.21.15.7:0/2295316478 conn(0x7f63d40b0e10 msgr2=0x7f63d406a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:13.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.716+0000 7f63d8e63700 1 -- 172.21.15.7:0/2295316478 shutdown_connections 2024-09-13T18:53:13.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.716+0000 7f63d8e63700 1 -- 172.21.15.7:0/2295316478 wait complete. 2024-09-13T18:53:13.740 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-13T18:53:13.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.844+0000 7f5dd7fff700 1 Processor -- start 2024-09-13T18:53:13.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.844+0000 7f5dd7fff700 1 -- start start 2024-09-13T18:53:13.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.844+0000 7f5dd7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5dd806d8c0 0x7f5dd8164a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.844+0000 7f5dd7fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dd8164f90 0x7f5dd80704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.844+0000 7f5dd7fff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f5dd8070a00 0x7f5dd8161460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.844+0000 7f5dd7fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd8162e60 con 0x7f5dd806d8c0 2024-09-13T18:53:13.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.844+0000 7f5dd7fff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd8162570 con 0x7f5dd8070a00 2024-09-13T18:53:13.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.844+0000 7f5dd7fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd81653f0 con 0x7f5dd8164f90 2024-09-13T18:53:13.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.844+0000 7f5dd6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5dd806d8c0 0x7f5dd8164a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.844+0000 7f5dd6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5dd806d8c0 0x7f5dd8164a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46222/0 (socket says 172.21.15.7:46222) 2024-09-13T18:53:13.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.844+0000 7f5dd6ffd700 1 -- 172.21.15.7:0/418302926 learned_addr learned my addr 172.21.15.7:0/418302926 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:13.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.844+0000 7f5dd67fc700 1 --2- 172.21.15.7:0/418302926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dd8164f90 0x7f5dd80704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.844+0000 7f5dd77fe700 1 --2- 172.21.15.7:0/418302926 >> v2:172.21.15.171:3300/0 conn(0x7f5dd8070a00 0x7f5dd8161460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd6ffd700 1 -- 172.21.15.7:0/418302926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dd8164f90 msgr2=0x7f5dd80704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd6ffd700 1 --2- 172.21.15.7:0/418302926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dd8164f90 0x7f5dd80704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd6ffd700 1 -- 172.21.15.7:0/418302926 >> v2:172.21.15.171:3300/0 conn(0x7f5dd8070a00 msgr2=0x7f5dd8161460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd6ffd700 1 --2- 172.21.15.7:0/418302926 >> v2:172.21.15.171:3300/0 conn(0x7f5dd8070a00 0x7f5dd8161460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd6ffd700 1 -- 172.21.15.7:0/418302926 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd8071240 con 0x7f5dd806d8c0 2024-09-13T18:53:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd6ffd700 1 --2- 172.21.15.7:0/418302926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dd806d8c0 0x7f5dd8164a50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5dd815bab0 tx=0x7f5dc8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add784f60959109e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd5ffb700 1 -- 172.21.15.7:0/418302926 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8011830 con 0x7f5dd806d8c0 2024-09-13T18:53:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd5ffb700 1 -- 172.21.15.7:0/418302926 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc8011990 con 0x7f5dd806d8c0 2024-09-13T18:53:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd5ffb700 1 -- 172.21.15.7:0/418302926 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8011d20 con 0x7f5dd806d8c0 2024-09-13T18:53:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd7fff700 1 -- 172.21.15.7:0/418302926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dd806d8c0 msgr2=0x7f5dd8164a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd7fff700 1 --2- 172.21.15.7:0/418302926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dd806d8c0 0x7f5dd8164a50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5dd815bab0 tx=0x7f5dc8009bf0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd7fff700 1 -- 172.21.15.7:0/418302926 shutdown_connections 2024-09-13T18:53:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd7fff700 1 --2- 172.21.15.7:0/418302926 >> v2:172.21.15.171:3300/0 conn(0x7f5dd8070a00 0x7f5dd8161460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd7fff700 1 --2- 172.21.15.7:0/418302926 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dd8164f90 0x7f5dd80704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd7fff700 1 --2- 172.21.15.7:0/418302926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dd806d8c0 0x7f5dd8164a50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd7fff700 1 -- 172.21.15.7:0/418302926 >> 172.21.15.7:0/418302926 conn(0x7f5dd80b0e10 msgr2=0x7f5dd80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd7fff700 1 -- 172.21.15.7:0/418302926 shutdown_connections 2024-09-13T18:53:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd7fff700 1 -- 172.21.15.7:0/418302926 wait complete. 2024-09-13T18:53:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd7fff700 1 Processor -- start 2024-09-13T18:53:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.848+0000 7f5dd7fff700 1 -- start start 2024-09-13T18:53:13.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dd8070a00 0x7f5dd8148830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd7fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dd8164f90 0x7f5dd815c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd7fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dd815ca30 0x7f5dd8163820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd7fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd815db50 con 0x7f5dd8070a00 2024-09-13T18:53:13.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd7fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd815ced0 con 0x7f5dd815ca30 2024-09-13T18:53:13.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd7fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd815d1b0 con 0x7f5dd8164f90 2024-09-13T18:53:13.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dd8070a00 0x7f5dd8148830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dd8070a00 0x7f5dd8148830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46232/0 (socket says 172.21.15.7:46232) 2024-09-13T18:53:13.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd6ffd700 1 -- 172.21.15.7:0/1329940747 learned_addr learned my addr 172.21.15.7:0/1329940747 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:13.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd77fe700 1 --2- 172.21.15.7:0/1329940747 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dd815ca30 0x7f5dd8163820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd67fc700 1 --2- 172.21.15.7:0/1329940747 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dd8164f90 0x7f5dd815c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd6ffd700 1 -- 172.21.15.7:0/1329940747 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dd8164f90 msgr2=0x7f5dd815c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd6ffd700 1 --2- 172.21.15.7:0/1329940747 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dd8164f90 0x7f5dd815c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd6ffd700 1 -- 172.21.15.7:0/1329940747 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dd815ca30 msgr2=0x7f5dd8163820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:13.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd6ffd700 1 --2- 172.21.15.7:0/1329940747 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dd815ca30 0x7f5dd8163820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:13.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd6ffd700 1 -- 172.21.15.7:0/1329940747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc8009570 con 0x7f5dd8070a00 2024-09-13T18:53:13.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd6ffd700 1 --2- 172.21.15.7:0/1329940747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dd8070a00 0x7f5dd8148830 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8013000 tx=0x7f5dc8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:13.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dcffff700 1 -- 172.21.15.7:0/1329940747 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8004450 con 0x7f5dd8070a00 2024-09-13T18:53:13.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dcffff700 1 -- 172.21.15.7:0/1329940747 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc801f070 con 0x7f5dd8070a00 2024-09-13T18:53:13.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd7fff700 1 -- 172.21.15.7:0/1329940747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd815d490 con 0x7f5dd8070a00 2024-09-13T18:53:13.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dcffff700 1 -- 172.21.15.7:0/1329940747 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8003900 con 0x7f5dd8070a00 2024-09-13T18:53:13.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd7fff700 1 -- 172.21.15.7:0/1329940747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd8148d70 con 0x7f5dd8070a00 2024-09-13T18:53:13.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dd7fff700 1 -- 172.21.15.7:0/1329940747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd8148f00 con 0x7f5dd8070a00 2024-09-13T18:53:13.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.852+0000 7f5dcffff700 1 -- 172.21.15.7:0/1329940747 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5dc8003a60 con 0x7f5dd8070a00 2024-09-13T18:53:13.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.856+0000 7f5dcffff700 1 --2- 172.21.15.7:0/1329940747 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5dbc066730 0x7f5dbc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:13.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.856+0000 7f5dcffff700 1 -- 172.21.15.7:0/1329940747 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5dc8021070 con 0x7f5dd8070a00 2024-09-13T18:53:13.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.856+0000 7f5dd67fc700 1 --2- 172.21.15.7:0/1329940747 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5dbc066730 0x7f5dbc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.856+0000 7f5dd67fc700 1 --2- 172.21.15.7:0/1329940747 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5dbc066730 0x7f5dbc068bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5dd81fa130 tx=0x7f5dc0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:13.860+0000 7f5dcffff700 1 -- 172.21.15.7:0/1329940747 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5dd8148f00 con 0x7f5dd8070a00 2024-09-13T18:53:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.096+0000 7f5dd7fff700 1 -- 172.21.15.7:0/1329940747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f5dd8148f00 con 0x7f5dd8070a00 2024-09-13T18:53:14.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.104+0000 7f5dcffff700 1 -- 172.21.15.7:0/1329940747 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5dd8148f00 con 0x7f5dd8070a00 2024-09-13T18:53:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.108+0000 7f5dd7fff700 1 -- 172.21.15.7:0/1329940747 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5dbc066730 msgr2=0x7f5dbc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.108+0000 7f5dd7fff700 1 --2- 172.21.15.7:0/1329940747 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5dbc066730 0x7f5dbc068bf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5dd81fa130 tx=0x7f5dc0008040 comp rx=0 tx=0).stop 2024-09-13T18:53:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.108+0000 7f5dd7fff700 1 -- 172.21.15.7:0/1329940747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dd8070a00 msgr2=0x7f5dd8148830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.108+0000 7f5dd7fff700 1 --2- 172.21.15.7:0/1329940747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dd8070a00 0x7f5dd8148830 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8013000 tx=0x7f5dc8004770 comp rx=0 tx=0).stop 2024-09-13T18:53:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.108+0000 7f5dd7fff700 1 -- 172.21.15.7:0/1329940747 shutdown_connections 2024-09-13T18:53:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.108+0000 7f5dd7fff700 1 --2- 172.21.15.7:0/1329940747 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5dbc066730 0x7f5dbc068bf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.108+0000 7f5dd7fff700 1 --2- 172.21.15.7:0/1329940747 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5dd815ca30 0x7f5dd8163820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.108+0000 7f5dd7fff700 1 --2- 172.21.15.7:0/1329940747 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5dd8164f90 0x7f5dd815c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.108+0000 7f5dd7fff700 1 --2- 172.21.15.7:0/1329940747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5dd8070a00 0x7f5dd8148830 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.108+0000 7f5dd7fff700 1 -- 172.21.15.7:0/1329940747 >> 172.21.15.7:0/1329940747 conn(0x7f5dd80b0e10 msgr2=0x7f5dd80af300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:14.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.108+0000 7f5dd7fff700 1 -- 172.21.15.7:0/1329940747 shutdown_connections 2024-09-13T18:53:14.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.108+0000 7f5dd7fff700 1 -- 172.21.15.7:0/1329940747 wait complete. 2024-09-13T18:53:14.132 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-13T18:53:14.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.244+0000 7f2110e51700 1 Processor -- start 2024-09-13T18:53:14.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.244+0000 7f2110e51700 1 -- start start 2024-09-13T18:53:14.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.244+0000 7f2110e51700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f210c1573e0 0x7f210c070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:14.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.244+0000 7f2110e51700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f210c15a630 0x7f210c06c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:14.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.244+0000 7f2110e51700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210c071540 con 0x7f210c0710e0 2024-09-13T18:53:14.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.244+0000 7f2110e51700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f210c0cc1b0 con 0x7f210c15a630 2024-09-13T18:53:14.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.244+0000 7f2110e51700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210c15ac50 con 0x7f210c1573e0 2024-09-13T18:53:14.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.244+0000 7f210bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f210c15a630 0x7f210c06c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:14.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.244+0000 7f210bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f210c15a630 0x7f210c06c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46246/0 (socket says 172.21.15.7:46246) 2024-09-13T18:53:14.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.244+0000 7f210bfff700 1 -- 172.21.15.7:0/1363477273 learned_addr learned my addr 172.21.15.7:0/1363477273 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:14.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.244+0000 7f210b7fe700 1 --2- 172.21.15.7:0/1363477273 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f210c1573e0 0x7f210c070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f210bfff700 1 -- 172.21.15.7:0/1363477273 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f210c1573e0 msgr2=0x7f210c070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f210bfff700 1 --2- 172.21.15.7:0/1363477273 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f210c1573e0 0x7f210c070ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f210bfff700 1 -- 172.21.15.7:0/1363477273 >> v1:172.21.15.171:6789/0 conn(0x7f210c0710e0 legacy=0x7f210c159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f210affd700 1 -- 172.21.15.7:0/1363477273 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438629053 0 0) 0x7f210c071540 con 0x7f210c0710e0 2024-09-13T18:53:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f210bfff700 1 -- 172.21.15.7:0/1363477273 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c0cb3f0 con 0x7f210c15a630 2024-09-13T18:53:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f210bfff700 1 --2- 172.21.15.7:0/1363477273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c15a630 0x7f210c06c3e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2104009e80 tx=0x7f210400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2523da2014fc16a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f210affd700 1 -- 172.21.15.7:0/1363477273 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210400e070 con 0x7f210c15a630 2024-09-13T18:53:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f210affd700 1 -- 172.21.15.7:0/1363477273 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210400b8d0 con 0x7f210c15a630 2024-09-13T18:53:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f210affd700 1 -- 172.21.15.7:0/1363477273 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210400bc80 con 0x7f210c15a630 2024-09-13T18:53:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f2110e51700 1 -- 172.21.15.7:0/1363477273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c15a630 msgr2=0x7f210c06c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f2110e51700 1 --2- 172.21.15.7:0/1363477273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c15a630 0x7f210c06c3e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2104009e80 tx=0x7f210400b330 comp rx=0 tx=0).stop 2024-09-13T18:53:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f2110e51700 1 -- 172.21.15.7:0/1363477273 shutdown_connections 2024-09-13T18:53:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f2110e51700 1 --2- 172.21.15.7:0/1363477273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c15a630 0x7f210c06c3e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f2110e51700 1 --2- 172.21.15.7:0/1363477273 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f210c1573e0 0x7f210c070ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f2110e51700 1 -- 172.21.15.7:0/1363477273 >> 172.21.15.7:0/1363477273 conn(0x7f210c0b0e10 msgr2=0x7f210c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f2110e51700 1 -- 172.21.15.7:0/1363477273 shutdown_connections 2024-09-13T18:53:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f2110e51700 1 -- 172.21.15.7:0/1363477273 wait complete. 2024-09-13T18:53:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f2110e51700 1 Processor -- start 2024-09-13T18:53:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.248+0000 7f2110e51700 1 -- start start 2024-09-13T18:53:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f2110e51700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c0710e0 0x7f210c1419b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f2110e51700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f210c1573e0 0x7f210c0ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f2110e51700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f210c15a630 0x7f210c0ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f2110e51700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c0ce6b0 con 0x7f210c0710e0 2024-09-13T18:53:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f2110e51700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c157e00 con 0x7f210c15a630 2024-09-13T18:53:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f2110e51700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210c141ef0 con 0x7f210c1573e0 2024-09-13T18:53:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f210b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c0710e0 0x7f210c1419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f210b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c0710e0 0x7f210c1419b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46258/0 (socket says 172.21.15.7:46258) 2024-09-13T18:53:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f210b7fe700 1 -- 172.21.15.7:0/1416393536 learned_addr learned my addr 172.21.15.7:0/1416393536 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f2102ffd700 1 --2- 172.21.15.7:0/1416393536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f210c1573e0 0x7f210c0ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f210bfff700 1 --2- 172.21.15.7:0/1416393536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f210c15a630 0x7f210c0ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f210b7fe700 1 -- 172.21.15.7:0/1416393536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f210c1573e0 msgr2=0x7f210c0ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f210b7fe700 1 --2- 172.21.15.7:0/1416393536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f210c1573e0 0x7f210c0ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f210b7fe700 1 -- 172.21.15.7:0/1416393536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f210c15a630 msgr2=0x7f210c0ace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f210b7fe700 1 --2- 172.21.15.7:0/1416393536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f210c15a630 0x7f210c0ace00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f210b7fe700 1 -- 172.21.15.7:0/1416393536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2104009b30 con 0x7f210c0710e0 2024-09-13T18:53:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f210b7fe700 1 --2- 172.21.15.7:0/1416393536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c0710e0 0x7f210c1419b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00e9d0 tx=0x7f20fc00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f2108ff9700 1 -- 172.21.15.7:0/1416393536 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc009720 con 0x7f210c0710e0 2024-09-13T18:53:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f2108ff9700 1 -- 172.21.15.7:0/1416393536 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc00ee70 con 0x7f210c0710e0 2024-09-13T18:53:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f2110e51700 1 -- 172.21.15.7:0/1416393536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210c0ce0f0 con 0x7f210c0710e0 2024-09-13T18:53:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f2108ff9700 1 -- 172.21.15.7:0/1416393536 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc00ca50 con 0x7f210c0710e0 2024-09-13T18:53:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.252+0000 7f2110e51700 1 -- 172.21.15.7:0/1416393536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210c157a20 con 0x7f210c0710e0 2024-09-13T18:53:14.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.256+0000 7f2110e51700 1 -- 172.21.15.7:0/1416393536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210c158070 con 0x7f210c0710e0 2024-09-13T18:53:14.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.256+0000 7f2108ff9700 1 -- 172.21.15.7:0/1416393536 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f20fc00cbb0 con 0x7f210c0710e0 2024-09-13T18:53:14.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.256+0000 7f2108ff9700 1 --2- 172.21.15.7:0/1416393536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f20ec066760 0x7f20ec068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:14.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.256+0000 7f2108ff9700 1 -- 172.21.15.7:0/1416393536 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f20fc083db0 con 0x7f210c0710e0 2024-09-13T18:53:14.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.256+0000 7f2102ffd700 1 --2- 172.21.15.7:0/1416393536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f20ec066760 0x7f20ec068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:14.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.260+0000 7f2102ffd700 1 --2- 172.21.15.7:0/1416393536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f20ec066760 0x7f20ec068c20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f210c1f5e10 tx=0x7f20f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:14.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.264+0000 7f2108ff9700 1 -- 172.21.15.7:0/1416393536 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f20fc051470 con 0x7f210c0710e0 2024-09-13T18:53:14.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.496+0000 7f2110e51700 1 -- 172.21.15.7:0/1416393536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f210c157880 con 0x7f210c0710e0 2024-09-13T18:53:14.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.500+0000 7f2108ff9700 1 -- 172.21.15.7:0/1416393536 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f210c157880 con 0x7f210c0710e0 2024-09-13T18:53:14.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.500+0000 7f2110e51700 1 -- 172.21.15.7:0/1416393536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f20ec066760 msgr2=0x7f20ec068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.500+0000 7f2110e51700 1 --2- 172.21.15.7:0/1416393536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f20ec066760 0x7f20ec068c20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f210c1f5e10 tx=0x7f20f4009040 comp rx=0 tx=0).stop 2024-09-13T18:53:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.500+0000 7f2110e51700 1 -- 172.21.15.7:0/1416393536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c0710e0 msgr2=0x7f210c1419b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.500+0000 7f2110e51700 1 --2- 172.21.15.7:0/1416393536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c0710e0 0x7f210c1419b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00e9d0 tx=0x7f20fc00ece0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.500+0000 7f2110e51700 1 -- 172.21.15.7:0/1416393536 shutdown_connections 2024-09-13T18:53:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.500+0000 7f2110e51700 1 --2- 172.21.15.7:0/1416393536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f20ec066760 0x7f20ec068c20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.500+0000 7f2110e51700 1 --2- 172.21.15.7:0/1416393536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f210c15a630 0x7f210c0ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.500+0000 7f2110e51700 1 --2- 172.21.15.7:0/1416393536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f210c1573e0 0x7f210c0ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.500+0000 7f2110e51700 1 --2- 172.21.15.7:0/1416393536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c0710e0 0x7f210c1419b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.500+0000 7f2110e51700 1 -- 172.21.15.7:0/1416393536 >> 172.21.15.7:0/1416393536 conn(0x7f210c0b0e10 msgr2=0x7f210c15b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.504+0000 7f2110e51700 1 -- 172.21.15.7:0/1416393536 shutdown_connections 2024-09-13T18:53:14.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.504+0000 7f2110e51700 1 -- 172.21.15.7:0/1416393536 wait complete. 2024-09-13T18:53:14.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:53:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.628+0000 7f07d6994700 1 Processor -- start 2024-09-13T18:53:14.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07d6994700 1 -- start start 2024-09-13T18:53:14.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07d6994700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f07d015b890 0x7f07d0162220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:14.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07d6994700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f07d00cdcc0 0x7f07d00b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07d6994700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d00b2dc0 con 0x7f07d00b2930 2024-09-13T18:53:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07d6994700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d0162760 con 0x7f07d015b890 2024-09-13T18:53:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07d6994700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d015bc70 con 0x7f07d00cdcc0 2024-09-13T18:53:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07d4f31700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f07d00b2930 0x7f07d006ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47350/0 (socket says 172.21.15.7:47350) 2024-09-13T18:53:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07cffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f07d015b890 0x7f07d0162220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07d4f31700 1 -- 172.21.15.7:0/1012158565 learned_addr learned my addr 172.21.15.7:0/1012158565 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07cf7fe700 1 --2- 172.21.15.7:0/1012158565 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f07d00cdcc0 0x7f07d00b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07ceffd700 1 -- 172.21.15.7:0/1012158565 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973876134 0 0) 0x7f07c8001490 con 0x7f07d00b2930 2024-09-13T18:53:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07ceffd700 1 -- 172.21.15.7:0/1012158565 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07bc0035d0 con 0x7f07d00b2930 2024-09-13T18:53:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07cf7fe700 1 -- 172.21.15.7:0/1012158565 >> v2:172.21.15.7:3300/0 conn(0x7f07d015b890 msgr2=0x7f07d0162220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07cf7fe700 1 --2- 172.21.15.7:0/1012158565 >> v2:172.21.15.7:3300/0 conn(0x7f07d015b890 0x7f07d0162220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07cf7fe700 1 -- 172.21.15.7:0/1012158565 >> v1:172.21.15.7:6789/0 conn(0x7f07d00b2930 legacy=0x7f07d006ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07cf7fe700 1 -- 172.21.15.7:0/1012158565 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d00ccc20 con 0x7f07d00cdcc0 2024-09-13T18:53:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07cf7fe700 1 --2- 172.21.15.7:0/1012158565 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f07d00cdcc0 0x7f07d00b23f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009850 tx=0x7f07b8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80be3a960efa00f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07ceffd700 1 -- 172.21.15.7:0/1012158565 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8011830 con 0x7f07d00cdcc0 2024-09-13T18:53:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07ceffd700 1 -- 172.21.15.7:0/1012158565 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b8011990 con 0x7f07d00cdcc0 2024-09-13T18:53:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07ceffd700 1 -- 172.21.15.7:0/1012158565 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8011d00 con 0x7f07d00cdcc0 2024-09-13T18:53:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07d6994700 1 -- 172.21.15.7:0/1012158565 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f07d00cdcc0 msgr2=0x7f07d00b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07d6994700 1 --2- 172.21.15.7:0/1012158565 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f07d00cdcc0 0x7f07d00b23f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009850 tx=0x7f07b8009b60 comp rx=0 tx=0).stop 2024-09-13T18:53:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07d6994700 1 -- 172.21.15.7:0/1012158565 shutdown_connections 2024-09-13T18:53:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07d6994700 1 --2- 172.21.15.7:0/1012158565 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f07d00cdcc0 0x7f07d00b23f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.632+0000 7f07d6994700 1 --2- 172.21.15.7:0/1012158565 >> v2:172.21.15.7:3300/0 conn(0x7f07d015b890 0x7f07d0162220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.636+0000 7f07d6994700 1 -- 172.21.15.7:0/1012158565 >> 172.21.15.7:0/1012158565 conn(0x7f07d00ce980 msgr2=0x7f07d00ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.636+0000 7f07d6994700 1 -- 172.21.15.7:0/1012158565 shutdown_connections 2024-09-13T18:53:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.636+0000 7f07d6994700 1 -- 172.21.15.7:0/1012158565 wait complete. 2024-09-13T18:53:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.636+0000 7f07d6994700 1 Processor -- start 2024-09-13T18:53:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.636+0000 7f07d6994700 1 -- start start 2024-09-13T18:53:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07d6994700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07d00b2930 0x7f07d01f7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07d6994700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f07d00cdcc0 0x7f07d01559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07d6994700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07d015b890 0x7f07d0158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07d6994700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d01f7360 con 0x7f07d015b890 2024-09-13T18:53:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07d6994700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d01588b0 con 0x7f07d00b2930 2024-09-13T18:53:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07d6994700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d0158b90 con 0x7f07d00cdcc0 2024-09-13T18:53:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07d4f31700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07d015b890 0x7f07d0158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07d4f31700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07d015b890 0x7f07d0158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46280/0 (socket says 172.21.15.7:46280) 2024-09-13T18:53:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07d4f31700 1 -- 172.21.15.7:0/2331427543 learned_addr learned my addr 172.21.15.7:0/2331427543 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07cf7fe700 1 --2- 172.21.15.7:0/2331427543 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f07d00cdcc0 0x7f07d01559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07cffff700 1 --2- 172.21.15.7:0/2331427543 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07d00b2930 0x7f07d01f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07d4f31700 1 -- 172.21.15.7:0/2331427543 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f07d00cdcc0 msgr2=0x7f07d01559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07d4f31700 1 --2- 172.21.15.7:0/2331427543 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f07d00cdcc0 0x7f07d01559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07d4f31700 1 -- 172.21.15.7:0/2331427543 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07d00b2930 msgr2=0x7f07d01f7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07d4f31700 1 --2- 172.21.15.7:0/2331427543 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07d00b2930 0x7f07d01f7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07d4f31700 1 -- 172.21.15.7:0/2331427543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b80094b0 con 0x7f07d015b890 2024-09-13T18:53:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07d4f31700 1 --2- 172.21.15.7:0/2331427543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07d015b890 0x7f07d0158080 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f07c800cae0 tx=0x7f07c800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07ccff9700 1 -- 172.21.15.7:0/2331427543 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c800a7e0 con 0x7f07d015b890 2024-09-13T18:53:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07ccff9700 1 -- 172.21.15.7:0/2331427543 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c800a940 con 0x7f07d015b890 2024-09-13T18:53:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07d6994700 1 -- 172.21.15.7:0/2331427543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d01f6ca0 con 0x7f07d015b890 2024-09-13T18:53:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07ccff9700 1 -- 172.21.15.7:0/2331427543 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c800acd0 con 0x7f07d015b890 2024-09-13T18:53:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07d6994700 1 -- 172.21.15.7:0/2331427543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d01585c0 con 0x7f07d015b890 2024-09-13T18:53:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.640+0000 7f07d6994700 1 -- 172.21.15.7:0/2331427543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d01f74a0 con 0x7f07d015b890 2024-09-13T18:53:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.644+0000 7f07ccff9700 1 -- 172.21.15.7:0/2331427543 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f07c801b3b0 con 0x7f07d015b890 2024-09-13T18:53:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.644+0000 7f07ccff9700 1 --2- 172.21.15.7:0/2331427543 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f07b0066760 0x7f07b0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.644+0000 7f07ccff9700 1 -- 172.21.15.7:0/2331427543 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f07c80836a0 con 0x7f07d015b890 2024-09-13T18:53:14.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.644+0000 7f07cffff700 1 --2- 172.21.15.7:0/2331427543 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f07b0066760 0x7f07b0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:14.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.644+0000 7f07cffff700 1 --2- 172.21.15.7:0/2331427543 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f07b0066760 0x7f07b0068c20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009b80 tx=0x7f07c0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:14.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.652+0000 7f07ccff9700 1 -- 172.21.15.7:0/2331427543 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f07d01f74a0 con 0x7f07d015b890 2024-09-13T18:53:14.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.820+0000 7f07d6994700 1 -- 172.21.15.7:0/2331427543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07d00b2dc0 con 0x7f07d015b890 2024-09-13T18:53:14.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.824+0000 7f07ccff9700 1 -- 172.21.15.7:0/2331427543 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+12809 (secure 0 0 0) 0x7f07d00b2dc0 con 0x7f07d015b890 2024-09-13T18:53:14.830 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:14.830 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:53:11.683246+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:53:14.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.824+0000 7f07d6994700 1 -- 172.21.15.7:0/2331427543 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f07b0066760 msgr2=0x7f07b0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.824+0000 7f07d6994700 1 --2- 172.21.15.7:0/2331427543 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f07b0066760 0x7f07b0068c20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f07c0009b80 tx=0x7f07c0008040 comp rx=0 tx=0).stop 2024-09-13T18:53:14.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.824+0000 7f07d6994700 1 -- 172.21.15.7:0/2331427543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07d015b890 msgr2=0x7f07d0158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.824+0000 7f07d6994700 1 --2- 172.21.15.7:0/2331427543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07d015b890 0x7f07d0158080 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f07c800cae0 tx=0x7f07c800cbc0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.824+0000 7f07d6994700 1 -- 172.21.15.7:0/2331427543 shutdown_connections 2024-09-13T18:53:14.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.824+0000 7f07d6994700 1 --2- 172.21.15.7:0/2331427543 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f07b0066760 0x7f07b0068c20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.824+0000 7f07d6994700 1 --2- 172.21.15.7:0/2331427543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07d015b890 0x7f07d0158080 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.824+0000 7f07d6994700 1 --2- 172.21.15.7:0/2331427543 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f07d00cdcc0 0x7f07d01559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.824+0000 7f07d6994700 1 --2- 172.21.15.7:0/2331427543 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f07d00b2930 0x7f07d01f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.824+0000 7f07d6994700 1 -- 172.21.15.7:0/2331427543 >> 172.21.15.7:0/2331427543 conn(0x7f07d00ce980 msgr2=0x7f07d00ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:14.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.824+0000 7f07d6994700 1 -- 172.21.15.7:0/2331427543 shutdown_connections 2024-09-13T18:53:14.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.824+0000 7f07d6994700 1 -- 172.21.15.7:0/2331427543 wait complete. 2024-09-13T18:53:14.849 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:53:14.849 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:53:14.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.968+0000 7ff633fff700 1 Processor -- start 2024-09-13T18:53:14.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.968+0000 7ff633fff700 1 -- start start 2024-09-13T18:53:14.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.968+0000 7ff633fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff62c005e30 0x7ff62c006210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:14.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.968+0000 7ff633fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff62c0585e0 0x7ff62c006750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:14.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.968+0000 7ff633fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff62c057300 con 0x7ff62c0567d0 2024-09-13T18:53:14.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.968+0000 7ff633fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff62c0e2f80 con 0x7ff62c005e30 2024-09-13T18:53:14.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.968+0000 7ff633fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c058150 con 0x7ff62c0585e0 2024-09-13T18:53:14.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff632ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff62c005e30 0x7ff62c006210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:14.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff6337fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff62c0567d0 0x7ff62c056bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47354/0 (socket says 172.21.15.7:47354) 2024-09-13T18:53:14.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff6337fe700 1 -- 172.21.15.7:0/214255214 learned_addr learned my addr 172.21.15.7:0/214255214 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:14.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff6327fc700 1 --2- 172.21.15.7:0/214255214 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff62c0585e0 0x7ff62c006750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:14.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff632ffd700 1 -- 172.21.15.7:0/214255214 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff62c0585e0 msgr2=0x7ff62c006750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:14.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff632ffd700 1 --2- 172.21.15.7:0/214255214 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff62c0585e0 0x7ff62c006750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff632ffd700 1 -- 172.21.15.7:0/214255214 >> v1:172.21.15.7:6789/0 conn(0x7ff62c0567d0 legacy=0x7ff62c056bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff632ffd700 1 -- 172.21.15.7:0/214255214 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62c006c90 con 0x7ff62c005e30 2024-09-13T18:53:14.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff631ffb700 1 -- 172.21.15.7:0/214255214 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377236171 0 0) 0x7ff63404f5c0 con 0x7ff62c0567d0 2024-09-13T18:53:14.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff6327fc700 1 --2- 172.21.15.7:0/214255214 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff62c0585e0 0x7ff62c006750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:14.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff632ffd700 1 --2- 172.21.15.7:0/214255214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62c005e30 0x7ff62c006210 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff628009850 tx=0x7ff628009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b31ef02e9fd12f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:14.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff631ffb700 1 -- 172.21.15.7:0/214255214 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff628011830 con 0x7ff62c005e30 2024-09-13T18:53:14.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff631ffb700 1 -- 172.21.15.7:0/214255214 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff628011990 con 0x7ff62c005e30 2024-09-13T18:53:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff631ffb700 1 -- 172.21.15.7:0/214255214 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff628011d20 con 0x7ff62c005e30 2024-09-13T18:53:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff633fff700 1 -- 172.21.15.7:0/214255214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62c005e30 msgr2=0x7ff62c006210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff633fff700 1 --2- 172.21.15.7:0/214255214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62c005e30 0x7ff62c006210 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff628009850 tx=0x7ff628009b60 comp rx=0 tx=0).stop 2024-09-13T18:53:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff633fff700 1 -- 172.21.15.7:0/214255214 shutdown_connections 2024-09-13T18:53:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff633fff700 1 --2- 172.21.15.7:0/214255214 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff62c0585e0 0x7ff62c006750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff633fff700 1 --2- 172.21.15.7:0/214255214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62c005e30 0x7ff62c006210 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff633fff700 1 -- 172.21.15.7:0/214255214 >> 172.21.15.7:0/214255214 conn(0x7ff62c03a5c0 msgr2=0x7ff62c03a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff633fff700 1 -- 172.21.15.7:0/214255214 shutdown_connections 2024-09-13T18:53:14.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.972+0000 7ff633fff700 1 -- 172.21.15.7:0/214255214 wait complete. 2024-09-13T18:53:14.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff633fff700 1 Processor -- start 2024-09-13T18:53:14.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff633fff700 1 -- start start 2024-09-13T18:53:14.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff633fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62c005e30 0x7ff62c051ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:14.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff633fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff62c0567d0 0x7ff62c0d8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:14.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff633fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff62c0585e0 0x7ff62c16b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:14.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff633fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c04f490 con 0x7ff62c005e30 2024-09-13T18:53:14.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff633fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c04c4c0 con 0x7ff62c0567d0 2024-09-13T18:53:14.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff633fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c0521e0 con 0x7ff62c0585e0 2024-09-13T18:53:14.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff632ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62c005e30 0x7ff62c051ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:14.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff632ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62c005e30 0x7ff62c051ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46294/0 (socket says 172.21.15.7:46294) 2024-09-13T18:53:14.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff632ffd700 1 -- 172.21.15.7:0/2686137989 learned_addr learned my addr 172.21.15.7:0/2686137989 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:14.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff6327fc700 1 --2- 172.21.15.7:0/2686137989 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff62c0567d0 0x7ff62c0d8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:14.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff6337fe700 1 --2- 172.21.15.7:0/2686137989 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff62c0585e0 0x7ff62c16b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:14.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff632ffd700 1 -- 172.21.15.7:0/2686137989 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff62c0585e0 msgr2=0x7ff62c16b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff632ffd700 1 --2- 172.21.15.7:0/2686137989 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff62c0585e0 0x7ff62c16b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff632ffd700 1 -- 172.21.15.7:0/2686137989 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff62c0567d0 msgr2=0x7ff62c0d8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff632ffd700 1 --2- 172.21.15.7:0/2686137989 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff62c0567d0 0x7ff62c0d8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff632ffd700 1 -- 172.21.15.7:0/2686137989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6280094b0 con 0x7ff62c005e30 2024-09-13T18:53:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff632ffd700 1 --2- 172.21.15.7:0/2686137989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62c005e30 0x7ff62c051ca0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff6280090b0 tx=0x7ff628004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff61bfff700 1 -- 172.21.15.7:0/2686137989 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff628006ea0 con 0x7ff62c005e30 2024-09-13T18:53:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff61bfff700 1 -- 172.21.15.7:0/2686137989 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62801f070 con 0x7ff62c005e30 2024-09-13T18:53:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff633fff700 1 -- 172.21.15.7:0/2686137989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff62c04cd20 con 0x7ff62c005e30 2024-09-13T18:53:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff61bfff700 1 -- 172.21.15.7:0/2686137989 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280038c0 con 0x7ff62c005e30 2024-09-13T18:53:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.976+0000 7ff633fff700 1 -- 172.21.15.7:0/2686137989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62c04c6d0 con 0x7ff62c005e30 2024-09-13T18:53:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.980+0000 7ff633fff700 1 -- 172.21.15.7:0/2686137989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff62c056bf0 con 0x7ff62c005e30 2024-09-13T18:53:14.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.980+0000 7ff61bfff700 1 -- 172.21.15.7:0/2686137989 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff628003a20 con 0x7ff62c005e30 2024-09-13T18:53:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.980+0000 7ff61bfff700 1 --2- 172.21.15.7:0/2686137989 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff620066730 0x7ff620068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.980+0000 7ff61bfff700 1 -- 172.21.15.7:0/2686137989 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff628021070 con 0x7ff62c005e30 2024-09-13T18:53:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.980+0000 7ff6327fc700 1 --2- 172.21.15.7:0/2686137989 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff620066730 0x7ff620068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:14.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.984+0000 7ff6327fc700 1 --2- 172.21.15.7:0/2686137989 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff620066730 0x7ff620068bf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff62c03e410 tx=0x7ff624008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:14.988+0000 7ff61bfff700 1 -- 172.21.15.7:0/2686137989 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff628052ae0 con 0x7ff62c005e30 2024-09-13T18:53:15.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.160+0000 7ff633fff700 1 -- 172.21.15.7:0/2686137989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff62c055330 con 0x7ff62c005e30 2024-09-13T18:53:15.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.160+0000 7ff61bfff700 1 -- 172.21.15.7:0/2686137989 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff6280150a0 con 0x7ff62c005e30 2024-09-13T18:53:15.169 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:53:15.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.164+0000 7ff633fff700 1 -- 172.21.15.7:0/2686137989 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff620066730 msgr2=0x7ff620068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:15.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.164+0000 7ff633fff700 1 --2- 172.21.15.7:0/2686137989 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff620066730 0x7ff620068bf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff62c03e410 tx=0x7ff624008040 comp rx=0 tx=0).stop 2024-09-13T18:53:15.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.164+0000 7ff633fff700 1 -- 172.21.15.7:0/2686137989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62c005e30 msgr2=0x7ff62c051ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:15.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.164+0000 7ff633fff700 1 --2- 172.21.15.7:0/2686137989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62c005e30 0x7ff62c051ca0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff6280090b0 tx=0x7ff628004770 comp rx=0 tx=0).stop 2024-09-13T18:53:15.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.164+0000 7ff633fff700 1 -- 172.21.15.7:0/2686137989 shutdown_connections 2024-09-13T18:53:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.164+0000 7ff633fff700 1 --2- 172.21.15.7:0/2686137989 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff620066730 0x7ff620068bf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.164+0000 7ff633fff700 1 --2- 172.21.15.7:0/2686137989 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff62c0585e0 0x7ff62c16b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.164+0000 7ff633fff700 1 --2- 172.21.15.7:0/2686137989 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff62c0567d0 0x7ff62c0d8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.164+0000 7ff633fff700 1 --2- 172.21.15.7:0/2686137989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62c005e30 0x7ff62c051ca0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.164+0000 7ff633fff700 1 -- 172.21.15.7:0/2686137989 >> 172.21.15.7:0/2686137989 conn(0x7ff62c03a5c0 msgr2=0x7ff62c052fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.164+0000 7ff633fff700 1 -- 172.21.15.7:0/2686137989 shutdown_connections 2024-09-13T18:53:15.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.164+0000 7ff633fff700 1 -- 172.21.15.7:0/2686137989 wait complete. 2024-09-13T18:53:15.184 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T18:53:15.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.288+0000 7f305a023700 1 Processor -- start 2024-09-13T18:53:15.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.288+0000 7f305a023700 1 -- start start 2024-09-13T18:53:15.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f305a023700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3054160eb0 0x7f3054165590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:15.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f305a023700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f30540b33d0 0x7f30540b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:15.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f305a023700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30540b3d30 con 0x7f3054161290 2024-09-13T18:53:15.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f305a023700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3054071f30 con 0x7f30540b33d0 2024-09-13T18:53:15.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f305a023700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30541622e0 con 0x7f3054160eb0 2024-09-13T18:53:15.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f3059021700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3054160eb0 0x7f3054165590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:15.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f3059021700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3054160eb0 0x7f3054165590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:55456/0 (socket says 172.21.15.7:55456) 2024-09-13T18:53:15.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f3058820700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f30540b33d0 0x7f30540b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:15.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f3059822700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f3054161290 0x7f3054161740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:40588/0 (socket says 172.21.15.7:40588) 2024-09-13T18:53:15.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f3059021700 1 -- 172.21.15.7:0/3784916854 learned_addr learned my addr 172.21.15.7:0/3784916854 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:15.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f304bfff700 1 -- 172.21.15.7:0/3784916854 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 202086630 0 0) 0x7f30540b3d30 con 0x7f3054161290 2024-09-13T18:53:15.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f304bfff700 1 -- 172.21.15.7:0/3784916854 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30400035d0 con 0x7f3054161290 2024-09-13T18:53:15.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f3059021700 1 -- 172.21.15.7:0/3784916854 >> v2:172.21.15.171:3300/0 conn(0x7f30540b33d0 msgr2=0x7f30540b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:15.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f3059021700 1 --2- 172.21.15.7:0/3784916854 >> v2:172.21.15.171:3300/0 conn(0x7f30540b33d0 0x7f30540b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:15.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f3059021700 1 -- 172.21.15.7:0/3784916854 >> v1:172.21.15.171:6789/0 conn(0x7f3054161290 legacy=0x7f3054161740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:15.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f3059021700 1 -- 172.21.15.7:0/3784916854 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3054162000 con 0x7f3054160eb0 2024-09-13T18:53:15.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f3058820700 1 --2- 172.21.15.7:0/3784916854 >> v2:172.21.15.171:3300/0 conn(0x7f30540b33d0 0x7f30540b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:15.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f3059021700 1 --2- 172.21.15.7:0/3784916854 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3054160eb0 0x7f3054165590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3044009910 tx=0x7f3044009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c586a9e701f52a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:15.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f304bfff700 1 -- 172.21.15.7:0/3784916854 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044011830 con 0x7f3054160eb0 2024-09-13T18:53:15.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f304bfff700 1 -- 172.21.15.7:0/3784916854 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3044011990 con 0x7f3054160eb0 2024-09-13T18:53:15.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.292+0000 7f304bfff700 1 -- 172.21.15.7:0/3784916854 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044011d40 con 0x7f3054160eb0 2024-09-13T18:53:15.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.296+0000 7f305a023700 1 -- 172.21.15.7:0/3784916854 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3054160eb0 msgr2=0x7f3054165590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:15.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.296+0000 7f305a023700 1 --2- 172.21.15.7:0/3784916854 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3054160eb0 0x7f3054165590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3044009910 tx=0x7f3044009c20 comp rx=0 tx=0).stop 2024-09-13T18:53:15.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.296+0000 7f305a023700 1 -- 172.21.15.7:0/3784916854 shutdown_connections 2024-09-13T18:53:15.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.296+0000 7f305a023700 1 --2- 172.21.15.7:0/3784916854 >> v2:172.21.15.171:3300/0 conn(0x7f30540b33d0 0x7f30540b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:15.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.296+0000 7f305a023700 1 --2- 172.21.15.7:0/3784916854 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3054160eb0 0x7f3054165590 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:15.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.296+0000 7f305a023700 1 -- 172.21.15.7:0/3784916854 >> 172.21.15.7:0/3784916854 conn(0x7f30540b0f00 msgr2=0x7f30540b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:15.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.296+0000 7f305a023700 1 -- 172.21.15.7:0/3784916854 shutdown_connections 2024-09-13T18:53:15.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.296+0000 7f305a023700 1 -- 172.21.15.7:0/3784916854 wait complete. 2024-09-13T18:53:15.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.296+0000 7f305a023700 1 Processor -- start 2024-09-13T18:53:15.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.296+0000 7f305a023700 1 -- start start 2024-09-13T18:53:15.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f305a023700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30540b33d0 0x7f305415d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:15.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f305a023700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3054160eb0 0x7f30541f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:15.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f305a023700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3054161290 0x7f30541563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f305a023700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3054159590 con 0x7f30540b33d0 2024-09-13T18:53:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f305a023700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305415e1a0 con 0x7f3054161290 2024-09-13T18:53:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f305a023700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305415d900 con 0x7f3054160eb0 2024-09-13T18:53:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f3059021700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30540b33d0 0x7f305415d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f3059021700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30540b33d0 0x7f305415d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46298/0 (socket says 172.21.15.7:46298) 2024-09-13T18:53:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f3059021700 1 -- 172.21.15.7:0/1318558623 learned_addr learned my addr 172.21.15.7:0/1318558623 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f3059822700 1 --2- 172.21.15.7:0/1318558623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3054161290 0x7f30541563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f3058820700 1 --2- 172.21.15.7:0/1318558623 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3054160eb0 0x7f30541f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f3059021700 1 -- 172.21.15.7:0/1318558623 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3054160eb0 msgr2=0x7f30541f74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f3059021700 1 --2- 172.21.15.7:0/1318558623 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3054160eb0 0x7f30541f74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f3059021700 1 -- 172.21.15.7:0/1318558623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3054161290 msgr2=0x7f30541563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f3059021700 1 --2- 172.21.15.7:0/1318558623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3054161290 0x7f30541563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:15.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f3059021700 1 -- 172.21.15.7:0/1318558623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30500094b0 con 0x7f30540b33d0 2024-09-13T18:53:15.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f3059021700 1 --2- 172.21.15.7:0/1318558623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30540b33d0 0x7f305415d270 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3044000c00 tx=0x7f3044004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:15.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f3049ffb700 1 -- 172.21.15.7:0/1318558623 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044004510 con 0x7f30540b33d0 2024-09-13T18:53:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f3049ffb700 1 -- 172.21.15.7:0/1318558623 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30440036c0 con 0x7f30540b33d0 2024-09-13T18:53:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f305a023700 1 -- 172.21.15.7:0/1318558623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3044009570 con 0x7f30540b33d0 2024-09-13T18:53:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f3049ffb700 1 -- 172.21.15.7:0/1318558623 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044003a70 con 0x7f30540b33d0 2024-09-13T18:53:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f305a023700 1 -- 172.21.15.7:0/1318558623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305415dec0 con 0x7f30540b33d0 2024-09-13T18:53:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.300+0000 7f305a023700 1 -- 172.21.15.7:0/1318558623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3054165590 con 0x7f30540b33d0 2024-09-13T18:53:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.304+0000 7f3049ffb700 1 -- 172.21.15.7:0/1318558623 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3044003bd0 con 0x7f30540b33d0 2024-09-13T18:53:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.304+0000 7f3049ffb700 1 --2- 172.21.15.7:0/1318558623 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3038066730 0x7f3038068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.304+0000 7f3049ffb700 1 -- 172.21.15.7:0/1318558623 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3044021030 con 0x7f30540b33d0 2024-09-13T18:53:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.304+0000 7f3058820700 1 --2- 172.21.15.7:0/1318558623 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3038066730 0x7f3038068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:15.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.304+0000 7f3058820700 1 --2- 172.21.15.7:0/1318558623 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3038066730 0x7f3038068bf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f30540cd0d0 tx=0x7f3050009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.316+0000 7f3049ffb700 1 -- 172.21.15.7:0/1318558623 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3044052a00 con 0x7f30540b33d0 2024-09-13T18:53:15.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.500+0000 7f305a023700 1 -- 172.21.15.7:0/1318558623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f30540cce20 con 0x7f30540b33d0 2024-09-13T18:53:15.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.500+0000 7f3049ffb700 1 -- 172.21.15.7:0/1318558623 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3044056020 con 0x7f30540b33d0 2024-09-13T18:53:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:15.507 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-13T18:53:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:53:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-09-13T18:53:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-09-13T18:53:15.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:15.507 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-13T18:53:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.500+0000 7f305a023700 1 -- 172.21.15.7:0/1318558623 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3038066730 msgr2=0x7f3038068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.500+0000 7f305a023700 1 --2- 172.21.15.7:0/1318558623 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3038066730 0x7f3038068bf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f30540cd0d0 tx=0x7f3050009240 comp rx=0 tx=0).stop 2024-09-13T18:53:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.500+0000 7f305a023700 1 -- 172.21.15.7:0/1318558623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30540b33d0 msgr2=0x7f305415d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.500+0000 7f305a023700 1 --2- 172.21.15.7:0/1318558623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30540b33d0 0x7f305415d270 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3044000c00 tx=0x7f3044004770 comp rx=0 tx=0).stop 2024-09-13T18:53:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.500+0000 7f305a023700 1 -- 172.21.15.7:0/1318558623 shutdown_connections 2024-09-13T18:53:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.500+0000 7f305a023700 1 --2- 172.21.15.7:0/1318558623 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3038066730 0x7f3038068bf0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.500+0000 7f305a023700 1 --2- 172.21.15.7:0/1318558623 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3054161290 0x7f30541563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.500+0000 7f305a023700 1 --2- 172.21.15.7:0/1318558623 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3054160eb0 0x7f30541f74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.500+0000 7f305a023700 1 --2- 172.21.15.7:0/1318558623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30540b33d0 0x7f305415d270 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.500+0000 7f305a023700 1 -- 172.21.15.7:0/1318558623 >> 172.21.15.7:0/1318558623 conn(0x7f30540b0f00 msgr2=0x7f3054070350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.504+0000 7f305a023700 1 -- 172.21.15.7:0/1318558623 shutdown_connections 2024-09-13T18:53:15.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.504+0000 7f305a023700 1 -- 172.21.15.7:0/1318558623 wait complete. 2024-09-13T18:53:15.522 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T18:53:15.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T18:53:15.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.628+0000 7fcdfc87c700 1 Processor -- start 2024-09-13T18:53:15.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.628+0000 7fcdfc87c700 1 -- start start 2024-09-13T18:53:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdfc87c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcdf815a940 0x7fcdf806d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdfc87c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf806c950 con 0x7fcdf81433e0 2024-09-13T18:53:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdfc87c700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf806cc30 con 0x7fcdf806a310 2024-09-13T18:53:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdfc87c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf815add0 con 0x7fcdf815a940 2024-09-13T18:53:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdf5d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcdf81433e0 0x7fcdf815b290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47370/0 (socket says 172.21.15.7:47370) 2024-09-13T18:53:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdf5d9b700 1 -- 172.21.15.7:0/1403305640 learned_addr learned my addr 172.21.15.7:0/1403305640 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:15.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdf6d9d700 1 --2- 172.21.15.7:0/1403305640 >> v2:172.21.15.7:3300/0 conn(0x7fcdf815a940 0x7fcdf806d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:15.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdf559a700 1 -- 172.21.15.7:0/1403305640 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1146030216 0 0) 0x7fcdf806c950 con 0x7fcdf81433e0 2024-09-13T18:53:15.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdf559a700 1 -- 172.21.15.7:0/1403305640 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcde40035d0 con 0x7fcdf81433e0 2024-09-13T18:53:15.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdf559a700 1 -- 172.21.15.7:0/1403305640 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251557455 0 0) 0x7fcdf806cc30 con 0x7fcdf806a310 2024-09-13T18:53:15.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdf559a700 1 -- 172.21.15.7:0/1403305640 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdf806c950 con 0x7fcdf806a310 2024-09-13T18:53:15.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdf6d9d700 1 -- 172.21.15.7:0/1403305640 >> v1:172.21.15.171:6789/0 conn(0x7fcdf806a310 legacy=0x7fcdf806a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:15.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdf6d9d700 1 -- 172.21.15.7:0/1403305640 >> v1:172.21.15.7:6789/0 conn(0x7fcdf81433e0 legacy=0x7fcdf815b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:15.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdf6d9d700 1 -- 172.21.15.7:0/1403305640 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf806cf10 con 0x7fcdf815a940 2024-09-13T18:53:15.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdf6d9d700 1 --2- 172.21.15.7:0/1403305640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815a940 0x7fcdf806d710 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009850 tx=0x7fcde8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55ef389e4147910 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdf559a700 1 -- 172.21.15.7:0/1403305640 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8011830 con 0x7fcdf815a940 2024-09-13T18:53:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdf559a700 1 -- 172.21.15.7:0/1403305640 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde8011990 con 0x7fcdf815a940 2024-09-13T18:53:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdf559a700 1 -- 172.21.15.7:0/1403305640 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8011d40 con 0x7fcdf815a940 2024-09-13T18:53:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdfc87c700 1 -- 172.21.15.7:0/1403305640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815a940 msgr2=0x7fcdf806d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdfc87c700 1 --2- 172.21.15.7:0/1403305640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815a940 0x7fcdf806d710 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009850 tx=0x7fcde8009b60 comp rx=0 tx=0).stop 2024-09-13T18:53:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdfc87c700 1 -- 172.21.15.7:0/1403305640 shutdown_connections 2024-09-13T18:53:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdfc87c700 1 --2- 172.21.15.7:0/1403305640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815a940 0x7fcdf806d710 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdfc87c700 1 -- 172.21.15.7:0/1403305640 >> 172.21.15.7:0/1403305640 conn(0x7fcdf80af620 msgr2=0x7fcdf80afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdfc87c700 1 -- 172.21.15.7:0/1403305640 shutdown_connections 2024-09-13T18:53:15.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.632+0000 7fcdfc87c700 1 -- 172.21.15.7:0/1403305640 wait complete. 2024-09-13T18:53:15.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.636+0000 7fcdfc87c700 1 Processor -- start 2024-09-13T18:53:15.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.636+0000 7fcdfc87c700 1 -- start start 2024-09-13T18:53:15.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdfc87c700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcdf806a310 0x7fcdf8131720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdfc87c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf81433e0 0x7fcdf8131c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdfc87c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815a940 0x7fcdf815c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdfc87c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf815e300 con 0x7fcdf815a940 2024-09-13T18:53:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdfc87c700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf81321a0 con 0x7fcdf81433e0 2024-09-13T18:53:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdfc87c700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf81348e0 con 0x7fcdf806a310 2024-09-13T18:53:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdf6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815a940 0x7fcdf815c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdf5d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf81433e0 0x7fcdf8131c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdf5d9b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf81433e0 0x7fcdf8131c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:50498/0 (socket says 172.21.15.7:50498) 2024-09-13T18:53:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdf5d9b700 1 -- 172.21.15.7:0/1587198001 learned_addr learned my addr 172.21.15.7:0/1587198001 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdf6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815a940 0x7fcdf815c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46324/0 (socket says 172.21.15.7:46324) 2024-09-13T18:53:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdf659c700 1 --2- 172.21.15.7:0/1587198001 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcdf806a310 0x7fcdf8131720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdf6d9d700 1 -- 172.21.15.7:0/1587198001 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcdf806a310 msgr2=0x7fcdf8131720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:15.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdf6d9d700 1 --2- 172.21.15.7:0/1587198001 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcdf806a310 0x7fcdf8131720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdf6d9d700 1 -- 172.21.15.7:0/1587198001 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf81433e0 msgr2=0x7fcdf8131c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdf6d9d700 1 --2- 172.21.15.7:0/1587198001 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf81433e0 0x7fcdf8131c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdf6d9d700 1 -- 172.21.15.7:0/1587198001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde80094b0 con 0x7fcdf815a940 2024-09-13T18:53:15.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdf6d9d700 1 --2- 172.21.15.7:0/1587198001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815a940 0x7fcdf815c0b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcde8016000 tx=0x7fcde8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdeeffd700 1 -- 172.21.15.7:0/1587198001 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde801a8f0 con 0x7fcdf815a940 2024-09-13T18:53:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdeeffd700 1 -- 172.21.15.7:0/1587198001 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde801e070 con 0x7fcdf815a940 2024-09-13T18:53:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdfc87c700 1 -- 172.21.15.7:0/1587198001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf8139890 con 0x7fcdf815a940 2024-09-13T18:53:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdeeffd700 1 -- 172.21.15.7:0/1587198001 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde801acb0 con 0x7fcdf815a940 2024-09-13T18:53:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdfc87c700 1 -- 172.21.15.7:0/1587198001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf815dd40 con 0x7fcdf815a940 2024-09-13T18:53:15.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.640+0000 7fcdfc87c700 1 -- 172.21.15.7:0/1587198001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf806cc30 con 0x7fcdf815a940 2024-09-13T18:53:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.644+0000 7fcdeeffd700 1 -- 172.21.15.7:0/1587198001 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcde802d450 con 0x7fcdf815a940 2024-09-13T18:53:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.644+0000 7fcdeeffd700 1 --2- 172.21.15.7:0/1587198001 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcdd8066730 0x7fcdd8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:15.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.644+0000 7fcdeeffd700 1 -- 172.21.15.7:0/1587198001 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcde8084c10 con 0x7fcdf815a940 2024-09-13T18:53:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.644+0000 7fcdf659c700 1 --2- 172.21.15.7:0/1587198001 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcdd8066730 0x7fcdd8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:15.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.644+0000 7fcdf659c700 1 --2- 172.21.15.7:0/1587198001 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcdd8066730 0x7fcdd8068bf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcde00011e0 tx=0x7fcde0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:15.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.648+0000 7fcdeeffd700 1 -- 172.21.15.7:0/1587198001 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcde80522c0 con 0x7fcdf815a940 2024-09-13T18:53:15.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:15.816+0000 7fcdfc87c700 1 -- 172.21.15.7:0/1587198001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcdf80b4970 con 0x7fcdf815a940 2024-09-13T18:53:16.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.504+0000 7fcdeeffd700 1 -- 172.21.15.7:0/1587198001 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcde80558e0 con 0x7fcdf815a940 2024-09-13T18:53:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.512+0000 7fcdfc87c700 1 -- 172.21.15.7:0/1587198001 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcdd8066730 msgr2=0x7fcdd8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.512+0000 7fcdfc87c700 1 --2- 172.21.15.7:0/1587198001 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcdd8066730 0x7fcdd8068bf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcde00011e0 tx=0x7fcde0009040 comp rx=0 tx=0).stop 2024-09-13T18:53:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.512+0000 7fcdfc87c700 1 -- 172.21.15.7:0/1587198001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815a940 msgr2=0x7fcdf815c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.512+0000 7fcdfc87c700 1 --2- 172.21.15.7:0/1587198001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815a940 0x7fcdf815c0b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcde8016000 tx=0x7fcde8004770 comp rx=0 tx=0).stop 2024-09-13T18:53:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.512+0000 7fcdfc87c700 1 -- 172.21.15.7:0/1587198001 shutdown_connections 2024-09-13T18:53:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.512+0000 7fcdfc87c700 1 --2- 172.21.15.7:0/1587198001 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcdd8066730 0x7fcdd8068bf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.512+0000 7fcdfc87c700 1 --2- 172.21.15.7:0/1587198001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815a940 0x7fcdf815c0b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.512+0000 7fcdfc87c700 1 --2- 172.21.15.7:0/1587198001 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf81433e0 0x7fcdf8131c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.512+0000 7fcdfc87c700 1 --2- 172.21.15.7:0/1587198001 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcdf806a310 0x7fcdf8131720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.512+0000 7fcdfc87c700 1 -- 172.21.15.7:0/1587198001 >> 172.21.15.7:0/1587198001 conn(0x7fcdf80af620 msgr2=0x7fcdf81421b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:16.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.512+0000 7fcdfc87c700 1 -- 172.21.15.7:0/1587198001 shutdown_connections 2024-09-13T18:53:16.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.512+0000 7fcdfc87c700 1 -- 172.21.15.7:0/1587198001 wait complete. 2024-09-13T18:53:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-09-13T18:53:16.534 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T18:53:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.648+0000 7f640b30f700 1 Processor -- start 2024-09-13T18:53:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.648+0000 7f640b30f700 1 -- start start 2024-09-13T18:53:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640b30f700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f640415cda0 0x7f640406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640b30f700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64040b8430 con 0x7f64040ceb70 2024-09-13T18:53:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640b30f700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64040cf350 con 0x7f640406a520 2024-09-13T18:53:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640b30f700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f640406be40 con 0x7f640415cda0 2024-09-13T18:53:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f6409b0c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f64040ceb70 0x7f64040b5de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47386/0 (socket says 172.21.15.7:47386) 2024-09-13T18:53:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f6409b0c700 1 -- 172.21.15.7:0/2476579340 learned_addr learned my addr 172.21.15.7:0/2476579340 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640a30d700 1 --2- 172.21.15.7:0/2476579340 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f640415cda0 0x7f640406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640930b700 1 -- 172.21.15.7:0/2476579340 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446831633 0 0) 0x7f64040b8430 con 0x7f64040ceb70 2024-09-13T18:53:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640930b700 1 -- 172.21.15.7:0/2476579340 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63f00035d0 con 0x7f64040ceb70 2024-09-13T18:53:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640a30d700 1 -- 172.21.15.7:0/2476579340 >> v1:172.21.15.171:6789/0 conn(0x7f640406a520 legacy=0x7f64040d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640a30d700 1 -- 172.21.15.7:0/2476579340 >> v1:172.21.15.7:6789/0 conn(0x7f64040ceb70 legacy=0x7f64040b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640930b700 1 -- 172.21.15.7:0/2476579340 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1712397347 0 0) 0x7f63f00035d0 con 0x7f64040ceb70 2024-09-13T18:53:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640a30d700 1 -- 172.21.15.7:0/2476579340 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64040d0880 con 0x7f640415cda0 2024-09-13T18:53:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640a30d700 1 --2- 172.21.15.7:0/2476579340 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f640415cda0 0x7f640406b900 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009880 tx=0x7f63f4009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3f1febb7950ee2e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640930b700 1 -- 172.21.15.7:0/2476579340 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f4011830 con 0x7f640415cda0 2024-09-13T18:53:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640930b700 1 -- 172.21.15.7:0/2476579340 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f4011990 con 0x7f640415cda0 2024-09-13T18:53:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640930b700 1 -- 172.21.15.7:0/2476579340 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f4011d00 con 0x7f640415cda0 2024-09-13T18:53:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640b30f700 1 -- 172.21.15.7:0/2476579340 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f640415cda0 msgr2=0x7f640406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640b30f700 1 --2- 172.21.15.7:0/2476579340 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f640415cda0 0x7f640406b900 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009880 tx=0x7f63f4009b90 comp rx=0 tx=0).stop 2024-09-13T18:53:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640b30f700 1 -- 172.21.15.7:0/2476579340 shutdown_connections 2024-09-13T18:53:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640b30f700 1 --2- 172.21.15.7:0/2476579340 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f640415cda0 0x7f640406b900 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.652+0000 7f640b30f700 1 -- 172.21.15.7:0/2476579340 >> 172.21.15.7:0/2476579340 conn(0x7f64040d2cb0 msgr2=0x7f64040d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.656+0000 7f640b30f700 1 -- 172.21.15.7:0/2476579340 shutdown_connections 2024-09-13T18:53:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.656+0000 7f640b30f700 1 -- 172.21.15.7:0/2476579340 wait complete. 2024-09-13T18:53:16.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.656+0000 7f640b30f700 1 Processor -- start 2024-09-13T18:53:16.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.656+0000 7f640b30f700 1 -- start start 2024-09-13T18:53:16.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.656+0000 7f640b30f700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f640406a520 0x7f640415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.656+0000 7f640b30f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64040ceb70 0x7f640415f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.656+0000 7f640b30f700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f640415cda0 0x7f640415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.656+0000 7f640b30f700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64040c9d80 con 0x7f64040ceb70 2024-09-13T18:53:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.656+0000 7f640b30f700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64041f2b00 con 0x7f640406a520 2024-09-13T18:53:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.656+0000 7f640b30f700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64040c93b0 con 0x7f640415cda0 2024-09-13T18:53:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.660+0000 7f6409b0c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64040ceb70 0x7f640415f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.660+0000 7f6409b0c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64040ceb70 0x7f640415f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46328/0 (socket says 172.21.15.7:46328) 2024-09-13T18:53:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.660+0000 7f6409b0c700 1 -- 172.21.15.7:0/3406359010 learned_addr learned my addr 172.21.15.7:0/3406359010 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.660+0000 7f640a30d700 1 --2- 172.21.15.7:0/3406359010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f640406a520 0x7f640415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.660+0000 7f640ab0e700 1 --2- 172.21.15.7:0/3406359010 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f640415cda0 0x7f640415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.660+0000 7f6409b0c700 1 -- 172.21.15.7:0/3406359010 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f640415cda0 msgr2=0x7f640415fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.660+0000 7f6409b0c700 1 --2- 172.21.15.7:0/3406359010 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f640415cda0 0x7f640415fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.660+0000 7f6409b0c700 1 -- 172.21.15.7:0/3406359010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f640406a520 msgr2=0x7f640415a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.660+0000 7f6409b0c700 1 --2- 172.21.15.7:0/3406359010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f640406a520 0x7f640415a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.660+0000 7f6409b0c700 1 -- 172.21.15.7:0/3406359010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f40094b0 con 0x7f64040ceb70 2024-09-13T18:53:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.660+0000 7f6409b0c700 1 --2- 172.21.15.7:0/3406359010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64040ceb70 0x7f640415f750 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f640000c780 tx=0x7f640000ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.660+0000 7f63faffd700 1 -- 172.21.15.7:0/3406359010 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6400005d90 con 0x7f64040ceb70 2024-09-13T18:53:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.660+0000 7f63faffd700 1 -- 172.21.15.7:0/3406359010 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64000070d0 con 0x7f64040ceb70 2024-09-13T18:53:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.660+0000 7f640b30f700 1 -- 172.21.15.7:0/3406359010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64040c9690 con 0x7f64040ceb70 2024-09-13T18:53:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.660+0000 7f63faffd700 1 -- 172.21.15.7:0/3406359010 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6400007480 con 0x7f64040ceb70 2024-09-13T18:53:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.660+0000 7f640b30f700 1 -- 172.21.15.7:0/3406359010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64040ca270 con 0x7f64040ceb70 2024-09-13T18:53:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.660+0000 7f640b30f700 1 -- 172.21.15.7:0/3406359010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64040ca490 con 0x7f64040ceb70 2024-09-13T18:53:16.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.664+0000 7f63faffd700 1 -- 172.21.15.7:0/3406359010 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f64000075e0 con 0x7f64040ceb70 2024-09-13T18:53:16.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.664+0000 7f63faffd700 1 --2- 172.21.15.7:0/3406359010 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f63e8066760 0x7f63e8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.664+0000 7f63faffd700 1 -- 172.21.15.7:0/3406359010 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6400083310 con 0x7f64040ceb70 2024-09-13T18:53:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.664+0000 7f640a30d700 1 --2- 172.21.15.7:0/3406359010 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f63e8066760 0x7f63e8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.664+0000 7f640a30d700 1 --2- 172.21.15.7:0/3406359010 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f63e8066760 0x7f63e8068c20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f63f40097d0 tx=0x7f63f4010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.672+0000 7f63faffd700 1 -- 172.21.15.7:0/3406359010 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6400050850 con 0x7f64040ceb70 2024-09-13T18:53:16.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:16.840+0000 7f640b30f700 1 -- 172.21.15.7:0/3406359010 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f640406b900 con 0x7f64040ceb70 2024-09-13T18:53:17.435 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-13T18:53:17.435 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-13T18:53:17.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.516+0000 7f63faffd700 1 -- 172.21.15.7:0/3406359010 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6400053e70 con 0x7f64040ceb70 2024-09-13T18:53:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.528+0000 7f640b30f700 1 -- 172.21.15.7:0/3406359010 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f63e8066760 msgr2=0x7f63e8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.528+0000 7f640b30f700 1 --2- 172.21.15.7:0/3406359010 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f63e8066760 0x7f63e8068c20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f63f40097d0 tx=0x7f63f4010000 comp rx=0 tx=0).stop 2024-09-13T18:53:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.528+0000 7f640b30f700 1 -- 172.21.15.7:0/3406359010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64040ceb70 msgr2=0x7f640415f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:17.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.528+0000 7f640b30f700 1 --2- 172.21.15.7:0/3406359010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64040ceb70 0x7f640415f750 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f640000c780 tx=0x7f640000ca90 comp rx=0 tx=0).stop 2024-09-13T18:53:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.532+0000 7f640b30f700 1 -- 172.21.15.7:0/3406359010 shutdown_connections 2024-09-13T18:53:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.532+0000 7f640b30f700 1 --2- 172.21.15.7:0/3406359010 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f63e8066760 0x7f63e8068c20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.532+0000 7f640b30f700 1 --2- 172.21.15.7:0/3406359010 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f640415cda0 0x7f640415fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.532+0000 7f640b30f700 1 --2- 172.21.15.7:0/3406359010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64040ceb70 0x7f640415f750 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.532+0000 7f640b30f700 1 --2- 172.21.15.7:0/3406359010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f640406a520 0x7f640415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.532+0000 7f640b30f700 1 -- 172.21.15.7:0/3406359010 >> 172.21.15.7:0/3406359010 conn(0x7f64040d2cb0 msgr2=0x7f64040ce160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.532+0000 7f640b30f700 1 -- 172.21.15.7:0/3406359010 shutdown_connections 2024-09-13T18:53:17.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.532+0000 7f640b30f700 1 -- 172.21.15.7:0/3406359010 wait complete. 2024-09-13T18:53:17.539 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-09-13T18:53:17.551 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T18:53:17.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f5560956700 1 Processor -- start 2024-09-13T18:53:17.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f5560956700 1 -- start start 2024-09-13T18:53:17.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f5560956700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f555c06d6f0 0x7f555c0ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:17.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f5560956700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f555c15d200 0x7f555c06baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:17.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f5560956700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555c06dad0 con 0x7f555c0adbd0 2024-09-13T18:53:17.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f5560956700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f555c0ae3c0 con 0x7f555c06d6f0 2024-09-13T18:53:17.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f5560956700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f555c0705b0 con 0x7f555c15d200 2024-09-13T18:53:17.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f555b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f555c06d6f0 0x7f555c0ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:17.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f555b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f555c06d6f0 0x7f555c0ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46332/0 (socket says 172.21.15.7:46332) 2024-09-13T18:53:17.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f555b7fe700 1 -- 172.21.15.7:0/4174818318 learned_addr learned my addr 172.21.15.7:0/4174818318 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:17.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f555bfff700 1 --2- 172.21.15.7:0/4174818318 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f555c15d200 0x7f555c06baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:17.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f555b7fe700 1 -- 172.21.15.7:0/4174818318 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f555c15d200 msgr2=0x7f555c06baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:17.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f555b7fe700 1 --2- 172.21.15.7:0/4174818318 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f555c15d200 0x7f555c06baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f555b7fe700 1 -- 172.21.15.7:0/4174818318 >> v1:172.21.15.171:6789/0 conn(0x7f555c0adbd0 legacy=0x7f555c15caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:17.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f555b7fe700 1 -- 172.21.15.7:0/4174818318 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c070890 con 0x7f555c06d6f0 2024-09-13T18:53:17.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f555bfff700 1 --2- 172.21.15.7:0/4174818318 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f555c15d200 0x7f555c06baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:53:17.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f555b7fe700 1 --2- 172.21.15.7:0/4174818318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f555c06d6f0 0x7f555c0ce270 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5554009e50 tx=0x7f555400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f0fb04f01dd35c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:17.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f555a7fc700 1 -- 172.21.15.7:0/4174818318 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555400e070 con 0x7f555c06d6f0 2024-09-13T18:53:17.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f555a7fc700 1 -- 172.21.15.7:0/4174818318 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5554003a40 con 0x7f555c06d6f0 2024-09-13T18:53:17.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.656+0000 7f555a7fc700 1 -- 172.21.15.7:0/4174818318 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5554003df0 con 0x7f555c06d6f0 2024-09-13T18:53:17.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.660+0000 7f5560956700 1 -- 172.21.15.7:0/4174818318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f555c06d6f0 msgr2=0x7f555c0ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:17.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.660+0000 7f5560956700 1 --2- 172.21.15.7:0/4174818318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f555c06d6f0 0x7f555c0ce270 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5554009e50 tx=0x7f555400b330 comp rx=0 tx=0).stop 2024-09-13T18:53:17.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.660+0000 7f5560956700 1 -- 172.21.15.7:0/4174818318 shutdown_connections 2024-09-13T18:53:17.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.660+0000 7f5560956700 1 --2- 172.21.15.7:0/4174818318 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f555c15d200 0x7f555c06baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.660+0000 7f5560956700 1 --2- 172.21.15.7:0/4174818318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f555c06d6f0 0x7f555c0ce270 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.660+0000 7f5560956700 1 -- 172.21.15.7:0/4174818318 >> 172.21.15.7:0/4174818318 conn(0x7f555c0b0e10 msgr2=0x7f555c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:17.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.660+0000 7f5560956700 1 -- 172.21.15.7:0/4174818318 shutdown_connections 2024-09-13T18:53:17.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.660+0000 7f5560956700 1 -- 172.21.15.7:0/4174818318 wait complete. 2024-09-13T18:53:17.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.660+0000 7f5560956700 1 Processor -- start 2024-09-13T18:53:17.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.660+0000 7f5560956700 1 -- start start 2024-09-13T18:53:17.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.660+0000 7f5560956700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f555c06d6f0 0x7f555c14a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.660+0000 7f5560956700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f555c0adbd0 0x7f555c153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.660+0000 7f5560956700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f555c15d200 0x7f555c0ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.660+0000 7f5560956700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c154510 con 0x7f555c0adbd0 2024-09-13T18:53:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.660+0000 7f5560956700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555c1547f0 con 0x7f555c15d200 2024-09-13T18:53:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.660+0000 7f5560956700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f555c1662e0 con 0x7f555c06d6f0 2024-09-13T18:53:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.664+0000 7f555affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f555c0adbd0 0x7f555c153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.664+0000 7f555b7fe700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f555c06d6f0 0x7f555c14a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.664+0000 7f555b7fe700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f555c06d6f0 0x7f555c14a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:55502/0 (socket says 172.21.15.7:55502) 2024-09-13T18:53:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.664+0000 7f555b7fe700 1 -- 172.21.15.7:0/404653640 learned_addr learned my addr 172.21.15.7:0/404653640 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.664+0000 7f555bfff700 1 --2- 172.21.15.7:0/404653640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f555c15d200 0x7f555c0ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.664+0000 7f555b7fe700 1 -- 172.21.15.7:0/404653640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f555c15d200 msgr2=0x7f555c0ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:17.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.664+0000 7f555b7fe700 1 --2- 172.21.15.7:0/404653640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f555c15d200 0x7f555c0ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.664+0000 7f555b7fe700 1 -- 172.21.15.7:0/404653640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f555c0adbd0 msgr2=0x7f555c153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.664+0000 7f555b7fe700 1 --2- 172.21.15.7:0/404653640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f555c0adbd0 0x7f555c153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.664+0000 7f555b7fe700 1 -- 172.21.15.7:0/404653640 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5554009b30 con 0x7f555c06d6f0 2024-09-13T18:53:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.664+0000 7f555affd700 1 --2- 172.21.15.7:0/404653640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f555c0adbd0 0x7f555c153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:17.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.664+0000 7f555b7fe700 1 --2- 172.21.15.7:0/404653640 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f555c06d6f0 0x7f555c14a430 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f555c0b3df0 tx=0x7f5554004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.664+0000 7f5543fff700 1 -- 172.21.15.7:0/404653640 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5554004440 con 0x7f555c06d6f0 2024-09-13T18:53:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.664+0000 7f5543fff700 1 -- 172.21.15.7:0/404653640 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55540045a0 con 0x7f555c06d6f0 2024-09-13T18:53:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.664+0000 7f5560956700 1 -- 172.21.15.7:0/404653640 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555c1665c0 con 0x7f555c06d6f0 2024-09-13T18:53:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.664+0000 7f5543fff700 1 -- 172.21.15.7:0/404653640 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5554004950 con 0x7f555c06d6f0 2024-09-13T18:53:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.664+0000 7f5560956700 1 -- 172.21.15.7:0/404653640 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555c14aa50 con 0x7f555c06d6f0 2024-09-13T18:53:17.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.664+0000 7f5560956700 1 -- 172.21.15.7:0/404653640 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555c0ae000 con 0x7f555c06d6f0 2024-09-13T18:53:17.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.668+0000 7f5543fff700 1 -- 172.21.15.7:0/404653640 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5554025450 con 0x7f555c06d6f0 2024-09-13T18:53:17.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.668+0000 7f5543fff700 1 --2- 172.21.15.7:0/404653640 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5548066730 0x7f5548068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:17.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.668+0000 7f5543fff700 1 -- 172.21.15.7:0/404653640 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f55540855c0 con 0x7f555c06d6f0 2024-09-13T18:53:17.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.668+0000 7f555affd700 1 --2- 172.21.15.7:0/404653640 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5548066730 0x7f5548068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:17.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.668+0000 7f555affd700 1 --2- 172.21.15.7:0/404653640 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5548066730 0x7f5548068bf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f555c14de80 tx=0x7f5550007ee0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:17.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.676+0000 7f5543fff700 1 -- 172.21.15.7:0/404653640 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f55540566b0 con 0x7f555c06d6f0 2024-09-13T18:53:17.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.888+0000 7f5560956700 1 -- 172.21.15.7:0/404653640 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f555c0ae000 con 0x7f555c06d6f0 2024-09-13T18:53:17.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.908+0000 7f5543fff700 1 -- 172.21.15.7:0/404653640 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f555c0ae000 con 0x7f555c06d6f0 2024-09-13T18:53:17.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.912+0000 7f5560956700 1 -- 172.21.15.7:0/404653640 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5548066730 msgr2=0x7f5548068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:17.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.912+0000 7f5560956700 1 --2- 172.21.15.7:0/404653640 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5548066730 0x7f5548068bf0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f555c14de80 tx=0x7f5550007ee0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.912+0000 7f5560956700 1 -- 172.21.15.7:0/404653640 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f555c06d6f0 msgr2=0x7f555c14a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:17.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.912+0000 7f5560956700 1 --2- 172.21.15.7:0/404653640 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f555c06d6f0 0x7f555c14a430 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f555c0b3df0 tx=0x7f5554004af0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.912+0000 7f5560956700 1 -- 172.21.15.7:0/404653640 shutdown_connections 2024-09-13T18:53:17.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.912+0000 7f5560956700 1 --2- 172.21.15.7:0/404653640 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5548066730 0x7f5548068bf0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.912+0000 7f5560956700 1 --2- 172.21.15.7:0/404653640 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f555c15d200 0x7f555c0ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.912+0000 7f5560956700 1 --2- 172.21.15.7:0/404653640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f555c0adbd0 0x7f555c153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.912+0000 7f5560956700 1 --2- 172.21.15.7:0/404653640 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f555c06d6f0 0x7f555c14a430 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:17.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.912+0000 7f5560956700 1 -- 172.21.15.7:0/404653640 >> 172.21.15.7:0/404653640 conn(0x7f555c0b0e10 msgr2=0x7f555c0afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:17.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.912+0000 7f5560956700 1 -- 172.21.15.7:0/404653640 shutdown_connections 2024-09-13T18:53:17.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:17.912+0000 7f5560956700 1 -- 172.21.15.7:0/404653640 wait complete. 2024-09-13T18:53:17.921 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 4 and data pool 5 2024-09-13T18:53:17.942 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:53:18.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.032+0000 7fc2a6777700 1 Processor -- start 2024-09-13T18:53:18.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.032+0000 7fc2a6777700 1 -- start start 2024-09-13T18:53:18.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.032+0000 7fc2a6777700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2a0160eb0 0x7fc2a0165590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:18.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.032+0000 7fc2a6777700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc2a0161290 0x7fc2a0161740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:18.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.032+0000 7fc2a6777700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a0071f30 con 0x7fc2a00b33d0 2024-09-13T18:53:18.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.032+0000 7fc2a6777700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a01622e0 con 0x7fc2a0161290 2024-09-13T18:53:18.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.032+0000 7fc2a6777700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a0162000 con 0x7fc2a0160eb0 2024-09-13T18:53:18.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.032+0000 7fc2a5f76700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc2a0161290 0x7fc2a0161740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:18.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.032+0000 7fc2a5f76700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc2a0161290 0x7fc2a0161740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46346/0 (socket says 172.21.15.7:46346) 2024-09-13T18:53:18.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.032+0000 7fc2a5f76700 1 -- 172.21.15.7:0/483481195 learned_addr learned my addr 172.21.15.7:0/483481195 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:18.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.032+0000 7fc2a5775700 1 --2- 172.21.15.7:0/483481195 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2a0160eb0 0x7fc2a0165590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:18.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.032+0000 7fc2a5f76700 1 -- 172.21.15.7:0/483481195 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2a0160eb0 msgr2=0x7fc2a0165590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:18.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.032+0000 7fc2a5f76700 1 --2- 172.21.15.7:0/483481195 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2a0160eb0 0x7fc2a0165590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.032+0000 7fc2a5f76700 1 -- 172.21.15.7:0/483481195 >> v1:172.21.15.171:6789/0 conn(0x7fc2a00b33d0 legacy=0x7fc2a00b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:18.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.032+0000 7fc2a5f76700 1 -- 172.21.15.7:0/483481195 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a0164ab0 con 0x7fc2a0161290 2024-09-13T18:53:18.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.036+0000 7fc2a5f76700 1 --2- 172.21.15.7:0/483481195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a0161290 0x7fc2a0161740 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc298009910 tx=0x7fc298009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7889f02f6dc9801e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:18.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.036+0000 7fc29ffff700 1 -- 172.21.15.7:0/483481195 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298011830 con 0x7fc2a0161290 2024-09-13T18:53:18.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.036+0000 7fc29ffff700 1 -- 172.21.15.7:0/483481195 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc298011990 con 0x7fc2a0161290 2024-09-13T18:53:18.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.036+0000 7fc29ffff700 1 -- 172.21.15.7:0/483481195 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298011d40 con 0x7fc2a0161290 2024-09-13T18:53:18.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.036+0000 7fc2a6777700 1 -- 172.21.15.7:0/483481195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a0161290 msgr2=0x7fc2a0161740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:18.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.036+0000 7fc2a6777700 1 --2- 172.21.15.7:0/483481195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a0161290 0x7fc2a0161740 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc298009910 tx=0x7fc298009c20 comp rx=0 tx=0).stop 2024-09-13T18:53:18.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.036+0000 7fc2a6777700 1 -- 172.21.15.7:0/483481195 shutdown_connections 2024-09-13T18:53:18.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.036+0000 7fc2a6777700 1 --2- 172.21.15.7:0/483481195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a0161290 0x7fc2a0161740 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.036+0000 7fc2a6777700 1 --2- 172.21.15.7:0/483481195 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2a0160eb0 0x7fc2a0165590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.036+0000 7fc2a6777700 1 -- 172.21.15.7:0/483481195 >> 172.21.15.7:0/483481195 conn(0x7fc2a00b0f00 msgr2=0x7fc2a00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:18.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.036+0000 7fc2a6777700 1 -- 172.21.15.7:0/483481195 shutdown_connections 2024-09-13T18:53:18.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.036+0000 7fc2a6777700 1 -- 172.21.15.7:0/483481195 wait complete. 2024-09-13T18:53:18.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.036+0000 7fc2a6777700 1 Processor -- start 2024-09-13T18:53:18.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.036+0000 7fc2a6777700 1 -- start start 2024-09-13T18:53:18.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a6777700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2a00b33d0 0x7fc2a014f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:18.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a6777700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2a0160eb0 0x7fc2a015eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:18.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a6777700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a0161290 0x7fc2a015f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:18.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a6777700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a00aaf20 con 0x7fc2a0161290 2024-09-13T18:53:18.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a6777700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a015b0d0 con 0x7fc2a0160eb0 2024-09-13T18:53:18.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a6777700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a015fa20 con 0x7fc2a00b33d0 2024-09-13T18:53:18.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a5f76700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a0161290 0x7fc2a015f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:18.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a5f76700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a0161290 0x7fc2a015f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46354/0 (socket says 172.21.15.7:46354) 2024-09-13T18:53:18.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a5f76700 1 -- 172.21.15.7:0/2792470698 learned_addr learned my addr 172.21.15.7:0/2792470698 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:18.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a5775700 1 --2- 172.21.15.7:0/2792470698 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2a00b33d0 0x7fc2a014f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:18.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a4f74700 1 --2- 172.21.15.7:0/2792470698 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2a0160eb0 0x7fc2a015eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:18.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a5f76700 1 -- 172.21.15.7:0/2792470698 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2a00b33d0 msgr2=0x7fc2a014f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:18.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a5f76700 1 --2- 172.21.15.7:0/2792470698 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2a00b33d0 0x7fc2a014f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a5f76700 1 -- 172.21.15.7:0/2792470698 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2a0160eb0 msgr2=0x7fc2a015eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:18.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a5f76700 1 --2- 172.21.15.7:0/2792470698 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2a0160eb0 0x7fc2a015eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a5f76700 1 -- 172.21.15.7:0/2792470698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc298009570 con 0x7fc2a0161290 2024-09-13T18:53:18.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a5f76700 1 --2- 172.21.15.7:0/2792470698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a0161290 0x7fc2a015f3f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0070ba0 tx=0x7fc298004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:18.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc29dffb700 1 -- 172.21.15.7:0/2792470698 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298004590 con 0x7fc2a0161290 2024-09-13T18:53:18.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc29dffb700 1 -- 172.21.15.7:0/2792470698 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29801e070 con 0x7fc2a0161290 2024-09-13T18:53:18.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a6777700 1 -- 172.21.15.7:0/2792470698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a015fd00 con 0x7fc2a0161290 2024-09-13T18:53:18.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc29dffb700 1 -- 172.21.15.7:0/2792470698 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298003900 con 0x7fc2a0161290 2024-09-13T18:53:18.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.040+0000 7fc2a6777700 1 -- 172.21.15.7:0/2792470698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a015ad60 con 0x7fc2a0161290 2024-09-13T18:53:18.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.044+0000 7fc2a6777700 1 -- 172.21.15.7:0/2792470698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a0165590 con 0x7fc2a0161290 2024-09-13T18:53:18.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.044+0000 7fc29dffb700 1 -- 172.21.15.7:0/2792470698 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc298003a60 con 0x7fc2a0161290 2024-09-13T18:53:18.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.044+0000 7fc29dffb700 1 --2- 172.21.15.7:0/2792470698 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc28c066730 0x7fc28c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:18.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.044+0000 7fc29dffb700 1 -- 172.21.15.7:0/2792470698 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2980850a0 con 0x7fc2a0161290 2024-09-13T18:53:18.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.044+0000 7fc2a5775700 1 --2- 172.21.15.7:0/2792470698 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc28c066730 0x7fc28c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:18.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.048+0000 7fc2a5775700 1 --2- 172.21.15.7:0/2792470698 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc28c066730 0x7fc28c068bf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc2a01f9f40 tx=0x7fc29000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:18.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.056+0000 7fc29dffb700 1 -- 172.21.15.7:0/2792470698 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc298052560 con 0x7fc2a0161290 2024-09-13T18:53:18.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.228+0000 7fc2a6777700 1 -- 172.21.15.7:0/2792470698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2a015b210 con 0x7fc2a0161290 2024-09-13T18:53:18.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.232+0000 7fc29dffb700 1 -- 172.21.15.7:0/2792470698 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16534 (secure 0 0 0) 0x7fc298055b80 con 0x7fc2a0161290 2024-09-13T18:53:18.239 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:18.239 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:53:17.896570+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:53:15.862087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T18:53:16.885280+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:53:18.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.232+0000 7fc2a6777700 1 -- 172.21.15.7:0/2792470698 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc28c066730 msgr2=0x7fc28c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:18.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.232+0000 7fc2a6777700 1 --2- 172.21.15.7:0/2792470698 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc28c066730 0x7fc28c068bf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc2a01f9f40 tx=0x7fc29000b040 comp rx=0 tx=0).stop 2024-09-13T18:53:18.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.232+0000 7fc2a6777700 1 -- 172.21.15.7:0/2792470698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a0161290 msgr2=0x7fc2a015f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:18.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.232+0000 7fc2a6777700 1 --2- 172.21.15.7:0/2792470698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a0161290 0x7fc2a015f3f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0070ba0 tx=0x7fc298004770 comp rx=0 tx=0).stop 2024-09-13T18:53:18.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.232+0000 7fc2a6777700 1 -- 172.21.15.7:0/2792470698 shutdown_connections 2024-09-13T18:53:18.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.232+0000 7fc2a6777700 1 --2- 172.21.15.7:0/2792470698 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc28c066730 0x7fc28c068bf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.232+0000 7fc2a6777700 1 --2- 172.21.15.7:0/2792470698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2a0161290 0x7fc2a015f3f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.232+0000 7fc2a6777700 1 --2- 172.21.15.7:0/2792470698 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc2a0160eb0 0x7fc2a015eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.232+0000 7fc2a6777700 1 --2- 172.21.15.7:0/2792470698 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc2a00b33d0 0x7fc2a014f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.232+0000 7fc2a6777700 1 -- 172.21.15.7:0/2792470698 >> 172.21.15.7:0/2792470698 conn(0x7fc2a00b0f00 msgr2=0x7fc2a015da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:18.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.232+0000 7fc2a6777700 1 -- 172.21.15.7:0/2792470698 shutdown_connections 2024-09-13T18:53:18.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.232+0000 7fc2a6777700 1 -- 172.21.15.7:0/2792470698 wait complete. 2024-09-13T18:53:18.254 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:53:18.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.356+0000 7f8ae43e1700 1 Processor -- start 2024-09-13T18:53:18.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.356+0000 7f8ae43e1700 1 -- start start 2024-09-13T18:53:18.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.356+0000 7f8ae43e1700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8adc0cf4a0 0x7f8adc0b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:18.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.356+0000 7f8ae43e1700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc0ce760 0x7f8adc0cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:18.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.356+0000 7f8ae43e1700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8adc0b2a80 con 0x7f8adc0cc450 2024-09-13T18:53:18.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.356+0000 7f8ae43e1700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8adc0cce00 con 0x7f8adc0cf4a0 2024-09-13T18:53:18.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.356+0000 7f8ae43e1700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc15af50 con 0x7f8adc0ce760 2024-09-13T18:53:18.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.356+0000 7f8ae217d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8adc0cf4a0 0x7f8adc0b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:18.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.356+0000 7f8ae217d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8adc0cf4a0 0x7f8adc0b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46364/0 (socket says 172.21.15.7:46364) 2024-09-13T18:53:18.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.356+0000 7f8ae217d700 1 -- 172.21.15.7:0/859967701 learned_addr learned my addr 172.21.15.7:0/859967701 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:18.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.356+0000 7f8ae297e700 1 --2- 172.21.15.7:0/859967701 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc0ce760 0x7f8adc0cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:18.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.356+0000 7f8ae217d700 1 -- 172.21.15.7:0/859967701 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc0ce760 msgr2=0x7f8adc0cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:18.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.356+0000 7f8ae117b700 1 -- 172.21.15.7:0/859967701 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525510348 0 0) 0x7f8adc0b2a80 con 0x7f8adc0cc450 2024-09-13T18:53:18.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.356+0000 7f8ae217d700 1 --2- 172.21.15.7:0/859967701 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc0ce760 0x7f8adc0cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.356+0000 7f8ae217d700 1 -- 172.21.15.7:0/859967701 >> v1:172.21.15.171:6789/0 conn(0x7f8adc0cc450 legacy=0x7f8adc0ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:18.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.356+0000 7f8ae217d700 1 -- 172.21.15.7:0/859967701 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc0cf880 con 0x7f8adc0cf4a0 2024-09-13T18:53:18.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.356+0000 7f8ae297e700 1 --2- 172.21.15.7:0/859967701 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc0ce760 0x7f8adc0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:18.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.356+0000 7f8ae217d700 1 --2- 172.21.15.7:0/859967701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0cf4a0 0x7f8adc0b37b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009910 tx=0x7f8acc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead074f48c21c30 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:18.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.360+0000 7f8ae117b700 1 -- 172.21.15.7:0/859967701 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc011830 con 0x7f8adc0cf4a0 2024-09-13T18:53:18.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.360+0000 7f8ae117b700 1 -- 172.21.15.7:0/859967701 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc011990 con 0x7f8adc0cf4a0 2024-09-13T18:53:18.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.360+0000 7f8ae117b700 1 -- 172.21.15.7:0/859967701 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc011d40 con 0x7f8adc0cf4a0 2024-09-13T18:53:18.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.360+0000 7f8ae43e1700 1 -- 172.21.15.7:0/859967701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0cf4a0 msgr2=0x7f8adc0b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:18.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.360+0000 7f8ae43e1700 1 --2- 172.21.15.7:0/859967701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0cf4a0 0x7f8adc0b37b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009910 tx=0x7f8acc009c20 comp rx=0 tx=0).stop 2024-09-13T18:53:18.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.360+0000 7f8ae43e1700 1 -- 172.21.15.7:0/859967701 shutdown_connections 2024-09-13T18:53:18.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.360+0000 7f8ae43e1700 1 --2- 172.21.15.7:0/859967701 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc0ce760 0x7f8adc0cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.360+0000 7f8ae43e1700 1 --2- 172.21.15.7:0/859967701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0cf4a0 0x7f8adc0b37b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.360+0000 7f8ae43e1700 1 -- 172.21.15.7:0/859967701 >> 172.21.15.7:0/859967701 conn(0x7f8adc0b1780 msgr2=0x7f8adc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:18.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.360+0000 7f8ae43e1700 1 -- 172.21.15.7:0/859967701 shutdown_connections 2024-09-13T18:53:18.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.360+0000 7f8ae43e1700 1 -- 172.21.15.7:0/859967701 wait complete. 2024-09-13T18:53:18.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.360+0000 7f8ae43e1700 1 Processor -- start 2024-09-13T18:53:18.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.360+0000 7f8ae43e1700 1 -- start start 2024-09-13T18:53:18.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae43e1700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc0cc450 0x7f8adc157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:18.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae43e1700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8adc0ce760 0x7f8adc06b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:18.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae43e1700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0cf4a0 0x7f8adc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:18.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae43e1700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc06c5c0 con 0x7f8adc0cf4a0 2024-09-13T18:53:18.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae43e1700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc06c000 con 0x7f8adc0ce760 2024-09-13T18:53:18.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae43e1700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc06c2e0 con 0x7f8adc0cc450 2024-09-13T18:53:18.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae297e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0cf4a0 0x7f8adc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:18.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae297e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0cf4a0 0x7f8adc06c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46378/0 (socket says 172.21.15.7:46378) 2024-09-13T18:53:18.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae297e700 1 -- 172.21.15.7:0/457311993 learned_addr learned my addr 172.21.15.7:0/457311993 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:18.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae197c700 1 --2- 172.21.15.7:0/457311993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8adc0ce760 0x7f8adc06b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:18.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae217d700 1 --2- 172.21.15.7:0/457311993 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc0cc450 0x7f8adc157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:18.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae297e700 1 -- 172.21.15.7:0/457311993 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc0cc450 msgr2=0x7f8adc157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:18.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae297e700 1 --2- 172.21.15.7:0/457311993 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc0cc450 0x7f8adc157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae297e700 1 -- 172.21.15.7:0/457311993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8adc0ce760 msgr2=0x7f8adc06b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:18.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae297e700 1 --2- 172.21.15.7:0/457311993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8adc0ce760 0x7f8adc06b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae297e700 1 -- 172.21.15.7:0/457311993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad4009a70 con 0x7f8adc0cf4a0 2024-09-13T18:53:18.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae297e700 1 --2- 172.21.15.7:0/457311993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0cf4a0 0x7f8adc06c920 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400efc0 tx=0x7f8ad400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:18.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ad2ffd700 1 -- 172.21.15.7:0/457311993 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad400ca90 con 0x7f8adc0cf4a0 2024-09-13T18:53:18.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ad2ffd700 1 -- 172.21.15.7:0/457311993 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad400f040 con 0x7f8adc0cf4a0 2024-09-13T18:53:18.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae43e1700 1 -- 172.21.15.7:0/457311993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8acc009570 con 0x7f8adc0cf4a0 2024-09-13T18:53:18.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ad2ffd700 1 -- 172.21.15.7:0/457311993 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad400ce10 con 0x7f8adc0cf4a0 2024-09-13T18:53:18.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.364+0000 7f8ae43e1700 1 -- 172.21.15.7:0/457311993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc0acf40 con 0x7f8adc0cf4a0 2024-09-13T18:53:18.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.368+0000 7f8ae43e1700 1 -- 172.21.15.7:0/457311993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8adc0ad160 con 0x7f8adc0cf4a0 2024-09-13T18:53:18.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.368+0000 7f8ad2ffd700 1 -- 172.21.15.7:0/457311993 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8ad4014480 con 0x7f8adc0cf4a0 2024-09-13T18:53:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.368+0000 7f8ad2ffd700 1 --2- 172.21.15.7:0/457311993 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8ac0066730 0x7f8ac0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.368+0000 7f8ad2ffd700 1 -- 172.21.15.7:0/457311993 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ad4084790 con 0x7f8adc0cf4a0 2024-09-13T18:53:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.368+0000 7f8ae217d700 1 --2- 172.21.15.7:0/457311993 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8ac0066730 0x7f8ac0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:18.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.368+0000 7f8ae217d700 1 --2- 172.21.15.7:0/457311993 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8ac0066730 0x7f8ac0068bf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8acc014040 tx=0x7f8acc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:18.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.376+0000 7f8ad2ffd700 1 -- 172.21.15.7:0/457311993 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8ad4051af0 con 0x7f8adc0cf4a0 2024-09-13T18:53:18.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.552+0000 7f8ae43e1700 1 -- 172.21.15.7:0/457311993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8adc0cc8c0 con 0x7f8adc0cf4a0 2024-09-13T18:53:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.556+0000 7f8ad2ffd700 1 -- 172.21.15.7:0/457311993 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16534 (secure 0 0 0) 0x7f8ad4009d70 con 0x7f8adc0cf4a0 2024-09-13T18:53:18.564 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:18.564 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:53:17.896570+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:53:15.862087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T18:53:16.885280+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:53:18.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.560+0000 7f8ae43e1700 1 -- 172.21.15.7:0/457311993 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8ac0066730 msgr2=0x7f8ac0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:18.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.560+0000 7f8ae43e1700 1 --2- 172.21.15.7:0/457311993 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8ac0066730 0x7f8ac0068bf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8acc014040 tx=0x7f8acc008030 comp rx=0 tx=0).stop 2024-09-13T18:53:18.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.560+0000 7f8ae43e1700 1 -- 172.21.15.7:0/457311993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0cf4a0 msgr2=0x7f8adc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:18.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.560+0000 7f8ae43e1700 1 --2- 172.21.15.7:0/457311993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0cf4a0 0x7f8adc06c920 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400efc0 tx=0x7f8ad400c3a0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.560+0000 7f8ae43e1700 1 -- 172.21.15.7:0/457311993 shutdown_connections 2024-09-13T18:53:18.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.560+0000 7f8ae43e1700 1 --2- 172.21.15.7:0/457311993 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8ac0066730 0x7f8ac0068bf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.560+0000 7f8ae43e1700 1 --2- 172.21.15.7:0/457311993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0cf4a0 0x7f8adc06c920 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.560+0000 7f8ae43e1700 1 --2- 172.21.15.7:0/457311993 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8adc0ce760 0x7f8adc06b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.560+0000 7f8ae43e1700 1 --2- 172.21.15.7:0/457311993 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8adc0cc450 0x7f8adc157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.560+0000 7f8ae43e1700 1 -- 172.21.15.7:0/457311993 >> 172.21.15.7:0/457311993 conn(0x7f8adc0b1780 msgr2=0x7f8adc15c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.560+0000 7f8ae43e1700 1 -- 172.21.15.7:0/457311993 shutdown_connections 2024-09-13T18:53:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.560+0000 7f8ae43e1700 1 -- 172.21.15.7:0/457311993 wait complete. 2024-09-13T18:53:18.578 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T18:53:18.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.692+0000 7f8597fff700 1 Processor -- start 2024-09-13T18:53:18.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.696+0000 7f8597fff700 1 -- start start 2024-09-13T18:53:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.696+0000 7f8597fff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f859815cda0 0x7f859806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.696+0000 7f8597fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f859806a520 0x7f859806a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.696+0000 7f8597fff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85980b8430 con 0x7f85980ceb70 2024-09-13T18:53:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.696+0000 7f8597fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859806be40 con 0x7f859806a520 2024-09-13T18:53:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.696+0000 7f8597fff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85980b64f0 con 0x7f859815cda0 2024-09-13T18:53:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.696+0000 7f85977fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f859806a520 0x7f859806a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.696+0000 7f85977fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f859806a520 0x7f859806a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46388/0 (socket says 172.21.15.7:46388) 2024-09-13T18:53:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.696+0000 7f85977fe700 1 -- 172.21.15.7:0/176598424 learned_addr learned my addr 172.21.15.7:0/176598424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.696+0000 7f8596ffd700 1 --2- 172.21.15.7:0/176598424 >> v2:172.21.15.171:3300/0 conn(0x7f859815cda0 0x7f859806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.696+0000 7f85977fe700 1 -- 172.21.15.7:0/176598424 >> v2:172.21.15.171:3300/0 conn(0x7f859815cda0 msgr2=0x7f859806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:18.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.696+0000 7f85977fe700 1 --2- 172.21.15.7:0/176598424 >> v2:172.21.15.171:3300/0 conn(0x7f859815cda0 0x7f859806b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.696+0000 7f85977fe700 1 -- 172.21.15.7:0/176598424 >> v1:172.21.15.171:6789/0 conn(0x7f85980ceb70 legacy=0x7f85980b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:18.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.696+0000 7f85977fe700 1 -- 172.21.15.7:0/176598424 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859806ae70 con 0x7f859806a520 2024-09-13T18:53:18.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.696+0000 7f85977fe700 1 --2- 172.21.15.7:0/176598424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859806a520 0x7f859806a930 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f85980d0800 tx=0x7f859000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4779800a55a570cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:18.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.696+0000 7f85967fc700 1 -- 172.21.15.7:0/176598424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859000e070 con 0x7f859806a520 2024-09-13T18:53:18.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.696+0000 7f85967fc700 1 -- 172.21.15.7:0/176598424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859000b910 con 0x7f859806a520 2024-09-13T18:53:18.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.696+0000 7f85967fc700 1 -- 172.21.15.7:0/176598424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859000bcc0 con 0x7f859806a520 2024-09-13T18:53:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.700+0000 7f8597fff700 1 -- 172.21.15.7:0/176598424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859806a520 msgr2=0x7f859806a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.700+0000 7f8597fff700 1 --2- 172.21.15.7:0/176598424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859806a520 0x7f859806a930 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f85980d0800 tx=0x7f859000b330 comp rx=0 tx=0).stop 2024-09-13T18:53:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.700+0000 7f8597fff700 1 -- 172.21.15.7:0/176598424 shutdown_connections 2024-09-13T18:53:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.700+0000 7f8597fff700 1 --2- 172.21.15.7:0/176598424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859806a520 0x7f859806a930 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.700+0000 7f8597fff700 1 --2- 172.21.15.7:0/176598424 >> v2:172.21.15.171:3300/0 conn(0x7f859815cda0 0x7f859806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.700+0000 7f8597fff700 1 -- 172.21.15.7:0/176598424 >> 172.21.15.7:0/176598424 conn(0x7f85980d2cb0 msgr2=0x7f85980d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.700+0000 7f8597fff700 1 -- 172.21.15.7:0/176598424 shutdown_connections 2024-09-13T18:53:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.700+0000 7f8597fff700 1 -- 172.21.15.7:0/176598424 wait complete. 2024-09-13T18:53:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.700+0000 7f8597fff700 1 Processor -- start 2024-09-13T18:53:18.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.700+0000 7f8597fff700 1 -- start start 2024-09-13T18:53:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f8597fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f859806a520 0x7f859815d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f8597fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85980ceb70 0x7f85981e8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f8597fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859815cda0 0x7f85980cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f8597fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85981e85e0 con 0x7f859815cda0 2024-09-13T18:53:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f8597fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85981e7fd0 con 0x7f85980ceb70 2024-09-13T18:53:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f8597fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859815fe80 con 0x7f859806a520 2024-09-13T18:53:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f85977fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859815cda0 0x7f85980cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f85977fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859815cda0 0x7f85980cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46400/0 (socket says 172.21.15.7:46400) 2024-09-13T18:53:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f85977fe700 1 -- 172.21.15.7:0/2717011760 learned_addr learned my addr 172.21.15.7:0/2717011760 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f858ffff700 1 --2- 172.21.15.7:0/2717011760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85980ceb70 0x7f85981e8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f8596ffd700 1 --2- 172.21.15.7:0/2717011760 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f859806a520 0x7f859815d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:18.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f858ffff700 1 -- 172.21.15.7:0/2717011760 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f859806a520 msgr2=0x7f859815d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:18.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f858ffff700 1 --2- 172.21.15.7:0/2717011760 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f859806a520 0x7f859815d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f858ffff700 1 -- 172.21.15.7:0/2717011760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859815cda0 msgr2=0x7f85980cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:18.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f858ffff700 1 --2- 172.21.15.7:0/2717011760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859815cda0 0x7f85980cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:18.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f858ffff700 1 -- 172.21.15.7:0/2717011760 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8590009b30 con 0x7f85980ceb70 2024-09-13T18:53:18.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f85977fe700 1 --2- 172.21.15.7:0/2717011760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859815cda0 0x7f85980cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:18.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f858ffff700 1 --2- 172.21.15.7:0/2717011760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85980ceb70 0x7f85981e8a70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f858000c400 tx=0x7f858000c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:18.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f858f7fe700 1 -- 172.21.15.7:0/2717011760 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85800052d0 con 0x7f85980ceb70 2024-09-13T18:53:18.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f858f7fe700 1 -- 172.21.15.7:0/2717011760 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8580005450 con 0x7f85980ceb70 2024-09-13T18:53:18.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f8597fff700 1 -- 172.21.15.7:0/2717011760 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8598160160 con 0x7f85980ceb70 2024-09-13T18:53:18.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f858f7fe700 1 -- 172.21.15.7:0/2717011760 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8580006ac0 con 0x7f85980ceb70 2024-09-13T18:53:18.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f8597fff700 1 -- 172.21.15.7:0/2717011760 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85980ccbe0 con 0x7f85980ceb70 2024-09-13T18:53:18.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.704+0000 7f8597fff700 1 -- 172.21.15.7:0/2717011760 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85981e8790 con 0x7f85980ceb70 2024-09-13T18:53:18.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.708+0000 7f858f7fe700 1 -- 172.21.15.7:0/2717011760 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f85800036a0 con 0x7f85980ceb70 2024-09-13T18:53:18.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.708+0000 7f858f7fe700 1 --2- 172.21.15.7:0/2717011760 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f857c066730 0x7f857c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:18.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.708+0000 7f858f7fe700 1 -- 172.21.15.7:0/2717011760 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8580083230 con 0x7f85980ceb70 2024-09-13T18:53:18.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.708+0000 7f8596ffd700 1 --2- 172.21.15.7:0/2717011760 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f857c066730 0x7f857c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:18.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.708+0000 7f8596ffd700 1 --2- 172.21.15.7:0/2717011760 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f857c066730 0x7f857c068bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8588009b50 tx=0x7f8588009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:18.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.720+0000 7f858f7fe700 1 -- 172.21.15.7:0/2717011760 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f85981e8790 con 0x7f85980ceb70 2024-09-13T18:53:18.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:18.932+0000 7f8597fff700 1 -- 172.21.15.7:0/2717011760 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f85981e8790 con 0x7f85980ceb70 2024-09-13T18:53:19.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:19.920+0000 7f858f7fe700 1 -- 172.21.15.7:0/2717011760 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f85981e8790 con 0x7f85980ceb70 2024-09-13T18:53:19.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:19.924+0000 7f8597fff700 1 -- 172.21.15.7:0/2717011760 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f857c066730 msgr2=0x7f857c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:19.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:19.924+0000 7f8597fff700 1 --2- 172.21.15.7:0/2717011760 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f857c066730 0x7f857c068bf0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8588009b50 tx=0x7f8588009240 comp rx=0 tx=0).stop 2024-09-13T18:53:19.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:19.924+0000 7f8597fff700 1 -- 172.21.15.7:0/2717011760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85980ceb70 msgr2=0x7f85981e8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:19.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:19.924+0000 7f8597fff700 1 --2- 172.21.15.7:0/2717011760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85980ceb70 0x7f85981e8a70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f858000c400 tx=0x7f858000c710 comp rx=0 tx=0).stop 2024-09-13T18:53:19.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:19.924+0000 7f8597fff700 1 -- 172.21.15.7:0/2717011760 shutdown_connections 2024-09-13T18:53:19.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:19.924+0000 7f8597fff700 1 --2- 172.21.15.7:0/2717011760 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f857c066730 0x7f857c068bf0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:19.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:19.924+0000 7f8597fff700 1 --2- 172.21.15.7:0/2717011760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859815cda0 0x7f85980cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:19.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:19.924+0000 7f8597fff700 1 --2- 172.21.15.7:0/2717011760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85980ceb70 0x7f85981e8a70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:19.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:19.924+0000 7f8597fff700 1 --2- 172.21.15.7:0/2717011760 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f859806a520 0x7f859815d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:19.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:19.924+0000 7f8597fff700 1 -- 172.21.15.7:0/2717011760 >> 172.21.15.7:0/2717011760 conn(0x7f85980d2cb0 msgr2=0x7f859806b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:19.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:19.924+0000 7f8597fff700 1 -- 172.21.15.7:0/2717011760 shutdown_connections 2024-09-13T18:53:19.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:19.924+0000 7f8597fff700 1 -- 172.21.15.7:0/2717011760 wait complete. 2024-09-13T18:53:19.953 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T18:53:19.953 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T18:53:20.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.072+0000 7fb291594700 1 Processor -- start 2024-09-13T18:53:20.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.072+0000 7fb291594700 1 -- start start 2024-09-13T18:53:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.072+0000 7fb291594700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fb28c144590 0x7fb28c0b5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.072+0000 7fb291594700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb28c0b4040 0x7fb28c071c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.072+0000 7fb291594700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb28c15b0c0 0x7fb28c0721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.072+0000 7fb291594700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb28c0b3080 con 0x7fb28c0b4040 2024-09-13T18:53:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.072+0000 7fb291594700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb28c0b4b00 con 0x7fb28c144590 2024-09-13T18:53:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.072+0000 7fb291594700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28c144970 con 0x7fb28c15b0c0 2024-09-13T18:53:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.072+0000 7fb28a7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb28c0b4040 0x7fb28c071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.072+0000 7fb28affd700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fb28c144590 0x7fb28c0b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:20.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.072+0000 7fb28a7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb28c0b4040 0x7fb28c071c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46406/0 (socket says 172.21.15.7:46406) 2024-09-13T18:53:20.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.072+0000 7fb28b7fe700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb28c15b0c0 0x7fb28c0721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:20.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.072+0000 7fb28a7fc700 1 -- 172.21.15.7:0/1672379005 learned_addr learned my addr 172.21.15.7:0/1672379005 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:20.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.076+0000 7fb28affd700 1 -- 172.21.15.7:0/1672379005 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb28c15b0c0 msgr2=0x7fb28c0721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:20.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.076+0000 7fb28affd700 1 --2- 172.21.15.7:0/1672379005 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb28c15b0c0 0x7fb28c0721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:20.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.076+0000 7fb28affd700 1 -- 172.21.15.7:0/1672379005 >> v2:172.21.15.7:3300/0 conn(0x7fb28c0b4040 msgr2=0x7fb28c071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:20.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.076+0000 7fb28affd700 1 --2- 172.21.15.7:0/1672379005 >> v2:172.21.15.7:3300/0 conn(0x7fb28c0b4040 0x7fb28c071c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:20.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.076+0000 7fb28affd700 1 -- 172.21.15.7:0/1672379005 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28c0cdf30 con 0x7fb28c144590 2024-09-13T18:53:20.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.076+0000 7fb28affd700 1 --2- 172.21.15.7:0/1672379005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb28c144590 0x7fb28c0b5650 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb28c0d0a40 tx=0x7fb27c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5ee464e04de0af2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:20.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.076+0000 7fb289ffb700 1 -- 172.21.15.7:0/1672379005 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c00e070 con 0x7fb28c144590 2024-09-13T18:53:20.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.076+0000 7fb289ffb700 1 -- 172.21.15.7:0/1672379005 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27c00bae0 con 0x7fb28c144590 2024-09-13T18:53:20.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.076+0000 7fb289ffb700 1 -- 172.21.15.7:0/1672379005 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c00be90 con 0x7fb28c144590 2024-09-13T18:53:20.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.076+0000 7fb291594700 1 -- 172.21.15.7:0/1672379005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb28c144590 msgr2=0x7fb28c0b5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:20.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.076+0000 7fb291594700 1 --2- 172.21.15.7:0/1672379005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb28c144590 0x7fb28c0b5650 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb28c0d0a40 tx=0x7fb27c00b330 comp rx=0 tx=0).stop 2024-09-13T18:53:20.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.076+0000 7fb291594700 1 -- 172.21.15.7:0/1672379005 shutdown_connections 2024-09-13T18:53:20.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.076+0000 7fb291594700 1 --2- 172.21.15.7:0/1672379005 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb28c15b0c0 0x7fb28c0721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:20.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.076+0000 7fb291594700 1 --2- 172.21.15.7:0/1672379005 >> v2:172.21.15.7:3300/0 conn(0x7fb28c0b4040 0x7fb28c071c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:20.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.076+0000 7fb291594700 1 --2- 172.21.15.7:0/1672379005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb28c144590 0x7fb28c0b5650 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:20.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.076+0000 7fb291594700 1 -- 172.21.15.7:0/1672379005 >> 172.21.15.7:0/1672379005 conn(0x7fb28c0b1810 msgr2=0x7fb28c0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:20.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.076+0000 7fb291594700 1 -- 172.21.15.7:0/1672379005 shutdown_connections 2024-09-13T18:53:20.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.076+0000 7fb291594700 1 -- 172.21.15.7:0/1672379005 wait complete. 2024-09-13T18:53:20.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb291594700 1 Processor -- start 2024-09-13T18:53:20.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb291594700 1 -- start start 2024-09-13T18:53:20.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb291594700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb28c0b4040 0x7fb28c13b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:20.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb291594700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb28c144590 0x7fb28c1f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:20.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb291594700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb28c15b0c0 0x7fb28c1f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:20.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb291594700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28c13b9d0 con 0x7fb28c0b4040 2024-09-13T18:53:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb291594700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28c1f2c40 con 0x7fb28c144590 2024-09-13T18:53:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb291594700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28c1f2030 con 0x7fb28c15b0c0 2024-09-13T18:53:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb28affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb28c0b4040 0x7fb28c13b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb28affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb28c0b4040 0x7fb28c13b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46422/0 (socket says 172.21.15.7:46422) 2024-09-13T18:53:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb28affd700 1 -- 172.21.15.7:0/2849194499 learned_addr learned my addr 172.21.15.7:0/2849194499 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb28b7fe700 1 --2- 172.21.15.7:0/2849194499 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb28c15b0c0 0x7fb28c1f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb28a7fc700 1 --2- 172.21.15.7:0/2849194499 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb28c144590 0x7fb28c1f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb28affd700 1 -- 172.21.15.7:0/2849194499 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb28c15b0c0 msgr2=0x7fb28c1f15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb28affd700 1 --2- 172.21.15.7:0/2849194499 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb28c15b0c0 0x7fb28c1f15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb28affd700 1 -- 172.21.15.7:0/2849194499 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb28c144590 msgr2=0x7fb28c1f30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb28affd700 1 --2- 172.21.15.7:0/2849194499 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb28c144590 0x7fb28c1f30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb28affd700 1 -- 172.21.15.7:0/2849194499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27c009a70 con 0x7fb28c0b4040 2024-09-13T18:53:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb28affd700 1 --2- 172.21.15.7:0/2849194499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb28c0b4040 0x7fb28c13b490 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb27c000f80 tx=0x7fb27c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:20.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb2837fe700 1 -- 172.21.15.7:0/2849194499 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c004680 con 0x7fb28c0b4040 2024-09-13T18:53:20.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb2837fe700 1 -- 172.21.15.7:0/2849194499 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27c020070 con 0x7fb28c0b4040 2024-09-13T18:53:20.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb291594700 1 -- 172.21.15.7:0/2849194499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb28c1f2310 con 0x7fb28c0b4040 2024-09-13T18:53:20.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb2837fe700 1 -- 172.21.15.7:0/2849194499 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c027480 con 0x7fb28c0b4040 2024-09-13T18:53:20.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.080+0000 7fb291594700 1 -- 172.21.15.7:0/2849194499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb28c1f28d0 con 0x7fb28c0b4040 2024-09-13T18:53:20.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.084+0000 7fb291594700 1 -- 172.21.15.7:0/2849194499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb28c0b5650 con 0x7fb28c0b4040 2024-09-13T18:53:20.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.084+0000 7fb2837fe700 1 -- 172.21.15.7:0/2849194499 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb27c0275e0 con 0x7fb28c0b4040 2024-09-13T18:53:20.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.084+0000 7fb2837fe700 1 --2- 172.21.15.7:0/2849194499 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb26c066760 0x7fb26c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:20.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.084+0000 7fb2837fe700 1 -- 172.21.15.7:0/2849194499 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb27c022070 con 0x7fb28c0b4040 2024-09-13T18:53:20.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.084+0000 7fb28a7fc700 1 --2- 172.21.15.7:0/2849194499 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb26c066760 0x7fb26c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:20.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.084+0000 7fb28a7fc700 1 --2- 172.21.15.7:0/2849194499 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb26c066760 0x7fb26c068c20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb28c1560a0 tx=0x7fb274008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:20.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.092+0000 7fb2837fe700 1 -- 172.21.15.7:0/2849194499 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb27c0539b0 con 0x7fb28c0b4040 2024-09-13T18:53:20.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.304+0000 7fb291594700 1 -- 172.21.15.7:0/2849194499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb28c071a00 con 0x7fb28c0b4040 2024-09-13T18:53:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.932+0000 7fb2837fe700 1 -- 172.21.15.7:0/2849194499 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb27c056fd0 con 0x7fb28c0b4040 2024-09-13T18:53:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.936+0000 7fb291594700 1 -- 172.21.15.7:0/2849194499 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb26c066760 msgr2=0x7fb26c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.936+0000 7fb291594700 1 --2- 172.21.15.7:0/2849194499 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb26c066760 0x7fb26c068c20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb28c1560a0 tx=0x7fb274008040 comp rx=0 tx=0).stop 2024-09-13T18:53:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.936+0000 7fb291594700 1 -- 172.21.15.7:0/2849194499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb28c0b4040 msgr2=0x7fb28c13b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.936+0000 7fb291594700 1 --2- 172.21.15.7:0/2849194499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb28c0b4040 0x7fb28c13b490 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb27c000f80 tx=0x7fb27c004af0 comp rx=0 tx=0).stop 2024-09-13T18:53:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.936+0000 7fb291594700 1 -- 172.21.15.7:0/2849194499 shutdown_connections 2024-09-13T18:53:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.936+0000 7fb291594700 1 --2- 172.21.15.7:0/2849194499 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb26c066760 0x7fb26c068c20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.936+0000 7fb291594700 1 --2- 172.21.15.7:0/2849194499 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb28c15b0c0 0x7fb28c1f15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.936+0000 7fb291594700 1 --2- 172.21.15.7:0/2849194499 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb28c144590 0x7fb28c1f30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.936+0000 7fb291594700 1 --2- 172.21.15.7:0/2849194499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb28c0b4040 0x7fb28c13b490 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:20.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.936+0000 7fb291594700 1 -- 172.21.15.7:0/2849194499 >> 172.21.15.7:0/2849194499 conn(0x7fb28c0b1810 msgr2=0x7fb28c0b0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.936+0000 7fb291594700 1 -- 172.21.15.7:0/2849194499 shutdown_connections 2024-09-13T18:53:20.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:20.936+0000 7fb291594700 1 -- 172.21.15.7:0/2849194499 wait complete. 2024-09-13T18:53:20.962 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T18:53:20.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:21.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.076+0000 7eff8cafd700 1 Processor -- start 2024-09-13T18:53:21.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.076+0000 7eff8cafd700 1 -- start start 2024-09-13T18:53:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff8cafd700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff8815ac40 0x7eff8806ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff8cafd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7eff880adb70 0x7eff8815ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff8cafd700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff8815b020 con 0x7eff880ad710 2024-09-13T18:53:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff8cafd700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff880b31e0 con 0x7eff880adb70 2024-09-13T18:53:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff8cafd700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff880b3df0 con 0x7eff8815ac40 2024-09-13T18:53:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff86d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7eff880adb70 0x7eff8815ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff85d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7eff880ad710 0x7eff8815e730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47398/0 (socket says 172.21.15.7:47398) 2024-09-13T18:53:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff85d9b700 1 -- 172.21.15.7:0/2762334101 learned_addr learned my addr 172.21.15.7:0/2762334101 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff8659c700 1 --2- 172.21.15.7:0/2762334101 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff8815ac40 0x7eff8806ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff86d9d700 1 -- 172.21.15.7:0/2762334101 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff8815ac40 msgr2=0x7eff8806ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:21.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff86d9d700 1 --2- 172.21.15.7:0/2762334101 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff8815ac40 0x7eff8806ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff8559a700 1 -- 172.21.15.7:0/2762334101 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639644907 0 0) 0x7eff8815b020 con 0x7eff880ad710 2024-09-13T18:53:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff86d9d700 1 -- 172.21.15.7:0/2762334101 >> v1:172.21.15.7:6789/0 conn(0x7eff880ad710 legacy=0x7eff8815e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff86d9d700 1 -- 172.21.15.7:0/2762334101 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8806a6f0 con 0x7eff880adb70 2024-09-13T18:53:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff8659c700 1 --2- 172.21.15.7:0/2762334101 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff8815ac40 0x7eff8806ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff86d9d700 1 --2- 172.21.15.7:0/2762334101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff880adb70 0x7eff8815ee40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7eff80009dc0 tx=0x7eff8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde820ffadff59fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff8559a700 1 -- 172.21.15.7:0/2762334101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8000e070 con 0x7eff880adb70 2024-09-13T18:53:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff8559a700 1 -- 172.21.15.7:0/2762334101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8000bae0 con 0x7eff880adb70 2024-09-13T18:53:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff8559a700 1 -- 172.21.15.7:0/2762334101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8000be90 con 0x7eff880adb70 2024-09-13T18:53:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff8cafd700 1 -- 172.21.15.7:0/2762334101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff880adb70 msgr2=0x7eff8815ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.080+0000 7eff8cafd700 1 --2- 172.21.15.7:0/2762334101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff880adb70 0x7eff8815ee40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7eff80009dc0 tx=0x7eff8000b330 comp rx=0 tx=0).stop 2024-09-13T18:53:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.084+0000 7eff8cafd700 1 -- 172.21.15.7:0/2762334101 shutdown_connections 2024-09-13T18:53:21.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.084+0000 7eff8cafd700 1 --2- 172.21.15.7:0/2762334101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff880adb70 0x7eff8815ee40 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.084+0000 7eff8cafd700 1 --2- 172.21.15.7:0/2762334101 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff8815ac40 0x7eff8806ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.084+0000 7eff8cafd700 1 -- 172.21.15.7:0/2762334101 >> 172.21.15.7:0/2762334101 conn(0x7eff880ce980 msgr2=0x7eff880ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.084+0000 7eff8cafd700 1 -- 172.21.15.7:0/2762334101 shutdown_connections 2024-09-13T18:53:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.084+0000 7eff8cafd700 1 -- 172.21.15.7:0/2762334101 wait complete. 2024-09-13T18:53:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.084+0000 7eff8cafd700 1 Processor -- start 2024-09-13T18:53:21.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.084+0000 7eff8cafd700 1 -- start start 2024-09-13T18:53:21.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff8cafd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff880ad710 0x7eff8814a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff8cafd700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff880adb70 0x7eff88152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff8cafd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff8815ac40 0x7eff8813ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff8cafd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8815bd30 con 0x7eff8815ac40 2024-09-13T18:53:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff8cafd700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8815c010 con 0x7eff880ad710 2024-09-13T18:53:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff8cafd700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8815c2f0 con 0x7eff880adb70 2024-09-13T18:53:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff86d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff8815ac40 0x7eff8813ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff85d9b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff880adb70 0x7eff88152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff8659c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff880ad710 0x7eff8814a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff86d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff8815ac40 0x7eff8813ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46444/0 (socket says 172.21.15.7:46444) 2024-09-13T18:53:21.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff86d9d700 1 -- 172.21.15.7:0/4048483023 learned_addr learned my addr 172.21.15.7:0/4048483023 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff8659c700 1 -- 172.21.15.7:0/4048483023 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff880adb70 msgr2=0x7eff88152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff8659c700 1 --2- 172.21.15.7:0/4048483023 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff880adb70 0x7eff88152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff8659c700 1 -- 172.21.15.7:0/4048483023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff8815ac40 msgr2=0x7eff8813ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff8659c700 1 --2- 172.21.15.7:0/4048483023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff8815ac40 0x7eff8813ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff8659c700 1 -- 172.21.15.7:0/4048483023 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff780094b0 con 0x7eff880ad710 2024-09-13T18:53:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff8659c700 1 --2- 172.21.15.7:0/4048483023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff880ad710 0x7eff8814a370 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7eff7800e9b0 tx=0x7eff7800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff7effd700 1 -- 172.21.15.7:0/4048483023 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7800c640 con 0x7eff880ad710 2024-09-13T18:53:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff7effd700 1 -- 172.21.15.7:0/4048483023 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7800c7a0 con 0x7eff880ad710 2024-09-13T18:53:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff8cafd700 1 -- 172.21.15.7:0/4048483023 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff80009a70 con 0x7eff880ad710 2024-09-13T18:53:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff7effd700 1 -- 172.21.15.7:0/4048483023 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7800cb50 con 0x7eff880ad710 2024-09-13T18:53:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff8cafd700 1 -- 172.21.15.7:0/4048483023 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff881559e0 con 0x7eff880ad710 2024-09-13T18:53:21.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.088+0000 7eff8cafd700 1 -- 172.21.15.7:0/4048483023 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8815be70 con 0x7eff880ad710 2024-09-13T18:53:21.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.092+0000 7eff7effd700 1 -- 172.21.15.7:0/4048483023 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7eff7800ccb0 con 0x7eff880ad710 2024-09-13T18:53:21.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.092+0000 7eff7effd700 1 --2- 172.21.15.7:0/4048483023 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7eff68066730 0x7eff68068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:21.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.092+0000 7eff7effd700 1 -- 172.21.15.7:0/4048483023 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff780846a0 con 0x7eff880ad710 2024-09-13T18:53:21.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.092+0000 7eff85d9b700 1 --2- 172.21.15.7:0/4048483023 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7eff68066730 0x7eff68068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:21.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.092+0000 7eff85d9b700 1 --2- 172.21.15.7:0/4048483023 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7eff68066730 0x7eff68068bf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7eff88153b80 tx=0x7eff70004a70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:21.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.100+0000 7eff7effd700 1 -- 172.21.15.7:0/4048483023 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7eff78051af0 con 0x7eff880ad710 2024-09-13T18:53:21.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.304+0000 7eff8cafd700 1 -- 172.21.15.7:0/4048483023 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff8815be70 con 0x7eff880ad710 2024-09-13T18:53:21.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.304+0000 7eff7effd700 1 -- 172.21.15.7:0/4048483023 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4734 (secure 0 0 0) 0x7eff8815be70 con 0x7eff880ad710 2024-09-13T18:53:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:21.313 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3430789885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3430789885},{"type":"v1","addr":"172.21.15.171:6837","nonce":3430789885}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1714488523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1714488523},{"type":"v1","addr":"172.21.15.171:6835","nonce":1714488523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1159212209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1159212209},{"type":"v1","addr":"172.21.15.7:6833","nonce":1159212209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:53:17.897317+0000","modified":"2024-09-13T18:53:20.920860+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2843879273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2843879273},{"type":"v1","addr":"172.21.15.7:6835","nonce":2843879273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T18:53:21.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.308+0000 7eff8cafd700 1 -- 172.21.15.7:0/4048483023 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7eff68066730 msgr2=0x7eff68068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.308+0000 7eff8cafd700 1 --2- 172.21.15.7:0/4048483023 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7eff68066730 0x7eff68068bf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7eff88153b80 tx=0x7eff70004a70 comp rx=0 tx=0).stop 2024-09-13T18:53:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.308+0000 7eff8cafd700 1 -- 172.21.15.7:0/4048483023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff880ad710 msgr2=0x7eff8814a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.308+0000 7eff8cafd700 1 --2- 172.21.15.7:0/4048483023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff880ad710 0x7eff8814a370 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7eff7800e9b0 tx=0x7eff7800ecc0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.308+0000 7eff8cafd700 1 -- 172.21.15.7:0/4048483023 shutdown_connections 2024-09-13T18:53:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.308+0000 7eff8cafd700 1 --2- 172.21.15.7:0/4048483023 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7eff68066730 0x7eff68068bf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.308+0000 7eff8cafd700 1 --2- 172.21.15.7:0/4048483023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff8815ac40 0x7eff8813ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.308+0000 7eff8cafd700 1 --2- 172.21.15.7:0/4048483023 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7eff880adb70 0x7eff88152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.308+0000 7eff8cafd700 1 --2- 172.21.15.7:0/4048483023 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7eff880ad710 0x7eff8814a370 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.308+0000 7eff8cafd700 1 -- 172.21.15.7:0/4048483023 >> 172.21.15.7:0/4048483023 conn(0x7eff880ce980 msgr2=0x7eff8806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.312+0000 7eff8cafd700 1 -- 172.21.15.7:0/4048483023 shutdown_connections 2024-09-13T18:53:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.312+0000 7eff8cafd700 1 -- 172.21.15.7:0/4048483023 wait complete. 2024-09-13T18:53:21.319 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2024-09-13T18:53:21.334 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:53:21.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.440+0000 7fd44e3bb700 1 Processor -- start 2024-09-13T18:53:21.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.440+0000 7fd44e3bb700 1 -- start start 2024-09-13T18:53:21.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd44e3bb700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd4480b3ae0 0x7fd44806cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:21.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd44e3bb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44806af40 con 0x7fd44806aa90 2024-09-13T18:53:21.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd44e3bb700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44806d690 con 0x7fd448159720 2024-09-13T18:53:21.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd44e3bb700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4480cbfe0 con 0x7fd4480b3ae0 2024-09-13T18:53:21.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd44dbba700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd44806aa90 0x7fd4480ccac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39190/0 (socket says 172.21.15.7:39190) 2024-09-13T18:53:21.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd44dbba700 1 -- 172.21.15.7:0/1224745458 learned_addr learned my addr 172.21.15.7:0/1224745458 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:21.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd44cbb8700 1 --2- 172.21.15.7:0/1224745458 >> v2:172.21.15.171:3300/0 conn(0x7fd4480b3ae0 0x7fd44806cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:21.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd43ffff700 1 -- 172.21.15.7:0/1224745458 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997514790 0 0) 0x7fd44806af40 con 0x7fd44806aa90 2024-09-13T18:53:21.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd43ffff700 1 -- 172.21.15.7:0/1224745458 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4340057e0 con 0x7fd44806aa90 2024-09-13T18:53:21.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd44cbb8700 1 -- 172.21.15.7:0/1224745458 >> v1:172.21.15.171:6789/0 conn(0x7fd448159720 legacy=0x7fd4480b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd44cbb8700 1 -- 172.21.15.7:0/1224745458 >> v1:172.21.15.7:6789/0 conn(0x7fd44806aa90 legacy=0x7fd4480ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd44cbb8700 1 -- 172.21.15.7:0/1224745458 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4480cc2c0 con 0x7fd4480b3ae0 2024-09-13T18:53:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd43ffff700 1 -- 172.21.15.7:0/1224745458 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 102808777 0 0) 0x7fd4340057e0 con 0x7fd44806aa90 2024-09-13T18:53:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd44cbb8700 1 --2- 172.21.15.7:0/1224745458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4480b3ae0 0x7fd44806cf90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd438009850 tx=0x7fd438009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f7be5b6605e2262 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd43ffff700 1 -- 172.21.15.7:0/1224745458 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438011830 con 0x7fd4480b3ae0 2024-09-13T18:53:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd43ffff700 1 -- 172.21.15.7:0/1224745458 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd438011990 con 0x7fd4480b3ae0 2024-09-13T18:53:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd43ffff700 1 -- 172.21.15.7:0/1224745458 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438011d00 con 0x7fd4480b3ae0 2024-09-13T18:53:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd44e3bb700 1 -- 172.21.15.7:0/1224745458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4480b3ae0 msgr2=0x7fd44806cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd44e3bb700 1 --2- 172.21.15.7:0/1224745458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4480b3ae0 0x7fd44806cf90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd438009850 tx=0x7fd438009b60 comp rx=0 tx=0).stop 2024-09-13T18:53:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd44e3bb700 1 -- 172.21.15.7:0/1224745458 shutdown_connections 2024-09-13T18:53:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd44e3bb700 1 --2- 172.21.15.7:0/1224745458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4480b3ae0 0x7fd44806cf90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd44e3bb700 1 -- 172.21.15.7:0/1224745458 >> 172.21.15.7:0/1224745458 conn(0x7fd4480b0f00 msgr2=0x7fd4480b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd44e3bb700 1 -- 172.21.15.7:0/1224745458 shutdown_connections 2024-09-13T18:53:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.444+0000 7fd44e3bb700 1 -- 172.21.15.7:0/1224745458 wait complete. 2024-09-13T18:53:21.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44e3bb700 1 Processor -- start 2024-09-13T18:53:21.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44e3bb700 1 -- start start 2024-09-13T18:53:21.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44e3bb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44806aa90 0x7fd4481503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:21.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44e3bb700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4480b3ae0 0x7fd448156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:21.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44e3bb700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd448159720 0x7fd448156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44e3bb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd448150d50 con 0x7fd44806aa90 2024-09-13T18:53:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44e3bb700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd448152460 con 0x7fd4480b3ae0 2024-09-13T18:53:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44e3bb700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd448152740 con 0x7fd448159720 2024-09-13T18:53:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44d3b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44806aa90 0x7fd4481503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44d3b9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44806aa90 0x7fd4481503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53234/0 (socket says 172.21.15.7:53234) 2024-09-13T18:53:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44d3b9700 1 -- 172.21.15.7:0/235699821 learned_addr learned my addr 172.21.15.7:0/235699821 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44cbb8700 1 --2- 172.21.15.7:0/235699821 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4480b3ae0 0x7fd448156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44dbba700 1 --2- 172.21.15.7:0/235699821 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd448159720 0x7fd448156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44d3b9700 1 -- 172.21.15.7:0/235699821 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd448159720 msgr2=0x7fd448156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44d3b9700 1 --2- 172.21.15.7:0/235699821 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd448159720 0x7fd448156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44d3b9700 1 -- 172.21.15.7:0/235699821 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4480b3ae0 msgr2=0x7fd448156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44d3b9700 1 --2- 172.21.15.7:0/235699821 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4480b3ae0 0x7fd448156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44d3b9700 1 -- 172.21.15.7:0/235699821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4380094b0 con 0x7fd44806aa90 2024-09-13T18:53:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44d3b9700 1 --2- 172.21.15.7:0/235699821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44806aa90 0x7fd4481503c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd44400c500 tx=0x7fd44400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:21.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd43dffb700 1 -- 172.21.15.7:0/235699821 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4440052d0 con 0x7fd44806aa90 2024-09-13T18:53:21.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd43dffb700 1 -- 172.21.15.7:0/235699821 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd444005430 con 0x7fd44806aa90 2024-09-13T18:53:21.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44e3bb700 1 -- 172.21.15.7:0/235699821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd448152a20 con 0x7fd44806aa90 2024-09-13T18:53:21.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd43dffb700 1 -- 172.21.15.7:0/235699821 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4440055a0 con 0x7fd44806aa90 2024-09-13T18:53:21.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.448+0000 7fd44e3bb700 1 -- 172.21.15.7:0/235699821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd448152110 con 0x7fd44806aa90 2024-09-13T18:53:21.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.452+0000 7fd44e3bb700 1 -- 172.21.15.7:0/235699821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4481525a0 con 0x7fd44806aa90 2024-09-13T18:53:21.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.452+0000 7fd43dffb700 1 -- 172.21.15.7:0/235699821 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd4440038e0 con 0x7fd44806aa90 2024-09-13T18:53:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.452+0000 7fd43dffb700 1 --2- 172.21.15.7:0/235699821 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd42c066730 0x7fd42c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.452+0000 7fd43dffb700 1 -- 172.21.15.7:0/235699821 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4440834a0 con 0x7fd44806aa90 2024-09-13T18:53:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.452+0000 7fd44cbb8700 1 --2- 172.21.15.7:0/235699821 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd42c066730 0x7fd42c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:21.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.456+0000 7fd44cbb8700 1 --2- 172.21.15.7:0/235699821 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd42c066730 0x7fd42c068bf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd4380090b0 tx=0x7fd438008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:21.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.460+0000 7fd43dffb700 1 -- 172.21.15.7:0/235699821 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd4481525a0 con 0x7fd44806aa90 2024-09-13T18:53:21.497 INFO:tasks.ceph.mds.a.smithi007.stderr:2024-09-13T18:53:21.488+0000 7fb7d4ef9700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T18:53:21.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.636+0000 7fd44e3bb700 1 -- 172.21.15.7:0/235699821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4481525a0 con 0x7fd44806aa90 2024-09-13T18:53:21.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.636+0000 7fd43dffb700 1 -- 172.21.15.7:0/235699821 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16521 (secure 0 0 0) 0x7fd4481525a0 con 0x7fd44806aa90 2024-09-13T18:53:21.644 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:21.644 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:53:18.900102+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:53:15.862087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T18:53:16.885280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:53:21.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.640+0000 7fd44e3bb700 1 -- 172.21.15.7:0/235699821 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd42c066730 msgr2=0x7fd42c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.640+0000 7fd44e3bb700 1 --2- 172.21.15.7:0/235699821 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd42c066730 0x7fd42c068bf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd4380090b0 tx=0x7fd438008030 comp rx=0 tx=0).stop 2024-09-13T18:53:21.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.640+0000 7fd44e3bb700 1 -- 172.21.15.7:0/235699821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44806aa90 msgr2=0x7fd4481503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.640+0000 7fd44e3bb700 1 --2- 172.21.15.7:0/235699821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44806aa90 0x7fd4481503c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd44400c500 tx=0x7fd44400c810 comp rx=0 tx=0).stop 2024-09-13T18:53:21.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.640+0000 7fd44e3bb700 1 -- 172.21.15.7:0/235699821 shutdown_connections 2024-09-13T18:53:21.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.640+0000 7fd44e3bb700 1 --2- 172.21.15.7:0/235699821 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd42c066730 0x7fd42c068bf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.640+0000 7fd44e3bb700 1 --2- 172.21.15.7:0/235699821 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd448159720 0x7fd448156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.640+0000 7fd44e3bb700 1 --2- 172.21.15.7:0/235699821 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd4480b3ae0 0x7fd448156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.640+0000 7fd44e3bb700 1 --2- 172.21.15.7:0/235699821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44806aa90 0x7fd4481503c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.640+0000 7fd44e3bb700 1 -- 172.21.15.7:0/235699821 >> 172.21.15.7:0/235699821 conn(0x7fd4480b0f00 msgr2=0x7fd4480adc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:21.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.640+0000 7fd44e3bb700 1 -- 172.21.15.7:0/235699821 shutdown_connections 2024-09-13T18:53:21.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.640+0000 7fd44e3bb700 1 -- 172.21.15.7:0/235699821 wait complete. 2024-09-13T18:53:21.659 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T18:53:21.659 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:53:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.768+0000 7f7fef59e700 1 Processor -- start 2024-09-13T18:53:21.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.768+0000 7f7fef59e700 1 -- start start 2024-09-13T18:53:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.768+0000 7f7fef59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7fe8006c80 0x7f7fe800aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.768+0000 7f7fef59e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe801d380 con 0x7f7fe800a5b0 2024-09-13T18:53:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.768+0000 7f7fef59e700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe801df20 con 0x7f7fe80e0760 2024-09-13T18:53:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.768+0000 7f7fef59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe800af60 con 0x7f7fe8006c80 2024-09-13T18:53:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.768+0000 7f7feed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7fe8006c80 0x7f7fe800aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.768+0000 7f7feed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7fe8006c80 0x7f7fe800aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53248/0 (socket says 172.21.15.7:53248) 2024-09-13T18:53:21.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.768+0000 7f7feed9d700 1 -- 172.21.15.7:0/1594923553 learned_addr learned my addr 172.21.15.7:0/1594923553 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:21.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.768+0000 7f7feed9d700 1 -- 172.21.15.7:0/1594923553 >> v1:172.21.15.171:6789/0 conn(0x7f7fe80e0760 legacy=0x7f7fe801d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:21.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.768+0000 7f7feed9d700 1 -- 172.21.15.7:0/1594923553 >> v1:172.21.15.7:6789/0 conn(0x7f7fe800a5b0 legacy=0x7f7fe8006570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.768+0000 7f7feed9d700 1 -- 172.21.15.7:0/1594923553 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe801c770 con 0x7f7fe8006c80 2024-09-13T18:53:21.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.772+0000 7f7feed9d700 1 --2- 172.21.15.7:0/1594923553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe8006c80 0x7f7fe800aa20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0063990 tx=0x7f7ff004f750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d682aaeedf41229c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:21.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.772+0000 7f7fed59a700 1 -- 172.21.15.7:0/1594923553 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff00730f0 con 0x7f7fe8006c80 2024-09-13T18:53:21.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.772+0000 7f7fed59a700 1 -- 172.21.15.7:0/1594923553 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff008e070 con 0x7f7fe8006c80 2024-09-13T18:53:21.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.772+0000 7f7fed59a700 1 -- 172.21.15.7:0/1594923553 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0073470 con 0x7f7fe8006c80 2024-09-13T18:53:21.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.772+0000 7f7fef59e700 1 -- 172.21.15.7:0/1594923553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe8006c80 msgr2=0x7f7fe800aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.772+0000 7f7fef59e700 1 --2- 172.21.15.7:0/1594923553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe8006c80 0x7f7fe800aa20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0063990 tx=0x7f7ff004f750 comp rx=0 tx=0).stop 2024-09-13T18:53:21.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.772+0000 7f7fef59e700 1 -- 172.21.15.7:0/1594923553 shutdown_connections 2024-09-13T18:53:21.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.772+0000 7f7fef59e700 1 --2- 172.21.15.7:0/1594923553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe8006c80 0x7f7fe800aa20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.772+0000 7f7fef59e700 1 -- 172.21.15.7:0/1594923553 >> 172.21.15.7:0/1594923553 conn(0x7f7fe8036140 msgr2=0x7f7fe8036550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:21.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.772+0000 7f7fef59e700 1 -- 172.21.15.7:0/1594923553 shutdown_connections 2024-09-13T18:53:21.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.772+0000 7f7fef59e700 1 -- 172.21.15.7:0/1594923553 wait complete. 2024-09-13T18:53:21.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7fef59e700 1 Processor -- start 2024-09-13T18:53:21.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7fef59e700 1 -- start start 2024-09-13T18:53:21.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7fef59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe8006c80 0x7f7fe8021a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:21.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7fef59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7fe800a5b0 0x7f7fe80dfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:21.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7fef59e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7fe80e0760 0x7f7fe804e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:21.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7fef59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe8021580 con 0x7f7fe8006c80 2024-09-13T18:53:21.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7fef59e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe80e0200 con 0x7f7fe80e0760 2024-09-13T18:53:21.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7fef59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe8020970 con 0x7f7fe800a5b0 2024-09-13T18:53:21.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7fee59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe8006c80 0x7f7fe8021a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:21.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7fee59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe8006c80 0x7f7fe8021a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53258/0 (socket says 172.21.15.7:53258) 2024-09-13T18:53:21.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7fee59c700 1 -- 172.21.15.7:0/1732873978 learned_addr learned my addr 172.21.15.7:0/1732873978 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:21.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7feed9d700 1 --2- 172.21.15.7:0/1732873978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7fe80e0760 0x7f7fe804e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:21.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7fedd9b700 1 --2- 172.21.15.7:0/1732873978 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7fe800a5b0 0x7f7fe80dfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:21.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7feed9d700 1 -- 172.21.15.7:0/1732873978 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7fe800a5b0 msgr2=0x7f7fe80dfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7feed9d700 1 --2- 172.21.15.7:0/1732873978 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7fe800a5b0 0x7f7fe80dfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7feed9d700 1 -- 172.21.15.7:0/1732873978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe8006c80 msgr2=0x7f7fe8021a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7feed9d700 1 --2- 172.21.15.7:0/1732873978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe8006c80 0x7f7fe8021a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7feed9d700 1 -- 172.21.15.7:0/1732873978 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff004f5c0 con 0x7f7fe80e0760 2024-09-13T18:53:21.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7feed9d700 1 --2- 172.21.15.7:0/1732873978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7fe80e0760 0x7f7fe804e240 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7ff004ff70 tx=0x7f7ff0074450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:21.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7fe6ffd700 1 -- 172.21.15.7:0/1732873978 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0067be0 con 0x7f7fe80e0760 2024-09-13T18:53:21.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7fef59e700 1 -- 172.21.15.7:0/1732873978 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe8020c50 con 0x7f7fe80e0760 2024-09-13T18:53:21.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7fe6ffd700 1 -- 172.21.15.7:0/1732873978 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff0067d40 con 0x7f7fe80e0760 2024-09-13T18:53:21.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7fef59e700 1 -- 172.21.15.7:0/1732873978 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe8021210 con 0x7f7fe80e0760 2024-09-13T18:53:21.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.776+0000 7f7fe6ffd700 1 -- 172.21.15.7:0/1732873978 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0068450 con 0x7f7fe80e0760 2024-09-13T18:53:21.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.780+0000 7f7fe6ffd700 1 -- 172.21.15.7:0/1732873978 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7ff00685b0 con 0x7f7fe80e0760 2024-09-13T18:53:21.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.780+0000 7f7fef59e700 1 -- 172.21.15.7:0/1732873978 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe80216f0 con 0x7f7fe80e0760 2024-09-13T18:53:21.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.780+0000 7f7fe6ffd700 1 --2- 172.21.15.7:0/1732873978 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7fd0066730 0x7f7fd0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:21.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.780+0000 7f7fe6ffd700 1 -- 172.21.15.7:0/1732873978 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ff00ff810 con 0x7f7fe80e0760 2024-09-13T18:53:21.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.780+0000 7f7fee59c700 1 --2- 172.21.15.7:0/1732873978 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7fd0066730 0x7f7fd0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:21.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.780+0000 7f7fee59c700 1 --2- 172.21.15.7:0/1732873978 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7fd0066730 0x7f7fd0068bf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0005250 tx=0x7f7fe0003900 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.792+0000 7f7fe6ffd700 1 -- 172.21.15.7:0/1732873978 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7ff00ccb90 con 0x7f7fe80e0760 2024-09-13T18:53:21.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.960+0000 7f7fef59e700 1 -- 172.21.15.7:0/1732873978 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7fe801d010 con 0x7f7fd0066730 2024-09-13T18:53:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.976+0000 7f7fe6ffd700 1 -- 172.21.15.7:0/1732873978 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+272861 (secure 0 0 0) 0x7f7fe0003bb0 con 0x7f7fd0066730 2024-09-13T18:53:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:21.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.980+0000 7f7fef59e700 1 -- 172.21.15.7:0/1732873978 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7fd0066730 msgr2=0x7f7fd0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.980+0000 7f7fef59e700 1 --2- 172.21.15.7:0/1732873978 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7fd0066730 0x7f7fd0068bf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0005250 tx=0x7f7fe0003900 comp rx=0 tx=0).stop 2024-09-13T18:53:21.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.980+0000 7f7fef59e700 1 -- 172.21.15.7:0/1732873978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7fe80e0760 msgr2=0x7f7fe804e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:21.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.980+0000 7f7fef59e700 1 --2- 172.21.15.7:0/1732873978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7fe80e0760 0x7f7fe804e240 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7ff004ff70 tx=0x7f7ff0074450 comp rx=0 tx=0).stop 2024-09-13T18:53:21.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.980+0000 7f7fef59e700 1 -- 172.21.15.7:0/1732873978 shutdown_connections 2024-09-13T18:53:21.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.980+0000 7f7fef59e700 1 --2- 172.21.15.7:0/1732873978 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7fd0066730 0x7f7fd0068bf0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.980+0000 7f7fef59e700 1 --2- 172.21.15.7:0/1732873978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7fe80e0760 0x7f7fe804e240 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.980+0000 7f7fef59e700 1 --2- 172.21.15.7:0/1732873978 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7fe800a5b0 0x7f7fe80dfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.980+0000 7f7fef59e700 1 --2- 172.21.15.7:0/1732873978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fe8006c80 0x7f7fe8021a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:21.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.980+0000 7f7fef59e700 1 -- 172.21.15.7:0/1732873978 >> 172.21.15.7:0/1732873978 conn(0x7f7fe8036140 msgr2=0x7f7fe80340b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:21.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.980+0000 7f7fef59e700 1 -- 172.21.15.7:0/1732873978 shutdown_connections 2024-09-13T18:53:21.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:21.980+0000 7f7fef59e700 1 -- 172.21.15.7:0/1732873978 wait complete. 2024-09-13T18:53:21.990 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-13T18:53:22.002 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-13T18:53:20.232191+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":244732,"kb_used_data":3944,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754729988,"statfs":{"total":773094113280,"available":772843507712,"internally_reserved":0,"allocated":4038656,"data_stored":2257632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":246402617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001072"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:17.504762+0000","last_change":"2024-09-13T18:53:17.504762+0000","last_active":"2024-09-13T18:53:17.504762+0000","last_peered":"2024-09-13T18:53:17.504762+0000","last_clean":"2024-09-13T18:53:17.504762+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:17.504762+0000","last_undegraded":"2024-09-13T18:53:17.504762+0000","last_fullsized":"2024-09-13T18:53:17.504762+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T18:53:16.495033+0000","last_change":"2024-09-13T18:53:16.495033+0000","last_active":"2024-09-13T18:53:16.495033+0000","last_peered":"2024-09-13T18:53:16.495033+0000","last_clean":"2024-09-13T18:53:16.495033+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T18:53:16.495033+0000","last_undegraded":"2024-09-13T18:53:16.495033+0000","last_fullsized":"2024-09-13T18:53:16.495033+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":169,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T18:53:11.123723+0000","last_change":"2024-09-13T18:52:15.080080+0000","last_active":"2024-09-13T18:53:11.123723+0000","last_peered":"2024-09-13T18:53:11.123723+0000","last_clean":"2024-09-13T18:53:11.123723+0000","last_became_active":"2024-09-13T18:52:15.078100+0000","last_became_peered":"2024-09-13T18:52:15.078100+0000","last_unstale":"2024-09-13T18:53:11.123723+0000","last_undegraded":"2024-09-13T18:53:11.123723+0000","last_fullsized":"2024-09-13T18:53:11.123723+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_clean_scrub_stamp":"2024-09-13T18:52:13.196473+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:29:59.452757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":950272,"data_stored":724980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":1.018},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.81100000000000005}]},{"osd":1,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":1.181},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.70399999999999996}]},{"osd":2,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.98099999999999998}]},{"osd":3,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.94499999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.872},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.89100000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.0800000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.139},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.1140000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":134612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.2949999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":1.1799999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.26}]},{"osd":3,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":1.0620000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":1.022}]},{"osd":5,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":1.139}]},{"osd":7,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.0859999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.3340000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":134612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.3300000000000001},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.98699999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.0169999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.753},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.1359999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.83699999999999997}]},{"osd":4,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.254},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.1060000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.403}]},{"osd":7,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":3.4369999999999998,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":1.075}]}]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31472,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":356352,"data_stored":134612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.54900000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":1.127},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.76700000000000002}]},{"osd":2,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.032}]},{"osd":3,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.877}]},{"osd":5,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.177},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.98799999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.0840000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.103}]}]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31488,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":356352,"data_stored":134612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.46800000000000003}]},{"osd":1,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.61599999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.375},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.83499999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.879}]},{"osd":6,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.71899999999999997}]},{"osd":7,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.89800000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35572,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94336268,"statfs":{"total":96636764160,"available":96600338432,"internally_reserved":0,"allocated":356352,"data_stored":134612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.442}]},{"osd":1,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.65100000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.83599999999999997}]},{"osd":4,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.125},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":1.1930000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.94699999999999995}]},{"osd":6,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":1.238}]},{"osd":7,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.016},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.1659999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31472,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":356352,"data_stored":134612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.786,"5min":2.786,"15min":2.786},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.57399999999999995}]},{"osd":2,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.113,"5min":3.113,"15min":3.113},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.056}]},{"osd":3,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.298},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.502}]},{"osd":4,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.1419999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.77200000000000002},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.4780000000000002,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":0.66400000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.22},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.89200000000000002}]},{"osd":7,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":1},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.4409999999999998,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.84499999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":950272,"data_stored":724980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.93300000000000005}]},{"osd":2,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.1200000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.53700000000000003}]},{"osd":4,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":1.323},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.105}]},{"osd":5,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.2370000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":1.181},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.88600000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.8340000000000001},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.0349999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:53:22.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:53:22.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.116+0000 7f3c391f3700 1 Processor -- start 2024-09-13T18:53:22.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.116+0000 7f3c391f3700 1 -- start start 2024-09-13T18:53:22.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.116+0000 7f3c391f3700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c340b51d0 0x7f3c3415ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:22.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.116+0000 7f3c391f3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c340b5680 con 0x7f3c340cf280 2024-09-13T18:53:22.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.116+0000 7f3c391f3700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c3415af70 con 0x7f3c340722c0 2024-09-13T18:53:22.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.116+0000 7f3c391f3700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3415f4b0 con 0x7f3c340b51d0 2024-09-13T18:53:22.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.116+0000 7f3c33fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3c340cf280 0x7f3c34071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39202/0 (socket says 172.21.15.7:39202) 2024-09-13T18:53:22.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.116+0000 7f3c33fff700 1 -- 172.21.15.7:0/2869405593 learned_addr learned my addr 172.21.15.7:0/2869405593 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:22.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.116+0000 7f3c389f2700 1 --2- 172.21.15.7:0/2869405593 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c340b51d0 0x7f3c3415ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c32ffd700 1 -- 172.21.15.7:0/2869405593 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428650902 0 0) 0x7f3c340b5680 con 0x7f3c340cf280 2024-09-13T18:53:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c32ffd700 1 -- 172.21.15.7:0/2869405593 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c140035d0 con 0x7f3c340cf280 2024-09-13T18:53:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c32ffd700 1 -- 172.21.15.7:0/2869405593 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992769772 0 0) 0x7f3c3415af70 con 0x7f3c340722c0 2024-09-13T18:53:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c389f2700 1 -- 172.21.15.7:0/2869405593 >> v1:172.21.15.171:6789/0 conn(0x7f3c340722c0 legacy=0x7f3c340cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c389f2700 1 -- 172.21.15.7:0/2869405593 >> v1:172.21.15.7:6789/0 conn(0x7f3c340cf280 legacy=0x7f3c34071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c389f2700 1 -- 172.21.15.7:0/2869405593 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c340b58d0 con 0x7f3c340b51d0 2024-09-13T18:53:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c389f2700 1 --2- 172.21.15.7:0/2869405593 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c340b51d0 0x7f3c3415ef70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009dc0 tx=0x7f3c2c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a991b50313c196fe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c32ffd700 1 -- 172.21.15.7:0/2869405593 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c00e070 con 0x7f3c340b51d0 2024-09-13T18:53:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c32ffd700 1 -- 172.21.15.7:0/2869405593 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c00bae0 con 0x7f3c340b51d0 2024-09-13T18:53:22.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c32ffd700 1 -- 172.21.15.7:0/2869405593 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c00be90 con 0x7f3c340b51d0 2024-09-13T18:53:22.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c391f3700 1 -- 172.21.15.7:0/2869405593 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c340b51d0 msgr2=0x7f3c3415ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:22.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c391f3700 1 --2- 172.21.15.7:0/2869405593 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c340b51d0 0x7f3c3415ef70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009dc0 tx=0x7f3c2c00b330 comp rx=0 tx=0).stop 2024-09-13T18:53:22.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c391f3700 1 -- 172.21.15.7:0/2869405593 shutdown_connections 2024-09-13T18:53:22.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c391f3700 1 --2- 172.21.15.7:0/2869405593 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c340b51d0 0x7f3c3415ef70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c391f3700 1 -- 172.21.15.7:0/2869405593 >> 172.21.15.7:0/2869405593 conn(0x7f3c340cf910 msgr2=0x7f3c340cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c391f3700 1 -- 172.21.15.7:0/2869405593 shutdown_connections 2024-09-13T18:53:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c391f3700 1 -- 172.21.15.7:0/2869405593 wait complete. 2024-09-13T18:53:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c391f3700 1 Processor -- start 2024-09-13T18:53:22.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.120+0000 7f3c391f3700 1 -- start start 2024-09-13T18:53:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c391f3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c340722c0 0x7f3c341f1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c391f3700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c340b51d0 0x7f3c3414e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c391f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c340cf280 0x7f3c34150050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c391f3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c341f18a0 con 0x7f3c340cf280 2024-09-13T18:53:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c391f3700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c341f0f40 con 0x7f3c340722c0 2024-09-13T18:53:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c391f3700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c34153be0 con 0x7f3c340b51d0 2024-09-13T18:53:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c389f2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c340cf280 0x7f3c34150050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c389f2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c340cf280 0x7f3c34150050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53260/0 (socket says 172.21.15.7:53260) 2024-09-13T18:53:22.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c389f2700 1 -- 172.21.15.7:0/3444815634 learned_addr learned my addr 172.21.15.7:0/3444815634 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:22.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c337fe700 1 --2- 172.21.15.7:0/3444815634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c340b51d0 0x7f3c3414e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:22.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c33fff700 1 --2- 172.21.15.7:0/3444815634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c340722c0 0x7f3c341f1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:22.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c337fe700 1 -- 172.21.15.7:0/3444815634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c340722c0 msgr2=0x7f3c341f1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:22.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c337fe700 1 --2- 172.21.15.7:0/3444815634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c340722c0 0x7f3c341f1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c337fe700 1 -- 172.21.15.7:0/3444815634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c340cf280 msgr2=0x7f3c34150050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:22.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c337fe700 1 --2- 172.21.15.7:0/3444815634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c340cf280 0x7f3c34150050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c337fe700 1 -- 172.21.15.7:0/3444815634 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c009a70 con 0x7f3c340b51d0 2024-09-13T18:53:22.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c389f2700 1 --2- 172.21.15.7:0/3444815634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c340cf280 0x7f3c34150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:22.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c337fe700 1 --2- 172.21.15.7:0/3444815634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c340b51d0 0x7f3c3414e830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3c340d0780 tx=0x7f3c1c00c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:22.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c30ff9700 1 -- 172.21.15.7:0/3444815634 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c0054e0 con 0x7f3c340b51d0 2024-09-13T18:53:22.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c30ff9700 1 -- 172.21.15.7:0/3444815634 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1c00ce90 con 0x7f3c340b51d0 2024-09-13T18:53:22.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c30ff9700 1 -- 172.21.15.7:0/3444815634 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c0038e0 con 0x7f3c340b51d0 2024-09-13T18:53:22.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c391f3700 1 -- 172.21.15.7:0/3444815634 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c34153ec0 con 0x7f3c340b51d0 2024-09-13T18:53:22.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.124+0000 7f3c391f3700 1 -- 172.21.15.7:0/3444815634 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3414ed70 con 0x7f3c340b51d0 2024-09-13T18:53:22.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.128+0000 7f3c391f3700 1 -- 172.21.15.7:0/3444815634 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c341543e0 con 0x7f3c340b51d0 2024-09-13T18:53:22.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.128+0000 7f3c30ff9700 1 -- 172.21.15.7:0/3444815634 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3c1c003a40 con 0x7f3c340b51d0 2024-09-13T18:53:22.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.128+0000 7f3c30ff9700 1 --2- 172.21.15.7:0/3444815634 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3c18066730 0x7f3c18068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:22.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.128+0000 7f3c30ff9700 1 -- 172.21.15.7:0/3444815634 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c1c083620 con 0x7f3c340b51d0 2024-09-13T18:53:22.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.128+0000 7f3c33fff700 1 --2- 172.21.15.7:0/3444815634 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3c18066730 0x7f3c18068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:22.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.128+0000 7f3c33fff700 1 --2- 172.21.15.7:0/3444815634 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3c18066730 0x7f3c18068bf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3c24007c60 tx=0x7f3c24007bb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:22.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.140+0000 7f3c30ff9700 1 -- 172.21.15.7:0/3444815634 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3c341543e0 con 0x7f3c340b51d0 2024-09-13T18:53:22.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.312+0000 7f3c391f3700 1 -- 172.21.15.7:0/3444815634 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3c34153a60 con 0x7f3c18066730 2024-09-13T18:53:22.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.328+0000 7f3c30ff9700 1 -- 172.21.15.7:0/3444815634 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287790 (secure 0 0 0) 0x7f3c240012a0 con 0x7f3c18066730 2024-09-13T18:53:22.341 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:22.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.336+0000 7f3c391f3700 1 -- 172.21.15.7:0/3444815634 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3c18066730 msgr2=0x7f3c18068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:22.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.336+0000 7f3c391f3700 1 --2- 172.21.15.7:0/3444815634 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3c18066730 0x7f3c18068bf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3c24007c60 tx=0x7f3c24007bb0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.336+0000 7f3c391f3700 1 -- 172.21.15.7:0/3444815634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c340b51d0 msgr2=0x7f3c3414e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:22.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.336+0000 7f3c391f3700 1 --2- 172.21.15.7:0/3444815634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c340b51d0 0x7f3c3414e830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3c340d0780 tx=0x7f3c1c00c790 comp rx=0 tx=0).stop 2024-09-13T18:53:22.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.336+0000 7f3c391f3700 1 -- 172.21.15.7:0/3444815634 shutdown_connections 2024-09-13T18:53:22.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.336+0000 7f3c391f3700 1 --2- 172.21.15.7:0/3444815634 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3c18066730 0x7f3c18068bf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.336+0000 7f3c391f3700 1 --2- 172.21.15.7:0/3444815634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c340cf280 0x7f3c34150050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.336+0000 7f3c391f3700 1 --2- 172.21.15.7:0/3444815634 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3c340b51d0 0x7f3c3414e830 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.336+0000 7f3c391f3700 1 --2- 172.21.15.7:0/3444815634 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c340722c0 0x7f3c341f1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.336+0000 7f3c391f3700 1 -- 172.21.15.7:0/3444815634 >> 172.21.15.7:0/3444815634 conn(0x7f3c340cf910 msgr2=0x7f3c340b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:22.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.336+0000 7f3c391f3700 1 -- 172.21.15.7:0/3444815634 shutdown_connections 2024-09-13T18:53:22.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.336+0000 7f3c391f3700 1 -- 172.21.15.7:0/3444815634 wait complete. 2024-09-13T18:53:22.344 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-13T18:53:22.358 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-09-13T18:53:22.232751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245168,"kb_used_data":4392,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754729552,"statfs":{"total":773094113280,"available":772843061248,"internally_reserved":0,"allocated":4497408,"data_stored":2463628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":246402617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.724894"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.912264+0000","last_change":"2024-09-13T18:53:17.935056+0000","last_active":"2024-09-13T18:53:18.912264+0000","last_peered":"2024-09-13T18:53:18.912264+0000","last_clean":"2024-09-13T18:53:18.912264+0000","last_became_active":"2024-09-13T18:53:17.934812+0000","last_became_peered":"2024-09-13T18:53:17.934812+0000","last_unstale":"2024-09-13T18:53:18.912264+0000","last_undegraded":"2024-09-13T18:53:18.912264+0000","last_fullsized":"2024-09-13T18:53:18.912264+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:55:23.108275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.908005+0000","last_change":"2024-09-13T18:53:17.927559+0000","last_active":"2024-09-13T18:53:18.908005+0000","last_peered":"2024-09-13T18:53:18.908005+0000","last_clean":"2024-09-13T18:53:18.908005+0000","last_became_active":"2024-09-13T18:53:17.925765+0000","last_became_peered":"2024-09-13T18:53:17.925765+0000","last_unstale":"2024-09-13T18:53:18.908005+0000","last_undegraded":"2024-09-13T18:53:18.908005+0000","last_fullsized":"2024-09-13T18:53:18.908005+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:24:01.478975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.054793+0000","last_change":"2024-09-13T18:53:17.931565+0000","last_active":"2024-09-13T18:53:19.054793+0000","last_peered":"2024-09-13T18:53:19.054793+0000","last_clean":"2024-09-13T18:53:19.054793+0000","last_became_active":"2024-09-13T18:53:17.931240+0000","last_became_peered":"2024-09-13T18:53:17.931240+0000","last_unstale":"2024-09-13T18:53:19.054793+0000","last_undegraded":"2024-09-13T18:53:19.054793+0000","last_fullsized":"2024-09-13T18:53:19.054793+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:04.777238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.051585+0000","last_change":"2024-09-13T18:53:17.931090+0000","last_active":"2024-09-13T18:53:19.051585+0000","last_peered":"2024-09-13T18:53:19.051585+0000","last_clean":"2024-09-13T18:53:19.051585+0000","last_became_active":"2024-09-13T18:53:17.930670+0000","last_became_peered":"2024-09-13T18:53:17.930670+0000","last_unstale":"2024-09-13T18:53:19.051585+0000","last_undegraded":"2024-09-13T18:53:19.051585+0000","last_fullsized":"2024-09-13T18:53:19.051585+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:26:07.568918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.922240+0000","last_change":"2024-09-13T18:53:17.934897+0000","last_active":"2024-09-13T18:53:18.922240+0000","last_peered":"2024-09-13T18:53:18.922240+0000","last_clean":"2024-09-13T18:53:18.922240+0000","last_became_active":"2024-09-13T18:53:17.934616+0000","last_became_peered":"2024-09-13T18:53:17.934616+0000","last_unstale":"2024-09-13T18:53:18.922240+0000","last_undegraded":"2024-09-13T18:53:18.922240+0000","last_fullsized":"2024-09-13T18:53:18.922240+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:03:53.287054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.907041+0000","last_change":"2024-09-13T18:53:17.933854+0000","last_active":"2024-09-13T18:53:18.907041+0000","last_peered":"2024-09-13T18:53:18.907041+0000","last_clean":"2024-09-13T18:53:18.907041+0000","last_became_active":"2024-09-13T18:53:17.933624+0000","last_became_peered":"2024-09-13T18:53:17.933624+0000","last_unstale":"2024-09-13T18:53:18.907041+0000","last_undegraded":"2024-09-13T18:53:18.907041+0000","last_fullsized":"2024-09-13T18:53:18.907041+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:05.749061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.053363+0000","last_change":"2024-09-13T18:53:17.924709+0000","last_active":"2024-09-13T18:53:19.053363+0000","last_peered":"2024-09-13T18:53:19.053363+0000","last_clean":"2024-09-13T18:53:19.053363+0000","last_became_active":"2024-09-13T18:53:17.924423+0000","last_became_peered":"2024-09-13T18:53:17.924423+0000","last_unstale":"2024-09-13T18:53:19.053363+0000","last_undegraded":"2024-09-13T18:53:19.053363+0000","last_fullsized":"2024-09-13T18:53:19.053363+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:04.777238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.052988+0000","last_change":"2024-09-13T18:53:17.922515+0000","last_active":"2024-09-13T18:53:19.052988+0000","last_peered":"2024-09-13T18:53:19.052988+0000","last_clean":"2024-09-13T18:53:19.052988+0000","last_became_active":"2024-09-13T18:53:17.922151+0000","last_became_peered":"2024-09-13T18:53:17.922151+0000","last_unstale":"2024-09-13T18:53:19.052988+0000","last_undegraded":"2024-09-13T18:53:19.052988+0000","last_fullsized":"2024-09-13T18:53:19.052988+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:55:41.992976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.914035+0000","last_change":"2024-09-13T18:53:17.932119+0000","last_active":"2024-09-13T18:53:18.914035+0000","last_peered":"2024-09-13T18:53:18.914035+0000","last_clean":"2024-09-13T18:53:18.914035+0000","last_became_active":"2024-09-13T18:53:17.931920+0000","last_became_peered":"2024-09-13T18:53:17.931920+0000","last_unstale":"2024-09-13T18:53:18.914035+0000","last_undegraded":"2024-09-13T18:53:18.914035+0000","last_fullsized":"2024-09-13T18:53:18.914035+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:05.005584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.051592+0000","last_change":"2024-09-13T18:53:17.929223+0000","last_active":"2024-09-13T18:53:19.051592+0000","last_peered":"2024-09-13T18:53:19.051592+0000","last_clean":"2024-09-13T18:53:19.051592+0000","last_became_active":"2024-09-13T18:53:17.928997+0000","last_became_peered":"2024-09-13T18:53:17.928997+0000","last_unstale":"2024-09-13T18:53:19.051592+0000","last_undegraded":"2024-09-13T18:53:19.051592+0000","last_fullsized":"2024-09-13T18:53:19.051592+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:55:23.108275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.053892+0000","last_change":"2024-09-13T18:53:17.930913+0000","last_active":"2024-09-13T18:53:19.053892+0000","last_peered":"2024-09-13T18:53:19.053892+0000","last_clean":"2024-09-13T18:53:19.053892+0000","last_became_active":"2024-09-13T18:53:17.930689+0000","last_became_peered":"2024-09-13T18:53:17.930689+0000","last_unstale":"2024-09-13T18:53:19.053892+0000","last_undegraded":"2024-09-13T18:53:19.053892+0000","last_fullsized":"2024-09-13T18:53:19.053892+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:13.134078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.054564+0000","last_change":"2024-09-13T18:53:17.941290+0000","last_active":"2024-09-13T18:53:19.054564+0000","last_peered":"2024-09-13T18:53:19.054564+0000","last_clean":"2024-09-13T18:53:19.054564+0000","last_became_active":"2024-09-13T18:53:17.940985+0000","last_became_peered":"2024-09-13T18:53:17.940985+0000","last_unstale":"2024-09-13T18:53:19.054564+0000","last_undegraded":"2024-09-13T18:53:19.054564+0000","last_fullsized":"2024-09-13T18:53:19.054564+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:26.505061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.922423+0000","last_change":"2024-09-13T18:53:17.929494+0000","last_active":"2024-09-13T18:53:18.922423+0000","last_peered":"2024-09-13T18:53:18.922423+0000","last_clean":"2024-09-13T18:53:18.922423+0000","last_became_active":"2024-09-13T18:53:17.929235+0000","last_became_peered":"2024-09-13T18:53:17.929235+0000","last_unstale":"2024-09-13T18:53:18.922423+0000","last_undegraded":"2024-09-13T18:53:18.922423+0000","last_fullsized":"2024-09-13T18:53:18.922423+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:08.892934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.907147+0000","last_change":"2024-09-13T18:53:17.926420+0000","last_active":"2024-09-13T18:53:18.907147+0000","last_peered":"2024-09-13T18:53:18.907147+0000","last_clean":"2024-09-13T18:53:18.907147+0000","last_became_active":"2024-09-13T18:53:17.925186+0000","last_became_peered":"2024-09-13T18:53:17.925186+0000","last_unstale":"2024-09-13T18:53:18.907147+0000","last_undegraded":"2024-09-13T18:53:18.907147+0000","last_fullsized":"2024-09-13T18:53:18.907147+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:29.474703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.054487+0000","last_change":"2024-09-13T18:53:17.934631+0000","last_active":"2024-09-13T18:53:19.054487+0000","last_peered":"2024-09-13T18:53:19.054487+0000","last_clean":"2024-09-13T18:53:19.054487+0000","last_became_active":"2024-09-13T18:53:17.934404+0000","last_became_peered":"2024-09-13T18:53:17.934404+0000","last_unstale":"2024-09-13T18:53:19.054487+0000","last_undegraded":"2024-09-13T18:53:19.054487+0000","last_fullsized":"2024-09-13T18:53:19.054487+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:29.474703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.053487+0000","last_change":"2024-09-13T18:53:17.941485+0000","last_active":"2024-09-13T18:53:19.053487+0000","last_peered":"2024-09-13T18:53:19.053487+0000","last_clean":"2024-09-13T18:53:19.053487+0000","last_became_active":"2024-09-13T18:53:17.941196+0000","last_became_peered":"2024-09-13T18:53:17.941196+0000","last_unstale":"2024-09-13T18:53:19.053487+0000","last_undegraded":"2024-09-13T18:53:19.053487+0000","last_fullsized":"2024-09-13T18:53:19.053487+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:26:07.568918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.053886+0000","last_change":"2024-09-13T18:53:17.921434+0000","last_active":"2024-09-13T18:53:19.053886+0000","last_peered":"2024-09-13T18:53:19.053886+0000","last_clean":"2024-09-13T18:53:19.053886+0000","last_became_active":"2024-09-13T18:53:17.920741+0000","last_became_peered":"2024-09-13T18:53:17.920741+0000","last_unstale":"2024-09-13T18:53:19.053886+0000","last_undegraded":"2024-09-13T18:53:19.053886+0000","last_fullsized":"2024-09-13T18:53:19.053886+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:51:06.500427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.907993+0000","last_change":"2024-09-13T18:53:17.928048+0000","last_active":"2024-09-13T18:53:18.907993+0000","last_peered":"2024-09-13T18:53:18.907993+0000","last_clean":"2024-09-13T18:53:18.907993+0000","last_became_active":"2024-09-13T18:53:17.927753+0000","last_became_peered":"2024-09-13T18:53:17.927753+0000","last_unstale":"2024-09-13T18:53:18.907993+0000","last_undegraded":"2024-09-13T18:53:18.907993+0000","last_fullsized":"2024-09-13T18:53:18.907993+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:25:06.778238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.907591+0000","last_change":"2024-09-13T18:53:17.933354+0000","last_active":"2024-09-13T18:53:18.907591+0000","last_peered":"2024-09-13T18:53:18.907591+0000","last_clean":"2024-09-13T18:53:18.907591+0000","last_became_active":"2024-09-13T18:53:17.932729+0000","last_became_peered":"2024-09-13T18:53:17.932729+0000","last_unstale":"2024-09-13T18:53:18.907591+0000","last_undegraded":"2024-09-13T18:53:18.907591+0000","last_fullsized":"2024-09-13T18:53:18.907591+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:55:41.992976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.922371+0000","last_change":"2024-09-13T18:53:17.933855+0000","last_active":"2024-09-13T18:53:18.922371+0000","last_peered":"2024-09-13T18:53:18.922371+0000","last_clean":"2024-09-13T18:53:18.922371+0000","last_became_active":"2024-09-13T18:53:17.933580+0000","last_became_peered":"2024-09-13T18:53:17.933580+0000","last_unstale":"2024-09-13T18:53:18.922371+0000","last_undegraded":"2024-09-13T18:53:18.922371+0000","last_fullsized":"2024-09-13T18:53:18.922371+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:11:29.249572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.912391+0000","last_change":"2024-09-13T18:53:17.926148+0000","last_active":"2024-09-13T18:53:18.912391+0000","last_peered":"2024-09-13T18:53:18.912391+0000","last_clean":"2024-09-13T18:53:18.912391+0000","last_became_active":"2024-09-13T18:53:17.925801+0000","last_became_peered":"2024-09-13T18:53:17.925801+0000","last_unstale":"2024-09-13T18:53:18.912391+0000","last_undegraded":"2024-09-13T18:53:18.912391+0000","last_fullsized":"2024-09-13T18:53:18.912391+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:58:26.836926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.912468+0000","last_change":"2024-09-13T18:53:17.920548+0000","last_active":"2024-09-13T18:53:18.912468+0000","last_peered":"2024-09-13T18:53:18.912468+0000","last_clean":"2024-09-13T18:53:18.912468+0000","last_became_active":"2024-09-13T18:53:17.920168+0000","last_became_peered":"2024-09-13T18:53:17.920168+0000","last_unstale":"2024-09-13T18:53:18.912468+0000","last_undegraded":"2024-09-13T18:53:18.912468+0000","last_fullsized":"2024-09-13T18:53:18.912468+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:10:46.001163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.052847+0000","last_change":"2024-09-13T18:53:17.921147+0000","last_active":"2024-09-13T18:53:19.052847+0000","last_peered":"2024-09-13T18:53:19.052847+0000","last_clean":"2024-09-13T18:53:19.052847+0000","last_became_active":"2024-09-13T18:53:17.920820+0000","last_became_peered":"2024-09-13T18:53:17.920820+0000","last_unstale":"2024-09-13T18:53:19.052847+0000","last_undegraded":"2024-09-13T18:53:19.052847+0000","last_fullsized":"2024-09-13T18:53:19.052847+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:27.294047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.053755+0000","last_change":"2024-09-13T18:53:17.931395+0000","last_active":"2024-09-13T18:53:19.053755+0000","last_peered":"2024-09-13T18:53:19.053755+0000","last_clean":"2024-09-13T18:53:19.053755+0000","last_became_active":"2024-09-13T18:53:17.931107+0000","last_became_peered":"2024-09-13T18:53:17.931107+0000","last_unstale":"2024-09-13T18:53:19.053755+0000","last_undegraded":"2024-09-13T18:53:19.053755+0000","last_fullsized":"2024-09-13T18:53:19.053755+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:33.536636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.051875+0000","last_change":"2024-09-13T18:53:17.925596+0000","last_active":"2024-09-13T18:53:19.051875+0000","last_peered":"2024-09-13T18:53:19.051875+0000","last_clean":"2024-09-13T18:53:19.051875+0000","last_became_active":"2024-09-13T18:53:17.925356+0000","last_became_peered":"2024-09-13T18:53:17.925356+0000","last_unstale":"2024-09-13T18:53:19.051875+0000","last_undegraded":"2024-09-13T18:53:19.051875+0000","last_fullsized":"2024-09-13T18:53:19.051875+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:52:35.563770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.908206+0000","last_change":"2024-09-13T18:53:17.911975+0000","last_active":"2024-09-13T18:53:18.908206+0000","last_peered":"2024-09-13T18:53:18.908206+0000","last_clean":"2024-09-13T18:53:18.908206+0000","last_became_active":"2024-09-13T18:53:17.911673+0000","last_became_peered":"2024-09-13T18:53:17.911673+0000","last_unstale":"2024-09-13T18:53:18.908206+0000","last_undegraded":"2024-09-13T18:53:18.908206+0000","last_fullsized":"2024-09-13T18:53:18.908206+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:04.777238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.051815+0000","last_change":"2024-09-13T18:53:17.935114+0000","last_active":"2024-09-13T18:53:19.051815+0000","last_peered":"2024-09-13T18:53:19.051815+0000","last_clean":"2024-09-13T18:53:19.051815+0000","last_became_active":"2024-09-13T18:53:17.934743+0000","last_became_peered":"2024-09-13T18:53:17.934743+0000","last_unstale":"2024-09-13T18:53:19.051815+0000","last_undegraded":"2024-09-13T18:53:19.051815+0000","last_fullsized":"2024-09-13T18:53:19.051815+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:05.108555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.054554+0000","last_change":"2024-09-13T18:53:17.933984+0000","last_active":"2024-09-13T18:53:19.054554+0000","last_peered":"2024-09-13T18:53:19.054554+0000","last_clean":"2024-09-13T18:53:19.054554+0000","last_became_active":"2024-09-13T18:53:17.933373+0000","last_became_peered":"2024-09-13T18:53:17.933373+0000","last_unstale":"2024-09-13T18:53:19.054554+0000","last_undegraded":"2024-09-13T18:53:19.054554+0000","last_fullsized":"2024-09-13T18:53:19.054554+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:55:41.992976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.912679+0000","last_change":"2024-09-13T18:53:17.938852+0000","last_active":"2024-09-13T18:53:18.912679+0000","last_peered":"2024-09-13T18:53:18.912679+0000","last_clean":"2024-09-13T18:53:18.912679+0000","last_became_active":"2024-09-13T18:53:17.938559+0000","last_became_peered":"2024-09-13T18:53:17.938559+0000","last_unstale":"2024-09-13T18:53:18.912679+0000","last_undegraded":"2024-09-13T18:53:18.912679+0000","last_fullsized":"2024-09-13T18:53:18.912679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:08.892934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.053664+0000","last_change":"2024-09-13T18:53:17.922420+0000","last_active":"2024-09-13T18:53:19.053664+0000","last_peered":"2024-09-13T18:53:19.053664+0000","last_clean":"2024-09-13T18:53:19.053664+0000","last_became_active":"2024-09-13T18:53:17.922144+0000","last_became_peered":"2024-09-13T18:53:17.922144+0000","last_unstale":"2024-09-13T18:53:19.053664+0000","last_undegraded":"2024-09-13T18:53:19.053664+0000","last_fullsized":"2024-09-13T18:53:19.053664+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:09.098069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.911526+0000","last_change":"2024-09-13T18:53:17.922689+0000","last_active":"2024-09-13T18:53:18.911526+0000","last_peered":"2024-09-13T18:53:18.911526+0000","last_clean":"2024-09-13T18:53:18.911526+0000","last_became_active":"2024-09-13T18:53:17.921707+0000","last_became_peered":"2024-09-13T18:53:17.921707+0000","last_unstale":"2024-09-13T18:53:18.911526+0000","last_undegraded":"2024-09-13T18:53:18.911526+0000","last_fullsized":"2024-09-13T18:53:18.911526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:01.931243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.053201+0000","last_change":"2024-09-13T18:53:17.930041+0000","last_active":"2024-09-13T18:53:19.053201+0000","last_peered":"2024-09-13T18:53:19.053201+0000","last_clean":"2024-09-13T18:53:19.053201+0000","last_became_active":"2024-09-13T18:53:17.929716+0000","last_became_peered":"2024-09-13T18:53:17.929716+0000","last_unstale":"2024-09-13T18:53:19.053201+0000","last_undegraded":"2024-09-13T18:53:19.053201+0000","last_fullsized":"2024-09-13T18:53:19.053201+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:18:29.474703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.053260+0000","last_change":"2024-09-13T18:53:17.921349+0000","last_active":"2024-09-13T18:53:19.053260+0000","last_peered":"2024-09-13T18:53:19.053260+0000","last_clean":"2024-09-13T18:53:19.053260+0000","last_became_active":"2024-09-13T18:53:17.921026+0000","last_became_peered":"2024-09-13T18:53:17.921026+0000","last_unstale":"2024-09-13T18:53:19.053260+0000","last_undegraded":"2024-09-13T18:53:19.053260+0000","last_fullsized":"2024-09-13T18:53:19.053260+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:49:29.274291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.912680+0000","last_change":"2024-09-13T18:53:17.538604+0000","last_active":"2024-09-13T18:53:18.912680+0000","last_peered":"2024-09-13T18:53:18.912680+0000","last_clean":"2024-09-13T18:53:18.912680+0000","last_became_active":"2024-09-13T18:53:17.538333+0000","last_became_peered":"2024-09-13T18:53:17.538333+0000","last_unstale":"2024-09-13T18:53:18.912680+0000","last_undegraded":"2024-09-13T18:53:18.912680+0000","last_fullsized":"2024-09-13T18:53:18.912680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:53:51.468967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.912477+0000","last_change":"2024-09-13T18:53:17.930597+0000","last_active":"2024-09-13T18:53:18.912477+0000","last_peered":"2024-09-13T18:53:18.912477+0000","last_clean":"2024-09-13T18:53:18.912477+0000","last_became_active":"2024-09-13T18:53:17.930280+0000","last_became_peered":"2024-09-13T18:53:17.930280+0000","last_unstale":"2024-09-13T18:53:18.912477+0000","last_undegraded":"2024-09-13T18:53:18.912477+0000","last_fullsized":"2024-09-13T18:53:18.912477+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:58.828340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.053140+0000","last_change":"2024-09-13T18:53:17.533575+0000","last_active":"2024-09-13T18:53:19.053140+0000","last_peered":"2024-09-13T18:53:19.053140+0000","last_clean":"2024-09-13T18:53:19.053140+0000","last_became_active":"2024-09-13T18:53:17.533161+0000","last_became_peered":"2024-09-13T18:53:17.533161+0000","last_unstale":"2024-09-13T18:53:19.053140+0000","last_undegraded":"2024-09-13T18:53:19.053140+0000","last_fullsized":"2024-09-13T18:53:19.053140+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:05.005778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.913145+0000","last_change":"2024-09-13T18:53:17.920310+0000","last_active":"2024-09-13T18:53:18.913145+0000","last_peered":"2024-09-13T18:53:18.913145+0000","last_clean":"2024-09-13T18:53:18.913145+0000","last_became_active":"2024-09-13T18:53:17.920069+0000","last_became_peered":"2024-09-13T18:53:17.920069+0000","last_unstale":"2024-09-13T18:53:18.913145+0000","last_undegraded":"2024-09-13T18:53:18.913145+0000","last_fullsized":"2024-09-13T18:53:18.913145+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:48:31.604140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.907834+0000","last_change":"2024-09-13T18:53:17.532776+0000","last_active":"2024-09-13T18:53:18.907834+0000","last_peered":"2024-09-13T18:53:18.907834+0000","last_clean":"2024-09-13T18:53:18.907834+0000","last_became_active":"2024-09-13T18:53:17.532253+0000","last_became_peered":"2024-09-13T18:53:17.532253+0000","last_unstale":"2024-09-13T18:53:18.907834+0000","last_undegraded":"2024-09-13T18:53:18.907834+0000","last_fullsized":"2024-09-13T18:53:18.907834+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:57.818611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.051929+0000","last_change":"2024-09-13T18:53:17.935598+0000","last_active":"2024-09-13T18:53:19.051929+0000","last_peered":"2024-09-13T18:53:19.051929+0000","last_clean":"2024-09-13T18:53:19.051929+0000","last_became_active":"2024-09-13T18:53:17.934714+0000","last_became_peered":"2024-09-13T18:53:17.934714+0000","last_unstale":"2024-09-13T18:53:19.051929+0000","last_undegraded":"2024-09-13T18:53:19.051929+0000","last_fullsized":"2024-09-13T18:53:19.051929+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:06:22.813859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.922412+0000","last_change":"2024-09-13T18:53:17.526733+0000","last_active":"2024-09-13T18:53:18.922412+0000","last_peered":"2024-09-13T18:53:18.922412+0000","last_clean":"2024-09-13T18:53:18.922412+0000","last_became_active":"2024-09-13T18:53:17.526338+0000","last_became_peered":"2024-09-13T18:53:17.526338+0000","last_unstale":"2024-09-13T18:53:18.922412+0000","last_undegraded":"2024-09-13T18:53:18.922412+0000","last_fullsized":"2024-09-13T18:53:18.922412+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:33:48.181275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.053763+0000","last_change":"2024-09-13T18:53:17.926440+0000","last_active":"2024-09-13T18:53:19.053763+0000","last_peered":"2024-09-13T18:53:19.053763+0000","last_clean":"2024-09-13T18:53:19.053763+0000","last_became_active":"2024-09-13T18:53:17.926196+0000","last_became_peered":"2024-09-13T18:53:17.926196+0000","last_unstale":"2024-09-13T18:53:19.053763+0000","last_undegraded":"2024-09-13T18:53:19.053763+0000","last_fullsized":"2024-09-13T18:53:19.053763+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:00:05.108555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.912930+0000","last_change":"2024-09-13T18:53:17.531822+0000","last_active":"2024-09-13T18:53:18.912930+0000","last_peered":"2024-09-13T18:53:18.912930+0000","last_clean":"2024-09-13T18:53:18.912930+0000","last_became_active":"2024-09-13T18:53:17.531481+0000","last_became_peered":"2024-09-13T18:53:17.531481+0000","last_unstale":"2024-09-13T18:53:18.912930+0000","last_undegraded":"2024-09-13T18:53:18.912930+0000","last_fullsized":"2024-09-13T18:53:18.912930+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:38:13.653646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.054559+0000","last_change":"2024-09-13T18:53:17.933736+0000","last_active":"2024-09-13T18:53:19.054559+0000","last_peered":"2024-09-13T18:53:19.054559+0000","last_clean":"2024-09-13T18:53:19.054559+0000","last_became_active":"2024-09-13T18:53:17.933437+0000","last_became_peered":"2024-09-13T18:53:17.933437+0000","last_unstale":"2024-09-13T18:53:19.054559+0000","last_undegraded":"2024-09-13T18:53:19.054559+0000","last_fullsized":"2024-09-13T18:53:19.054559+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:24:01.478975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.912572+0000","last_change":"2024-09-13T18:53:17.534273+0000","last_active":"2024-09-13T18:53:18.912572+0000","last_peered":"2024-09-13T18:53:18.912572+0000","last_clean":"2024-09-13T18:53:18.912572+0000","last_became_active":"2024-09-13T18:53:17.533990+0000","last_became_peered":"2024-09-13T18:53:17.533990+0000","last_unstale":"2024-09-13T18:53:18.912572+0000","last_undegraded":"2024-09-13T18:53:18.912572+0000","last_fullsized":"2024-09-13T18:53:18.912572+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:24:26.004399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.912469+0000","last_change":"2024-09-13T18:53:17.925405+0000","last_active":"2024-09-13T18:53:18.912469+0000","last_peered":"2024-09-13T18:53:18.912469+0000","last_clean":"2024-09-13T18:53:18.912469+0000","last_became_active":"2024-09-13T18:53:17.925049+0000","last_became_peered":"2024-09-13T18:53:17.925049+0000","last_unstale":"2024-09-13T18:53:18.912469+0000","last_undegraded":"2024-09-13T18:53:18.912469+0000","last_fullsized":"2024-09-13T18:53:18.912469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:03:53.287054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.906681+0000","last_change":"2024-09-13T18:53:17.522534+0000","last_active":"2024-09-13T18:53:18.906681+0000","last_peered":"2024-09-13T18:53:18.906681+0000","last_clean":"2024-09-13T18:53:18.906681+0000","last_became_active":"2024-09-13T18:53:17.522268+0000","last_became_peered":"2024-09-13T18:53:17.522268+0000","last_unstale":"2024-09-13T18:53:18.906681+0000","last_undegraded":"2024-09-13T18:53:18.906681+0000","last_fullsized":"2024-09-13T18:53:18.906681+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:25:44.941528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.050960+0000","last_change":"2024-09-13T18:53:17.933512+0000","last_active":"2024-09-13T18:53:19.050960+0000","last_peered":"2024-09-13T18:53:19.050960+0000","last_clean":"2024-09-13T18:53:19.050960+0000","last_became_active":"2024-09-13T18:53:17.933233+0000","last_became_peered":"2024-09-13T18:53:17.933233+0000","last_unstale":"2024-09-13T18:53:19.050960+0000","last_undegraded":"2024-09-13T18:53:19.050960+0000","last_fullsized":"2024-09-13T18:53:19.050960+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:03:10.996303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.912675+0000","last_change":"2024-09-13T18:53:17.532606+0000","last_active":"2024-09-13T18:53:18.912675+0000","last_peered":"2024-09-13T18:53:18.912675+0000","last_clean":"2024-09-13T18:53:18.912675+0000","last_became_active":"2024-09-13T18:53:17.532244+0000","last_became_peered":"2024-09-13T18:53:17.532244+0000","last_unstale":"2024-09-13T18:53:18.912675+0000","last_undegraded":"2024-09-13T18:53:18.912675+0000","last_fullsized":"2024-09-13T18:53:18.912675+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:19:35.651442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.055021+0000","last_change":"2024-09-13T18:53:17.935326+0000","last_active":"2024-09-13T18:53:19.055021+0000","last_peered":"2024-09-13T18:53:19.055021+0000","last_clean":"2024-09-13T18:53:19.055021+0000","last_became_active":"2024-09-13T18:53:17.935093+0000","last_became_peered":"2024-09-13T18:53:17.935093+0000","last_unstale":"2024-09-13T18:53:19.055021+0000","last_undegraded":"2024-09-13T18:53:19.055021+0000","last_fullsized":"2024-09-13T18:53:19.055021+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:21.219347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.907687+0000","last_change":"2024-09-13T18:53:17.527214+0000","last_active":"2024-09-13T18:53:18.907687+0000","last_peered":"2024-09-13T18:53:18.907687+0000","last_clean":"2024-09-13T18:53:18.907687+0000","last_became_active":"2024-09-13T18:53:17.526285+0000","last_became_peered":"2024-09-13T18:53:17.526285+0000","last_unstale":"2024-09-13T18:53:18.907687+0000","last_undegraded":"2024-09-13T18:53:18.907687+0000","last_fullsized":"2024-09-13T18:53:18.907687+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:08.624574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.907598+0000","last_change":"2024-09-13T18:53:17.928088+0000","last_active":"2024-09-13T18:53:18.907598+0000","last_peered":"2024-09-13T18:53:18.907598+0000","last_clean":"2024-09-13T18:53:18.907598+0000","last_became_active":"2024-09-13T18:53:17.927847+0000","last_became_peered":"2024-09-13T18:53:17.927847+0000","last_unstale":"2024-09-13T18:53:18.907598+0000","last_undegraded":"2024-09-13T18:53:18.907598+0000","last_fullsized":"2024-09-13T18:53:18.907598+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:23:26.505061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.053420+0000","last_change":"2024-09-13T18:53:17.534341+0000","last_active":"2024-09-13T18:53:19.053420+0000","last_peered":"2024-09-13T18:53:19.053420+0000","last_clean":"2024-09-13T18:53:19.053420+0000","last_became_active":"2024-09-13T18:53:17.533951+0000","last_became_peered":"2024-09-13T18:53:17.533951+0000","last_unstale":"2024-09-13T18:53:19.053420+0000","last_undegraded":"2024-09-13T18:53:19.053420+0000","last_fullsized":"2024-09-13T18:53:19.053420+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:14.518730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.907629+0000","last_change":"2024-09-13T18:53:17.928263+0000","last_active":"2024-09-13T18:53:18.907629+0000","last_peered":"2024-09-13T18:53:18.907629+0000","last_clean":"2024-09-13T18:53:18.907629+0000","last_became_active":"2024-09-13T18:53:17.926009+0000","last_became_peered":"2024-09-13T18:53:17.926009+0000","last_unstale":"2024-09-13T18:53:18.907629+0000","last_undegraded":"2024-09-13T18:53:18.907629+0000","last_fullsized":"2024-09-13T18:53:18.907629+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:37.175566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.054823+0000","last_change":"2024-09-13T18:53:17.534592+0000","last_active":"2024-09-13T18:53:19.054823+0000","last_peered":"2024-09-13T18:53:19.054823+0000","last_clean":"2024-09-13T18:53:19.054823+0000","last_became_active":"2024-09-13T18:53:17.533922+0000","last_became_peered":"2024-09-13T18:53:17.533922+0000","last_unstale":"2024-09-13T18:53:19.054823+0000","last_undegraded":"2024-09-13T18:53:19.054823+0000","last_fullsized":"2024-09-13T18:53:19.054823+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:57.818611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.054386+0000","last_change":"2024-09-13T18:53:17.934028+0000","last_active":"2024-09-13T18:53:19.054386+0000","last_peered":"2024-09-13T18:53:19.054386+0000","last_clean":"2024-09-13T18:53:19.054386+0000","last_became_active":"2024-09-13T18:53:17.933760+0000","last_became_peered":"2024-09-13T18:53:17.933760+0000","last_unstale":"2024-09-13T18:53:19.054386+0000","last_undegraded":"2024-09-13T18:53:19.054386+0000","last_fullsized":"2024-09-13T18:53:19.054386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:37.175566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.908354+0000","last_change":"2024-09-13T18:53:17.528129+0000","last_active":"2024-09-13T18:53:18.908354+0000","last_peered":"2024-09-13T18:53:18.908354+0000","last_clean":"2024-09-13T18:53:18.908354+0000","last_became_active":"2024-09-13T18:53:17.527942+0000","last_became_peered":"2024-09-13T18:53:17.527942+0000","last_unstale":"2024-09-13T18:53:18.908354+0000","last_undegraded":"2024-09-13T18:53:18.908354+0000","last_fullsized":"2024-09-13T18:53:18.908354+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:00.921514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.051134+0000","last_change":"2024-09-13T18:53:17.933957+0000","last_active":"2024-09-13T18:53:19.051134+0000","last_peered":"2024-09-13T18:53:19.051134+0000","last_clean":"2024-09-13T18:53:19.051134+0000","last_became_active":"2024-09-13T18:53:17.933627+0000","last_became_peered":"2024-09-13T18:53:17.933627+0000","last_unstale":"2024-09-13T18:53:19.051134+0000","last_undegraded":"2024-09-13T18:53:19.051134+0000","last_fullsized":"2024-09-13T18:53:19.051134+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:10:09.098069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.912958+0000","last_change":"2024-09-13T18:53:17.532663+0000","last_active":"2024-09-13T18:53:18.912958+0000","last_peered":"2024-09-13T18:53:18.912958+0000","last_clean":"2024-09-13T18:53:18.912958+0000","last_became_active":"2024-09-13T18:53:17.532320+0000","last_became_peered":"2024-09-13T18:53:17.532320+0000","last_unstale":"2024-09-13T18:53:18.912958+0000","last_undegraded":"2024-09-13T18:53:18.912958+0000","last_fullsized":"2024-09-13T18:53:18.912958+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:04:38.867717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.907483+0000","last_change":"2024-09-13T18:53:17.926061+0000","last_active":"2024-09-13T18:53:18.907483+0000","last_peered":"2024-09-13T18:53:18.907483+0000","last_clean":"2024-09-13T18:53:18.907483+0000","last_became_active":"2024-09-13T18:53:17.925779+0000","last_became_peered":"2024-09-13T18:53:17.925779+0000","last_unstale":"2024-09-13T18:53:18.907483+0000","last_undegraded":"2024-09-13T18:53:18.907483+0000","last_fullsized":"2024-09-13T18:53:18.907483+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:59:21.219347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.051500+0000","last_change":"2024-09-13T18:53:17.531734+0000","last_active":"2024-09-13T18:53:19.051500+0000","last_peered":"2024-09-13T18:53:19.051500+0000","last_clean":"2024-09-13T18:53:19.051500+0000","last_became_active":"2024-09-13T18:53:17.531477+0000","last_became_peered":"2024-09-13T18:53:17.531477+0000","last_unstale":"2024-09-13T18:53:19.051500+0000","last_undegraded":"2024-09-13T18:53:19.051500+0000","last_fullsized":"2024-09-13T18:53:19.051500+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:12:45.068528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.053249+0000","last_change":"2024-09-13T18:53:17.931093+0000","last_active":"2024-09-13T18:53:19.053249+0000","last_peered":"2024-09-13T18:53:19.053249+0000","last_clean":"2024-09-13T18:53:19.053249+0000","last_became_active":"2024-09-13T18:53:17.930344+0000","last_became_peered":"2024-09-13T18:53:17.930344+0000","last_unstale":"2024-09-13T18:53:19.053249+0000","last_undegraded":"2024-09-13T18:53:19.053249+0000","last_fullsized":"2024-09-13T18:53:19.053249+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:25:06.778238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.906910+0000","last_change":"2024-09-13T18:53:17.527681+0000","last_active":"2024-09-13T18:53:18.906910+0000","last_peered":"2024-09-13T18:53:18.906910+0000","last_clean":"2024-09-13T18:53:18.906910+0000","last_became_active":"2024-09-13T18:53:17.527407+0000","last_became_peered":"2024-09-13T18:53:17.527407+0000","last_unstale":"2024-09-13T18:53:18.906910+0000","last_undegraded":"2024-09-13T18:53:18.906910+0000","last_fullsized":"2024-09-13T18:53:18.906910+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:40:25.128262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.911543+0000","last_change":"2024-09-13T18:53:17.926996+0000","last_active":"2024-09-13T18:53:18.911543+0000","last_peered":"2024-09-13T18:53:18.911543+0000","last_clean":"2024-09-13T18:53:18.911543+0000","last_became_active":"2024-09-13T18:53:17.926769+0000","last_became_peered":"2024-09-13T18:53:17.926769+0000","last_unstale":"2024-09-13T18:53:18.911543+0000","last_undegraded":"2024-09-13T18:53:18.911543+0000","last_fullsized":"2024-09-13T18:53:18.911543+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:27.294047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.906621+0000","last_change":"2024-09-13T18:53:17.525159+0000","last_active":"2024-09-13T18:53:18.906621+0000","last_peered":"2024-09-13T18:53:18.906621+0000","last_clean":"2024-09-13T18:53:18.906621+0000","last_became_active":"2024-09-13T18:53:17.524671+0000","last_became_peered":"2024-09-13T18:53:17.524671+0000","last_unstale":"2024-09-13T18:53:18.906621+0000","last_undegraded":"2024-09-13T18:53:18.906621+0000","last_fullsized":"2024-09-13T18:53:18.906621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:00:51.556312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.913023+0000","last_change":"2024-09-13T18:53:17.934401+0000","last_active":"2024-09-13T18:53:18.913023+0000","last_peered":"2024-09-13T18:53:18.913023+0000","last_clean":"2024-09-13T18:53:18.913023+0000","last_became_active":"2024-09-13T18:53:17.934052+0000","last_became_peered":"2024-09-13T18:53:17.934052+0000","last_unstale":"2024-09-13T18:53:18.913023+0000","last_undegraded":"2024-09-13T18:53:18.913023+0000","last_fullsized":"2024-09-13T18:53:18.913023+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:12.808582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.052100+0000","last_change":"2024-09-13T18:53:17.532754+0000","last_active":"2024-09-13T18:53:19.052100+0000","last_peered":"2024-09-13T18:53:19.052100+0000","last_clean":"2024-09-13T18:53:19.052100+0000","last_became_active":"2024-09-13T18:53:17.532487+0000","last_became_peered":"2024-09-13T18:53:17.532487+0000","last_unstale":"2024-09-13T18:53:19.052100+0000","last_undegraded":"2024-09-13T18:53:19.052100+0000","last_fullsized":"2024-09-13T18:53:19.052100+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:09:05.005778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.055192+0000","last_change":"2024-09-13T18:53:17.934379+0000","last_active":"2024-09-13T18:53:19.055192+0000","last_peered":"2024-09-13T18:53:19.055192+0000","last_clean":"2024-09-13T18:53:19.055192+0000","last_became_active":"2024-09-13T18:53:17.933928+0000","last_became_peered":"2024-09-13T18:53:17.933928+0000","last_unstale":"2024-09-13T18:53:19.055192+0000","last_undegraded":"2024-09-13T18:53:19.055192+0000","last_fullsized":"2024-09-13T18:53:19.055192+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:25:06.778238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.913252+0000","last_change":"2024-09-13T18:53:17.532971+0000","last_active":"2024-09-13T18:53:18.913252+0000","last_peered":"2024-09-13T18:53:18.913252+0000","last_clean":"2024-09-13T18:53:18.913252+0000","last_became_active":"2024-09-13T18:53:17.532660+0000","last_became_peered":"2024-09-13T18:53:17.532660+0000","last_unstale":"2024-09-13T18:53:18.913252+0000","last_undegraded":"2024-09-13T18:53:18.913252+0000","last_fullsized":"2024-09-13T18:53:18.913252+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:53:41.213755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.051829+0000","last_change":"2024-09-13T18:53:17.930925+0000","last_active":"2024-09-13T18:53:19.051829+0000","last_peered":"2024-09-13T18:53:19.051829+0000","last_clean":"2024-09-13T18:53:19.051829+0000","last_became_active":"2024-09-13T18:53:17.930638+0000","last_became_peered":"2024-09-13T18:53:17.930638+0000","last_unstale":"2024-09-13T18:53:19.051829+0000","last_undegraded":"2024-09-13T18:53:19.051829+0000","last_fullsized":"2024-09-13T18:53:19.051829+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:31:13.134078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.913702+0000","last_change":"2024-09-13T18:53:17.531563+0000","last_active":"2024-09-13T18:53:18.913702+0000","last_peered":"2024-09-13T18:53:18.913702+0000","last_clean":"2024-09-13T18:53:18.913702+0000","last_became_active":"2024-09-13T18:53:17.531207+0000","last_became_peered":"2024-09-13T18:53:17.531207+0000","last_unstale":"2024-09-13T18:53:18.913702+0000","last_undegraded":"2024-09-13T18:53:18.913702+0000","last_fullsized":"2024-09-13T18:53:18.913702+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:18:34.411258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.922590+0000","last_change":"2024-09-13T18:53:17.923816+0000","last_active":"2024-09-13T18:53:18.922590+0000","last_peered":"2024-09-13T18:53:18.922590+0000","last_clean":"2024-09-13T18:53:18.922590+0000","last_became_active":"2024-09-13T18:53:17.922884+0000","last_became_peered":"2024-09-13T18:53:17.922884+0000","last_unstale":"2024-09-13T18:53:18.922590+0000","last_undegraded":"2024-09-13T18:53:18.922590+0000","last_fullsized":"2024-09-13T18:53:18.922590+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:31:12.765938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.913207+0000","last_change":"2024-09-13T18:53:17.532001+0000","last_active":"2024-09-13T18:53:18.913207+0000","last_peered":"2024-09-13T18:53:18.913207+0000","last_clean":"2024-09-13T18:53:18.913207+0000","last_became_active":"2024-09-13T18:53:17.531597+0000","last_became_peered":"2024-09-13T18:53:17.531597+0000","last_unstale":"2024-09-13T18:53:18.913207+0000","last_undegraded":"2024-09-13T18:53:18.913207+0000","last_fullsized":"2024-09-13T18:53:18.913207+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:56.752152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.922234+0000","last_change":"2024-09-13T18:53:17.928702+0000","last_active":"2024-09-13T18:53:18.922234+0000","last_peered":"2024-09-13T18:53:18.922234+0000","last_clean":"2024-09-13T18:53:18.922234+0000","last_became_active":"2024-09-13T18:53:17.928439+0000","last_became_peered":"2024-09-13T18:53:17.928439+0000","last_unstale":"2024-09-13T18:53:18.922234+0000","last_undegraded":"2024-09-13T18:53:18.922234+0000","last_fullsized":"2024-09-13T18:53:18.922234+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:08:12.808582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.053406+0000","last_change":"2024-09-13T18:53:17.532762+0000","last_active":"2024-09-13T18:53:19.053406+0000","last_peered":"2024-09-13T18:53:19.053406+0000","last_clean":"2024-09-13T18:53:19.053406+0000","last_became_active":"2024-09-13T18:53:17.532471+0000","last_became_peered":"2024-09-13T18:53:17.532471+0000","last_unstale":"2024-09-13T18:53:19.053406+0000","last_undegraded":"2024-09-13T18:53:19.053406+0000","last_fullsized":"2024-09-13T18:53:19.053406+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:38:58.453219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.922432+0000","last_change":"2024-09-13T18:53:17.923635+0000","last_active":"2024-09-13T18:53:18.922432+0000","last_peered":"2024-09-13T18:53:18.922432+0000","last_clean":"2024-09-13T18:53:18.922432+0000","last_became_active":"2024-09-13T18:53:17.923361+0000","last_became_peered":"2024-09-13T18:53:17.923361+0000","last_unstale":"2024-09-13T18:53:18.922432+0000","last_undegraded":"2024-09-13T18:53:18.922432+0000","last_fullsized":"2024-09-13T18:53:18.922432+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:48:31.604140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.911629+0000","last_change":"2024-09-13T18:53:17.533620+0000","last_active":"2024-09-13T18:53:18.911629+0000","last_peered":"2024-09-13T18:53:18.911629+0000","last_clean":"2024-09-13T18:53:18.911629+0000","last_became_active":"2024-09-13T18:53:17.532995+0000","last_became_peered":"2024-09-13T18:53:17.532995+0000","last_unstale":"2024-09-13T18:53:18.911629+0000","last_undegraded":"2024-09-13T18:53:18.911629+0000","last_fullsized":"2024-09-13T18:53:18.911629+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:38:58.453219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.922165+0000","last_change":"2024-09-13T18:53:17.918587+0000","last_active":"2024-09-13T18:53:18.922165+0000","last_peered":"2024-09-13T18:53:18.922165+0000","last_clean":"2024-09-13T18:53:18.922165+0000","last_became_active":"2024-09-13T18:53:17.918311+0000","last_became_peered":"2024-09-13T18:53:17.918311+0000","last_unstale":"2024-09-13T18:53:18.922165+0000","last_undegraded":"2024-09-13T18:53:18.922165+0000","last_fullsized":"2024-09-13T18:53:18.922165+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:58:26.836926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.912261+0000","last_change":"2024-09-13T18:53:17.529307+0000","last_active":"2024-09-13T18:53:18.912261+0000","last_peered":"2024-09-13T18:53:18.912261+0000","last_clean":"2024-09-13T18:53:18.912261+0000","last_became_active":"2024-09-13T18:53:17.528671+0000","last_became_peered":"2024-09-13T18:53:17.528671+0000","last_unstale":"2024-09-13T18:53:18.912261+0000","last_undegraded":"2024-09-13T18:53:18.912261+0000","last_fullsized":"2024-09-13T18:53:18.912261+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:20:53.208344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.053891+0000","last_change":"2024-09-13T18:53:17.922279+0000","last_active":"2024-09-13T18:53:19.053891+0000","last_peered":"2024-09-13T18:53:19.053891+0000","last_clean":"2024-09-13T18:53:19.053891+0000","last_became_active":"2024-09-13T18:53:17.921973+0000","last_became_peered":"2024-09-13T18:53:17.921973+0000","last_unstale":"2024-09-13T18:53:19.053891+0000","last_undegraded":"2024-09-13T18:53:19.053891+0000","last_fullsized":"2024-09-13T18:53:19.053891+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:33.536636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.054070+0000","last_change":"2024-09-13T18:53:17.534234+0000","last_active":"2024-09-13T18:53:19.054070+0000","last_peered":"2024-09-13T18:53:19.054070+0000","last_clean":"2024-09-13T18:53:19.054070+0000","last_became_active":"2024-09-13T18:53:17.533948+0000","last_became_peered":"2024-09-13T18:53:17.533948+0000","last_unstale":"2024-09-13T18:53:19.054070+0000","last_undegraded":"2024-09-13T18:53:19.054070+0000","last_fullsized":"2024-09-13T18:53:19.054070+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:00.921514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.922581+0000","last_change":"2024-09-13T18:53:17.933895+0000","last_active":"2024-09-13T18:53:18.922581+0000","last_peered":"2024-09-13T18:53:18.922581+0000","last_clean":"2024-09-13T18:53:18.922581+0000","last_became_active":"2024-09-13T18:53:17.933645+0000","last_became_peered":"2024-09-13T18:53:17.933645+0000","last_unstale":"2024-09-13T18:53:18.922581+0000","last_undegraded":"2024-09-13T18:53:18.922581+0000","last_fullsized":"2024-09-13T18:53:18.922581+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:50.975295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.052171+0000","last_change":"2024-09-13T18:53:17.535754+0000","last_active":"2024-09-13T18:53:19.052171+0000","last_peered":"2024-09-13T18:53:19.052171+0000","last_clean":"2024-09-13T18:53:19.052171+0000","last_became_active":"2024-09-13T18:53:17.535370+0000","last_became_peered":"2024-09-13T18:53:17.535370+0000","last_unstale":"2024-09-13T18:53:19.052171+0000","last_undegraded":"2024-09-13T18:53:19.052171+0000","last_fullsized":"2024-09-13T18:53:19.052171+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:14.518730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.055196+0000","last_change":"2024-09-13T18:53:17.933751+0000","last_active":"2024-09-13T18:53:19.055196+0000","last_peered":"2024-09-13T18:53:19.055196+0000","last_clean":"2024-09-13T18:53:19.055196+0000","last_became_active":"2024-09-13T18:53:17.933521+0000","last_became_peered":"2024-09-13T18:53:17.933521+0000","last_unstale":"2024-09-13T18:53:19.055196+0000","last_undegraded":"2024-09-13T18:53:19.055196+0000","last_fullsized":"2024-09-13T18:53:19.055196+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:43:05.749061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:20.619052+0000","last_change":"2024-09-13T18:53:17.532227+0000","last_active":"2024-09-13T18:53:20.619052+0000","last_peered":"2024-09-13T18:53:20.619052+0000","last_clean":"2024-09-13T18:53:20.619052+0000","last_became_active":"2024-09-13T18:53:17.531959+0000","last_became_peered":"2024-09-13T18:53:17.531959+0000","last_unstale":"2024-09-13T18:53:20.619052+0000","last_undegraded":"2024-09-13T18:53:20.619052+0000","last_fullsized":"2024-09-13T18:53:20.619052+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:33:08.624574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.053018+0000","last_change":"2024-09-13T18:53:17.926288+0000","last_active":"2024-09-13T18:53:19.053018+0000","last_peered":"2024-09-13T18:53:19.053018+0000","last_clean":"2024-09-13T18:53:19.053018+0000","last_became_active":"2024-09-13T18:53:17.926030+0000","last_became_peered":"2024-09-13T18:53:17.926030+0000","last_unstale":"2024-09-13T18:53:19.053018+0000","last_undegraded":"2024-09-13T18:53:19.053018+0000","last_fullsized":"2024-09-13T18:53:19.053018+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:55:23.108275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.051796+0000","last_change":"2024-09-13T18:53:17.530326+0000","last_active":"2024-09-13T18:53:19.051796+0000","last_peered":"2024-09-13T18:53:19.051796+0000","last_clean":"2024-09-13T18:53:19.051796+0000","last_became_active":"2024-09-13T18:53:17.529936+0000","last_became_peered":"2024-09-13T18:53:17.529936+0000","last_unstale":"2024-09-13T18:53:19.051796+0000","last_undegraded":"2024-09-13T18:53:19.051796+0000","last_fullsized":"2024-09-13T18:53:19.051796+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:38:58.453219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.911419+0000","last_change":"2024-09-13T18:53:17.922688+0000","last_active":"2024-09-13T18:53:18.911419+0000","last_peered":"2024-09-13T18:53:18.911419+0000","last_clean":"2024-09-13T18:53:18.911419+0000","last_became_active":"2024-09-13T18:53:17.921989+0000","last_became_peered":"2024-09-13T18:53:17.921989+0000","last_unstale":"2024-09-13T18:53:18.911419+0000","last_undegraded":"2024-09-13T18:53:18.911419+0000","last_fullsized":"2024-09-13T18:53:18.911419+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:49:29.274291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.053486+0000","last_change":"2024-09-13T18:53:17.537278+0000","last_active":"2024-09-13T18:53:19.053486+0000","last_peered":"2024-09-13T18:53:19.053486+0000","last_clean":"2024-09-13T18:53:19.053486+0000","last_became_active":"2024-09-13T18:53:17.536938+0000","last_became_peered":"2024-09-13T18:53:17.536938+0000","last_unstale":"2024-09-13T18:53:19.053486+0000","last_undegraded":"2024-09-13T18:53:19.053486+0000","last_fullsized":"2024-09-13T18:53:19.053486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:53:51.468967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.053251+0000","last_change":"2024-09-13T18:53:17.922168+0000","last_active":"2024-09-13T18:53:19.053251+0000","last_peered":"2024-09-13T18:53:19.053251+0000","last_clean":"2024-09-13T18:53:19.053251+0000","last_became_active":"2024-09-13T18:53:17.921723+0000","last_became_peered":"2024-09-13T18:53:17.921723+0000","last_unstale":"2024-09-13T18:53:19.053251+0000","last_undegraded":"2024-09-13T18:53:19.053251+0000","last_fullsized":"2024-09-13T18:53:19.053251+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:03:01.931243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.052018+0000","last_change":"2024-09-13T18:53:17.535295+0000","last_active":"2024-09-13T18:53:19.052018+0000","last_peered":"2024-09-13T18:53:19.052018+0000","last_clean":"2024-09-13T18:53:19.052018+0000","last_became_active":"2024-09-13T18:53:17.535059+0000","last_became_peered":"2024-09-13T18:53:17.535059+0000","last_unstale":"2024-09-13T18:53:19.052018+0000","last_undegraded":"2024-09-13T18:53:19.052018+0000","last_fullsized":"2024-09-13T18:53:19.052018+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:55.721404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.906598+0000","last_change":"2024-09-13T18:53:17.928479+0000","last_active":"2024-09-13T18:53:18.906598+0000","last_peered":"2024-09-13T18:53:18.906598+0000","last_clean":"2024-09-13T18:53:18.906598+0000","last_became_active":"2024-09-13T18:53:17.927627+0000","last_became_peered":"2024-09-13T18:53:17.927627+0000","last_unstale":"2024-09-13T18:53:18.906598+0000","last_undegraded":"2024-09-13T18:53:18.906598+0000","last_fullsized":"2024-09-13T18:53:18.906598+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:33.536636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.912451+0000","last_change":"2024-09-13T18:53:17.537924+0000","last_active":"2024-09-13T18:53:18.912451+0000","last_peered":"2024-09-13T18:53:18.912451+0000","last_clean":"2024-09-13T18:53:18.912451+0000","last_became_active":"2024-09-13T18:53:17.537625+0000","last_became_peered":"2024-09-13T18:53:17.537625+0000","last_unstale":"2024-09-13T18:53:18.912451+0000","last_undegraded":"2024-09-13T18:53:18.912451+0000","last_fullsized":"2024-09-13T18:53:18.912451+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:58:09.407484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.912211+0000","last_change":"2024-09-13T18:53:17.915622+0000","last_active":"2024-09-13T18:53:18.912211+0000","last_peered":"2024-09-13T18:53:18.912211+0000","last_clean":"2024-09-13T18:53:18.912211+0000","last_became_active":"2024-09-13T18:53:17.915288+0000","last_became_peered":"2024-09-13T18:53:17.915288+0000","last_unstale":"2024-09-13T18:53:18.912211+0000","last_undegraded":"2024-09-13T18:53:18.912211+0000","last_fullsized":"2024-09-13T18:53:18.912211+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:31:12.765938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.906883+0000","last_change":"2024-09-13T18:53:17.529158+0000","last_active":"2024-09-13T18:53:18.906883+0000","last_peered":"2024-09-13T18:53:18.906883+0000","last_clean":"2024-09-13T18:53:18.906883+0000","last_became_active":"2024-09-13T18:53:17.528790+0000","last_became_peered":"2024-09-13T18:53:17.528790+0000","last_unstale":"2024-09-13T18:53:18.906883+0000","last_undegraded":"2024-09-13T18:53:18.906883+0000","last_fullsized":"2024-09-13T18:53:18.906883+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:37.370138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:19.053533+0000","last_change":"2024-09-13T18:53:17.924810+0000","last_active":"2024-09-13T18:53:19.053533+0000","last_peered":"2024-09-13T18:53:19.053533+0000","last_clean":"2024-09-13T18:53:19.053533+0000","last_became_active":"2024-09-13T18:53:17.924557+0000","last_became_peered":"2024-09-13T18:53:17.924557+0000","last_unstale":"2024-09-13T18:53:19.053533+0000","last_undegraded":"2024-09-13T18:53:19.053533+0000","last_fullsized":"2024-09-13T18:53:19.053533+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:17.504762+0000","last_clean_scrub_stamp":"2024-09-13T18:53:17.504762+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:52:35.563770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T18:53:18.911932+0000","last_change":"2024-09-13T18:53:17.535468+0000","last_active":"2024-09-13T18:53:18.911932+0000","last_peered":"2024-09-13T18:53:18.911932+0000","last_clean":"2024-09-13T18:53:18.911932+0000","last_became_active":"2024-09-13T18:53:17.535257+0000","last_became_peered":"2024-09-13T18:53:17.535257+0000","last_unstale":"2024-09-13T18:53:18.911932+0000","last_undegraded":"2024-09-13T18:53:18.911932+0000","last_fullsized":"2024-09-13T18:53:18.911932+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:53:16.495033+0000","last_clean_scrub_stamp":"2024-09-13T18:53:16.495033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:12:45.068528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":173,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T18:53:17.916305+0000","last_change":"2024-09-13T18:52:15.080080+0000","last_active":"2024-09-13T18:53:17.916305+0000","last_peered":"2024-09-13T18:53:17.916305+0000","last_clean":"2024-09-13T18:53:17.916305+0000","last_became_active":"2024-09-13T18:52:15.078100+0000","last_became_peered":"2024-09-13T18:52:15.078100+0000","last_unstale":"2024-09-13T18:53:17.916305+0000","last_undegraded":"2024-09-13T18:53:17.916305+0000","last_fullsized":"2024-09-13T18:53:17.916305+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_clean_scrub_stamp":"2024-09-13T18:52:13.196473+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:29:59.452757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1003520,"data_stored":750229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.73299999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.55500000000000005}]},{"osd":2,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.68200000000000005}]},{"osd":4,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.93200000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.0660000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.97199999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":160387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":1.085},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.2230000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":1.3540000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.196}]},{"osd":3,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":1.131}]},{"osd":4,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.81000000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":1.2589999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.0859999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.99399999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":160443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.72799999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.90800000000000003}]},{"osd":2,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":1.046}]},{"osd":3,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.82199999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.0209999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":1.264},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.0009999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":3.4369999999999998,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":1.325}]}]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31532,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340308,"statfs":{"total":96636764160,"available":96604475392,"internally_reserved":0,"allocated":413696,"data_stored":159875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.80900000000000005}]},{"osd":1,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":1.079}]},{"osd":2,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.89600000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":1.012}]},{"osd":5,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.87}]},{"osd":6,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.038}]},{"osd":7,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.143},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.103}]}]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":421888,"data_stored":161357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.67000000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.754}]},{"osd":2,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.79300000000000004}]},{"osd":4,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.0940000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":0.878},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.0409999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":1.139},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.0169999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.1859999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35624,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94336216,"statfs":{"total":96636764160,"available":96600285184,"internally_reserved":0,"allocated":409600,"data_stored":159841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.40899999999999997}]},{"osd":1,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.46500000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.51500000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":0.56699999999999995},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.751},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.70999999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":421888,"data_stored":161357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.786,"5min":2.786,"15min":2.786},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.84999999999999998}]},{"osd":2,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.113,"5min":3.113,"15min":3.113},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.73099999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.169}]},{"osd":5,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.4780000000000002,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":0.88500000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.1180000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.4409999999999998,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.1459999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":999424,"data_stored":750139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.85899999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.83599999999999997}]},{"osd":4,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.0800000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.1180000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":1.226},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.88900000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.1799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:53:22.363 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T18:53:22.364 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T18:53:22.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.488+0000 7ffb96906700 1 Processor -- start 2024-09-13T18:53:22.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.488+0000 7ffb96906700 1 -- start start 2024-09-13T18:53:22.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb96906700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb9015aaa0 0x7ffb900b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:22.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb96906700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7ffb90070720 0x7ffb900b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:22.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb96906700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb900c9d20 con 0x7ffb9006db00 2024-09-13T18:53:22.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb96906700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb900b26c0 con 0x7ffb9015aaa0 2024-09-13T18:53:22.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb96906700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb900b4c00 con 0x7ffb90070720 2024-09-13T18:53:22.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb95103700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb9015aaa0 0x7ffb900b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:22.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb95103700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb9015aaa0 0x7ffb900b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53268/0 (socket says 172.21.15.7:53268) 2024-09-13T18:53:22.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb95103700 1 -- 172.21.15.7:0/524540139 learned_addr learned my addr 172.21.15.7:0/524540139 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:22.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb96105700 1 --2- 172.21.15.7:0/524540139 >> v2:172.21.15.171:3300/0 conn(0x7ffb90070720 0x7ffb900b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:22.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb95103700 1 -- 172.21.15.7:0/524540139 >> v2:172.21.15.171:3300/0 conn(0x7ffb90070720 msgr2=0x7ffb900b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:22.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb95103700 1 --2- 172.21.15.7:0/524540139 >> v2:172.21.15.171:3300/0 conn(0x7ffb90070720 0x7ffb900b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb95103700 1 -- 172.21.15.7:0/524540139 >> v1:172.21.15.171:6789/0 conn(0x7ffb9006db00 legacy=0x7ffb900b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:22.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb95103700 1 -- 172.21.15.7:0/524540139 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb900b28e0 con 0x7ffb9015aaa0 2024-09-13T18:53:22.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb95103700 1 --2- 172.21.15.7:0/524540139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb9015aaa0 0x7ffb900b4180 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ffb900cc8e0 tx=0x7ffb8000be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdebf35f43ea61f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:22.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb94902700 1 -- 172.21.15.7:0/524540139 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8000dd90 con 0x7ffb9015aaa0 2024-09-13T18:53:22.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb94902700 1 -- 172.21.15.7:0/524540139 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb80010470 con 0x7ffb9015aaa0 2024-09-13T18:53:22.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb94902700 1 -- 172.21.15.7:0/524540139 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb80010800 con 0x7ffb9015aaa0 2024-09-13T18:53:22.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb96906700 1 -- 172.21.15.7:0/524540139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb9015aaa0 msgr2=0x7ffb900b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:22.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb96906700 1 --2- 172.21.15.7:0/524540139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb9015aaa0 0x7ffb900b4180 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ffb900cc8e0 tx=0x7ffb8000be90 comp rx=0 tx=0).stop 2024-09-13T18:53:22.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb96906700 1 -- 172.21.15.7:0/524540139 shutdown_connections 2024-09-13T18:53:22.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb96906700 1 --2- 172.21.15.7:0/524540139 >> v2:172.21.15.171:3300/0 conn(0x7ffb90070720 0x7ffb900b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb96906700 1 --2- 172.21.15.7:0/524540139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb9015aaa0 0x7ffb900b4180 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb96906700 1 -- 172.21.15.7:0/524540139 >> 172.21.15.7:0/524540139 conn(0x7ffb900cba60 msgr2=0x7ffb900cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:22.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb96906700 1 -- 172.21.15.7:0/524540139 shutdown_connections 2024-09-13T18:53:22.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.492+0000 7ffb96906700 1 -- 172.21.15.7:0/524540139 wait complete. 2024-09-13T18:53:22.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.496+0000 7ffb96906700 1 Processor -- start 2024-09-13T18:53:22.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.496+0000 7ffb96906700 1 -- start start 2024-09-13T18:53:22.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb96906700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb9006db00 0x7ffb9006b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:22.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb96906700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffb90070720 0x7ffb900c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:22.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb96906700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffb9015aaa0 0x7ffb900ce740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:22.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb96906700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9006a4b0 con 0x7ffb9006db00 2024-09-13T18:53:22.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb96906700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9006a790 con 0x7ffb9015aaa0 2024-09-13T18:53:22.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb96906700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9006aa70 con 0x7ffb90070720 2024-09-13T18:53:22.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb95904700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb9006db00 0x7ffb9006b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:22.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb95904700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb9006db00 0x7ffb9006b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53282/0 (socket says 172.21.15.7:53282) 2024-09-13T18:53:22.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb95904700 1 -- 172.21.15.7:0/1224542045 learned_addr learned my addr 172.21.15.7:0/1224542045 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:22.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb96105700 1 --2- 172.21.15.7:0/1224542045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffb9015aaa0 0x7ffb900ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:22.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb95103700 1 --2- 172.21.15.7:0/1224542045 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffb90070720 0x7ffb900c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:22.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb95904700 1 -- 172.21.15.7:0/1224542045 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffb90070720 msgr2=0x7ffb900c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:22.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb95904700 1 --2- 172.21.15.7:0/1224542045 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffb90070720 0x7ffb900c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb95904700 1 -- 172.21.15.7:0/1224542045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffb9015aaa0 msgr2=0x7ffb900ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:22.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb95904700 1 --2- 172.21.15.7:0/1224542045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffb9015aaa0 0x7ffb900ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb95904700 1 -- 172.21.15.7:0/1224542045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8000b810 con 0x7ffb9006db00 2024-09-13T18:53:22.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb95103700 1 --2- 172.21.15.7:0/1224542045 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffb90070720 0x7ffb900c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:53:22.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb95904700 1 --2- 172.21.15.7:0/1224542045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb9006db00 0x7ffb9006b550 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00c500 tx=0x7ffb8c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:22.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb867fc700 1 -- 172.21.15.7:0/1224542045 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c0052d0 con 0x7ffb9006db00 2024-09-13T18:53:22.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb867fc700 1 -- 172.21.15.7:0/1224542045 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8c005430 con 0x7ffb9006db00 2024-09-13T18:53:22.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb96906700 1 -- 172.21.15.7:0/1224542045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb9006ad50 con 0x7ffb9006db00 2024-09-13T18:53:22.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb867fc700 1 -- 172.21.15.7:0/1224542045 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c0055a0 con 0x7ffb9006db00 2024-09-13T18:53:22.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb96906700 1 -- 172.21.15.7:0/1224542045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb900c8fd0 con 0x7ffb9006db00 2024-09-13T18:53:22.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.500+0000 7ffb96906700 1 -- 172.21.15.7:0/1224542045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb900b4fe0 con 0x7ffb9006db00 2024-09-13T18:53:22.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.504+0000 7ffb867fc700 1 -- 172.21.15.7:0/1224542045 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffb8c0038e0 con 0x7ffb9006db00 2024-09-13T18:53:22.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.504+0000 7ffb867fc700 1 --2- 172.21.15.7:0/1224542045 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ffb74066730 0x7ffb74068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:22.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.504+0000 7ffb867fc700 1 -- 172.21.15.7:0/1224542045 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb8c083720 con 0x7ffb9006db00 2024-09-13T18:53:22.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.504+0000 7ffb95103700 1 --2- 172.21.15.7:0/1224542045 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ffb74066730 0x7ffb74068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:22.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.504+0000 7ffb95103700 1 --2- 172.21.15.7:0/1224542045 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ffb74066730 0x7ffb74068bf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffb9015aef0 tx=0x7ffb80013000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:22.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.512+0000 7ffb867fc700 1 -- 172.21.15.7:0/1224542045 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ffb8c050a00 con 0x7ffb9006db00 2024-09-13T18:53:22.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.744+0000 7ffb96906700 1 -- 172.21.15.7:0/1224542045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ffb9006b270 con 0x7ffb9006db00 2024-09-13T18:53:22.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.752+0000 7ffb867fc700 1 -- 172.21.15.7:0/1224542045 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7ffb8c0170e0 con 0x7ffb9006db00 2024-09-13T18:53:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.756+0000 7ffb96906700 1 -- 172.21.15.7:0/1224542045 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ffb74066730 msgr2=0x7ffb74068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.756+0000 7ffb96906700 1 --2- 172.21.15.7:0/1224542045 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ffb74066730 0x7ffb74068bf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffb9015aef0 tx=0x7ffb80013000 comp rx=0 tx=0).stop 2024-09-13T18:53:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.756+0000 7ffb96906700 1 -- 172.21.15.7:0/1224542045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb9006db00 msgr2=0x7ffb9006b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.756+0000 7ffb96906700 1 --2- 172.21.15.7:0/1224542045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb9006db00 0x7ffb9006b550 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00c500 tx=0x7ffb8c00c8c0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.756+0000 7ffb96906700 1 -- 172.21.15.7:0/1224542045 shutdown_connections 2024-09-13T18:53:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.756+0000 7ffb96906700 1 --2- 172.21.15.7:0/1224542045 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ffb74066730 0x7ffb74068bf0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.756+0000 7ffb96906700 1 --2- 172.21.15.7:0/1224542045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffb9015aaa0 0x7ffb900ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.756+0000 7ffb96906700 1 --2- 172.21.15.7:0/1224542045 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffb90070720 0x7ffb900c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.756+0000 7ffb96906700 1 --2- 172.21.15.7:0/1224542045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb9006db00 0x7ffb9006b550 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.756+0000 7ffb96906700 1 -- 172.21.15.7:0/1224542045 >> 172.21.15.7:0/1224542045 conn(0x7ffb900cba60 msgr2=0x7ffb900cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.756+0000 7ffb96906700 1 -- 172.21.15.7:0/1224542045 shutdown_connections 2024-09-13T18:53:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.756+0000 7ffb96906700 1 -- 172.21.15.7:0/1224542045 wait complete. 2024-09-13T18:53:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-09-13T18:53:22.777 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:22.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee23770700 1 Processor -- start 2024-09-13T18:53:22.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee23770700 1 -- start start 2024-09-13T18:53:22.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee23770700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee1c1553e0 0x7fee1c071d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:22.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee23770700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fee1c15b9a0 0x7fee1c0cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:22.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee23770700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee1c1557c0 con 0x7fee1c072270 2024-09-13T18:53:22.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee23770700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee1c0726b0 con 0x7fee1c1553e0 2024-09-13T18:53:22.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee23770700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c0b2dd0 con 0x7fee1c15b9a0 2024-09-13T18:53:22.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee2150c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee1c1553e0 0x7fee1c071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:22.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee2150c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee1c1553e0 0x7fee1c071d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53292/0 (socket says 172.21.15.7:53292) 2024-09-13T18:53:22.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee2150c700 1 -- 172.21.15.7:0/3591556424 learned_addr learned my addr 172.21.15.7:0/3591556424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:22.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee21d0d700 1 --2- 172.21.15.7:0/3591556424 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fee1c15b9a0 0x7fee1c0cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:22.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee2150c700 1 -- 172.21.15.7:0/3591556424 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fee1c15b9a0 msgr2=0x7fee1c0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:22.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee2150c700 1 --2- 172.21.15.7:0/3591556424 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fee1c15b9a0 0x7fee1c0cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee2150c700 1 -- 172.21.15.7:0/3591556424 >> v1:172.21.15.171:6789/0 conn(0x7fee1c072270 legacy=0x7fee1c15b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:22.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee2150c700 1 -- 172.21.15.7:0/3591556424 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1c15bee0 con 0x7fee1c1553e0 2024-09-13T18:53:22.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee2150c700 1 --2- 172.21.15.7:0/3591556424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c1553e0 0x7fee1c071d30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fee18009910 tx=0x7fee18009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf105617d6fe6d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:22.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee13fff700 1 -- 172.21.15.7:0/3591556424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18011830 con 0x7fee1c1553e0 2024-09-13T18:53:22.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee13fff700 1 -- 172.21.15.7:0/3591556424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee18011990 con 0x7fee1c1553e0 2024-09-13T18:53:22.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee13fff700 1 -- 172.21.15.7:0/3591556424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18011d40 con 0x7fee1c1553e0 2024-09-13T18:53:22.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee23770700 1 -- 172.21.15.7:0/3591556424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c1553e0 msgr2=0x7fee1c071d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:22.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee23770700 1 --2- 172.21.15.7:0/3591556424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c1553e0 0x7fee1c071d30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fee18009910 tx=0x7fee18009c20 comp rx=0 tx=0).stop 2024-09-13T18:53:22.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee23770700 1 -- 172.21.15.7:0/3591556424 shutdown_connections 2024-09-13T18:53:22.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee23770700 1 --2- 172.21.15.7:0/3591556424 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fee1c15b9a0 0x7fee1c0cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee23770700 1 --2- 172.21.15.7:0/3591556424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c1553e0 0x7fee1c071d30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee23770700 1 -- 172.21.15.7:0/3591556424 >> 172.21.15.7:0/3591556424 conn(0x7fee1c0b17f0 msgr2=0x7fee1c0b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:22.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee23770700 1 -- 172.21.15.7:0/3591556424 shutdown_connections 2024-09-13T18:53:22.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.888+0000 7fee23770700 1 -- 172.21.15.7:0/3591556424 wait complete. 2024-09-13T18:53:22.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.892+0000 7fee23770700 1 Processor -- start 2024-09-13T18:53:22.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.892+0000 7fee23770700 1 -- start start 2024-09-13T18:53:22.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.892+0000 7fee23770700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c072270 0x7fee1c155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:22.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.892+0000 7fee23770700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee1c1553e0 0x7fee1c140800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:22.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.892+0000 7fee23770700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fee1c15b9a0 0x7fee1c15da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:22.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.892+0000 7fee23770700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c141ee0 con 0x7fee1c072270 2024-09-13T18:53:22.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.892+0000 7fee23770700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c156170 con 0x7fee1c1553e0 2024-09-13T18:53:22.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.892+0000 7fee23770700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c141260 con 0x7fee1c15b9a0 2024-09-13T18:53:22.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.892+0000 7fee2150c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c072270 0x7fee1c155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:22.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.896+0000 7fee2150c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c072270 0x7fee1c155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53294/0 (socket says 172.21.15.7:53294) 2024-09-13T18:53:22.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.896+0000 7fee2150c700 1 -- 172.21.15.7:0/1345404582 learned_addr learned my addr 172.21.15.7:0/1345404582 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:22.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.896+0000 7fee20d0b700 1 --2- 172.21.15.7:0/1345404582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee1c1553e0 0x7fee1c140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:22.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.896+0000 7fee21d0d700 1 --2- 172.21.15.7:0/1345404582 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fee1c15b9a0 0x7fee1c15da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:22.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.896+0000 7fee2150c700 1 -- 172.21.15.7:0/1345404582 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fee1c15b9a0 msgr2=0x7fee1c15da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:22.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.896+0000 7fee2150c700 1 --2- 172.21.15.7:0/1345404582 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fee1c15b9a0 0x7fee1c15da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.896+0000 7fee2150c700 1 -- 172.21.15.7:0/1345404582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee1c1553e0 msgr2=0x7fee1c140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:22.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.896+0000 7fee2150c700 1 --2- 172.21.15.7:0/1345404582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee1c1553e0 0x7fee1c140800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:22.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.896+0000 7fee2150c700 1 -- 172.21.15.7:0/1345404582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee18009570 con 0x7fee1c072270 2024-09-13T18:53:22.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.896+0000 7fee20d0b700 1 --2- 172.21.15.7:0/1345404582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee1c1553e0 0x7fee1c140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:22.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.896+0000 7fee21d0d700 1 --2- 172.21.15.7:0/1345404582 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fee1c15b9a0 0x7fee1c15da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:22.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.896+0000 7fee2150c700 1 --2- 172.21.15.7:0/1345404582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c072270 0x7fee1c155c30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0cd590 tx=0x7fee18004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:22.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.896+0000 7fee11ffb700 1 -- 172.21.15.7:0/1345404582 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18004510 con 0x7fee1c072270 2024-09-13T18:53:22.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.896+0000 7fee11ffb700 1 -- 172.21.15.7:0/1345404582 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee180036c0 con 0x7fee1c072270 2024-09-13T18:53:22.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.896+0000 7fee23770700 1 -- 172.21.15.7:0/1345404582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1c141540 con 0x7fee1c072270 2024-09-13T18:53:22.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.896+0000 7fee11ffb700 1 -- 172.21.15.7:0/1345404582 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18003a70 con 0x7fee1c072270 2024-09-13T18:53:22.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.896+0000 7fee23770700 1 -- 172.21.15.7:0/1345404582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1c141b00 con 0x7fee1c072270 2024-09-13T18:53:22.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.896+0000 7fee23770700 1 -- 172.21.15.7:0/1345404582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee1c071d30 con 0x7fee1c072270 2024-09-13T18:53:22.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.900+0000 7fee11ffb700 1 -- 172.21.15.7:0/1345404582 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fee18003bd0 con 0x7fee1c072270 2024-09-13T18:53:22.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.900+0000 7fee11ffb700 1 --2- 172.21.15.7:0/1345404582 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fee00066730 0x7fee00068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:22.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.900+0000 7fee11ffb700 1 -- 172.21.15.7:0/1345404582 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee18021030 con 0x7fee1c072270 2024-09-13T18:53:22.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.900+0000 7fee20d0b700 1 --2- 172.21.15.7:0/1345404582 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fee00066730 0x7fee00068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:22.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.900+0000 7fee20d0b700 1 --2- 172.21.15.7:0/1345404582 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fee00066730 0x7fee00068bf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0b47f0 tx=0x7fee0c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:22.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:22.908+0000 7fee11ffb700 1 -- 172.21.15.7:0/1345404582 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fee18052ac0 con 0x7fee1c072270 2024-09-13T18:53:23.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.116+0000 7fee23770700 1 -- 172.21.15.7:0/1345404582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee1c0b25b0 con 0x7fee1c072270 2024-09-13T18:53:23.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.116+0000 7fee11ffb700 1 -- 172.21.15.7:0/1345404582 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fee18015090 con 0x7fee1c072270 2024-09-13T18:53:23.124 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:23.124 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3430789885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3430789885},{"type":"v1","addr":"172.21.15.171:6837","nonce":3430789885}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5188,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1714488523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1714488523},{"type":"v1","addr":"172.21.15.171:6835","nonce":1714488523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1159212209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1159212209},{"type":"v1","addr":"172.21.15.7:6833","nonce":1159212209}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:53:17.897317+0000","modified":"2024-09-13T18:53:20.920860+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/2843879273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2843879273},{"type":"v1","addr":"172.21.15.7:6835","nonce":2843879273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T18:53:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.120+0000 7fee23770700 1 -- 172.21.15.7:0/1345404582 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fee00066730 msgr2=0x7fee00068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.120+0000 7fee23770700 1 --2- 172.21.15.7:0/1345404582 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fee00066730 0x7fee00068bf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0b47f0 tx=0x7fee0c009040 comp rx=0 tx=0).stop 2024-09-13T18:53:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.120+0000 7fee23770700 1 -- 172.21.15.7:0/1345404582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c072270 msgr2=0x7fee1c155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.120+0000 7fee23770700 1 --2- 172.21.15.7:0/1345404582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c072270 0x7fee1c155c30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0cd590 tx=0x7fee18004770 comp rx=0 tx=0).stop 2024-09-13T18:53:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.120+0000 7fee23770700 1 -- 172.21.15.7:0/1345404582 shutdown_connections 2024-09-13T18:53:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.120+0000 7fee23770700 1 --2- 172.21.15.7:0/1345404582 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fee00066730 0x7fee00068bf0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.120+0000 7fee23770700 1 --2- 172.21.15.7:0/1345404582 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fee1c15b9a0 0x7fee1c15da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.120+0000 7fee23770700 1 --2- 172.21.15.7:0/1345404582 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee1c1553e0 0x7fee1c140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.120+0000 7fee23770700 1 --2- 172.21.15.7:0/1345404582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c072270 0x7fee1c155c30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.120+0000 7fee23770700 1 -- 172.21.15.7:0/1345404582 >> 172.21.15.7:0/1345404582 conn(0x7fee1c0b17f0 msgr2=0x7fee1c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.120+0000 7fee23770700 1 -- 172.21.15.7:0/1345404582 shutdown_connections 2024-09-13T18:53:23.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.120+0000 7fee23770700 1 -- 172.21.15.7:0/1345404582 wait complete. 2024-09-13T18:53:23.128 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2024-09-13T18:53:23.543 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T18:53:17.897317+0000', 'modified': '2024-09-13T18:53:20.920860+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5232}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5232': {'gid': 5232, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/2843879273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 2843879273}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 2843879273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T18:53:23.543 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T18:53:23.543 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-13T18:53:23.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.664+0000 7fcdfe0e4700 1 Processor -- start 2024-09-13T18:53:23.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.664+0000 7fcdfe0e4700 1 -- start start 2024-09-13T18:53:23.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdfe0e4700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fcdf815caf0 0x7fcdf80ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdfe0e4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcdf815ced0 0x7fcdf80ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdfe0e4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf80cd110 con 0x7fcdf8071c80 2024-09-13T18:53:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdfe0e4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf80acc30 con 0x7fcdf815ced0 2024-09-13T18:53:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdfe0e4700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf80acf10 con 0x7fcdf815caf0 2024-09-13T18:53:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdeffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcdf815ced0 0x7fcdf80ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdf7fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcdf8071c80 0x7fcdf80b2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39208/0 (socket says 172.21.15.7:39208) 2024-09-13T18:53:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdf7fff700 1 -- 172.21.15.7:0/1620218745 learned_addr learned my addr 172.21.15.7:0/1620218745 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:23.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdf77fe700 1 --2- 172.21.15.7:0/1620218745 >> v2:172.21.15.171:3300/0 conn(0x7fcdf815caf0 0x7fcdf80ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdeffff700 1 -- 172.21.15.7:0/1620218745 >> v2:172.21.15.171:3300/0 conn(0x7fcdf815caf0 msgr2=0x7fcdf80ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:23.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdeffff700 1 --2- 172.21.15.7:0/1620218745 >> v2:172.21.15.171:3300/0 conn(0x7fcdf815caf0 0x7fcdf80ae3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdf6ffd700 1 -- 172.21.15.7:0/1620218745 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272991205 0 0) 0x7fcdf80cd110 con 0x7fcdf8071c80 2024-09-13T18:53:23.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdeffff700 1 -- 172.21.15.7:0/1620218745 >> v1:172.21.15.7:6789/0 conn(0x7fcdf8071c80 legacy=0x7fcdf80b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdeffff700 1 -- 172.21.15.7:0/1620218745 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf80ad1f0 con 0x7fcdf815ced0 2024-09-13T18:53:23.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdeffff700 1 --2- 172.21.15.7:0/1620218745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815ced0 0x7fcdf80ccbd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80ae140 tx=0x7fcde0009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac192adaa036f061 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:23.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdf6ffd700 1 -- 172.21.15.7:0/1620218745 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0011830 con 0x7fcdf815ced0 2024-09-13T18:53:23.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdf6ffd700 1 -- 172.21.15.7:0/1620218745 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde0011990 con 0x7fcdf815ced0 2024-09-13T18:53:23.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdf6ffd700 1 -- 172.21.15.7:0/1620218745 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0011d20 con 0x7fcdf815ced0 2024-09-13T18:53:23.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdfe0e4700 1 -- 172.21.15.7:0/1620218745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815ced0 msgr2=0x7fcdf80ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdfe0e4700 1 --2- 172.21.15.7:0/1620218745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815ced0 0x7fcdf80ccbd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80ae140 tx=0x7fcde0009b30 comp rx=0 tx=0).stop 2024-09-13T18:53:23.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdfe0e4700 1 -- 172.21.15.7:0/1620218745 shutdown_connections 2024-09-13T18:53:23.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdfe0e4700 1 --2- 172.21.15.7:0/1620218745 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815ced0 0x7fcdf80ccbd0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdfe0e4700 1 --2- 172.21.15.7:0/1620218745 >> v2:172.21.15.171:3300/0 conn(0x7fcdf815caf0 0x7fcdf80ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdfe0e4700 1 -- 172.21.15.7:0/1620218745 >> 172.21.15.7:0/1620218745 conn(0x7fcdf80b1810 msgr2=0x7fcdf80b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:23.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdfe0e4700 1 -- 172.21.15.7:0/1620218745 shutdown_connections 2024-09-13T18:53:23.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.668+0000 7fcdfe0e4700 1 -- 172.21.15.7:0/1620218745 wait complete. 2024-09-13T18:53:23.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.672+0000 7fcdfe0e4700 1 Processor -- start 2024-09-13T18:53:23.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.672+0000 7fcdfe0e4700 1 -- start start 2024-09-13T18:53:23.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.672+0000 7fcdfe0e4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf8071c80 0x7fcdf81517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.672+0000 7fcdfe0e4700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcdf815caf0 0x7fcdf8152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.672+0000 7fcdfe0e4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815ced0 0x7fcdf8160da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.672+0000 7fcdfe0e4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf8152f30 con 0x7fcdf815ced0 2024-09-13T18:53:23.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.672+0000 7fcdfe0e4700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf8153210 con 0x7fcdf8071c80 2024-09-13T18:53:23.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.672+0000 7fcdfe0e4700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf81534f0 con 0x7fcdf815caf0 2024-09-13T18:53:23.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.672+0000 7fcdf7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815ced0 0x7fcdf8160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.672+0000 7fcdf7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815ced0 0x7fcdf8160da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53314/0 (socket says 172.21.15.7:53314) 2024-09-13T18:53:23.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.672+0000 7fcdf7fff700 1 -- 172.21.15.7:0/2761887652 learned_addr learned my addr 172.21.15.7:0/2761887652 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:23.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.672+0000 7fcdf77fe700 1 --2- 172.21.15.7:0/2761887652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf8071c80 0x7fcdf81517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.672+0000 7fcdeffff700 1 --2- 172.21.15.7:0/2761887652 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcdf815caf0 0x7fcdf8152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.676+0000 7fcdf77fe700 1 -- 172.21.15.7:0/2761887652 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcdf815caf0 msgr2=0x7fcdf8152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.676+0000 7fcdf77fe700 1 --2- 172.21.15.7:0/2761887652 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcdf815caf0 0x7fcdf8152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.676+0000 7fcdf77fe700 1 -- 172.21.15.7:0/2761887652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815ced0 msgr2=0x7fcdf8160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.676+0000 7fcdf77fe700 1 --2- 172.21.15.7:0/2761887652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815ced0 0x7fcdf8160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.676+0000 7fcdf77fe700 1 -- 172.21.15.7:0/2761887652 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde00094b0 con 0x7fcdf8071c80 2024-09-13T18:53:23.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.676+0000 7fcdf7fff700 1 --2- 172.21.15.7:0/2761887652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815ced0 0x7fcdf8160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:23.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.676+0000 7fcdf77fe700 1 --2- 172.21.15.7:0/2761887652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf8071c80 0x7fcdf81517a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcde800e900 tx=0x7fcde800ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:23.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.676+0000 7fcdf4ff9700 1 -- 172.21.15.7:0/2761887652 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8013680 con 0x7fcdf8071c80 2024-09-13T18:53:23.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.676+0000 7fcdf4ff9700 1 -- 172.21.15.7:0/2761887652 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde80137e0 con 0x7fcdf8071c80 2024-09-13T18:53:23.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.676+0000 7fcdfe0e4700 1 -- 172.21.15.7:0/2761887652 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf81537d0 con 0x7fcdf8071c80 2024-09-13T18:53:23.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.676+0000 7fcdf4ff9700 1 -- 172.21.15.7:0/2761887652 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8013b50 con 0x7fcdf8071c80 2024-09-13T18:53:23.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.676+0000 7fcdfe0e4700 1 -- 172.21.15.7:0/2761887652 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf8152b70 con 0x7fcdf8071c80 2024-09-13T18:53:23.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.676+0000 7fcdf4ff9700 1 -- 172.21.15.7:0/2761887652 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcde8013cb0 con 0x7fcdf8071c80 2024-09-13T18:53:23.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fcdf4ff9700 1 --2- 172.21.15.7:0/2761887652 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcdd8066760 0x7fcdd8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fcdf4ff9700 1 -- 172.21.15.7:0/2761887652 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcde8084410 con 0x7fcdf8071c80 2024-09-13T18:53:23.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fcdeffff700 1 --2- 172.21.15.7:0/2761887652 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcdd8066760 0x7fcdd8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fce01395740 1 Processor -- start 2024-09-13T18:53:23.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fce01395740 1 -- start start 2024-09-13T18:53:23.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fcdeffff700 1 --2- 172.21.15.7:0/2761887652 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcdd8066760 0x7fcdd8068c20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80ae140 tx=0x7fcde0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:23.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fce01395740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x112bda0 0x112c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fce01395740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x110ad00 0x10f4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fce01395740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x112c6c0 0x10f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fce01395740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x1053d80 con 0x110ad00 2024-09-13T18:53:23.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fce01395740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x1051510 con 0x112bda0 2024-09-13T18:53:23.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fce01395740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x104ce00 con 0x112c6c0 2024-09-13T18:53:23.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fcdeffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x110ad00 0x10f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fcdf77fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x112bda0 0x112c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fcdeffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x110ad00 0x10f4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53320/0 (socket says 172.21.15.7:53320) 2024-09-13T18:53:23.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fcdeffff700 1 -- 172.21.15.7:0/139575973 learned_addr learned my addr 172.21.15.7:0/139575973 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:23.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fcdf7fff700 1 --2- 172.21.15.7:0/139575973 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x112c6c0 0x10f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fcdeffff700 1 -- 172.21.15.7:0/139575973 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x112c6c0 msgr2=0x10f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fcdeffff700 1 --2- 172.21.15.7:0/139575973 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x112c6c0 0x10f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fcdeffff700 1 -- 172.21.15.7:0/139575973 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x112bda0 msgr2=0x112c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fcdeffff700 1 --2- 172.21.15.7:0/139575973 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x112bda0 0x112c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fcdeffff700 1 -- 172.21.15.7:0/139575973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xebf960 con 0x110ad00 2024-09-13T18:53:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.680+0000 7fcdeffff700 1 --2- 172.21.15.7:0/139575973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x110ad00 0x10f4890 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcde0000c00 tx=0x7fcde0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9071d62004ce3b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:23.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fcded7fa700 1 -- 172.21.15.7:0/139575973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde002b070 con 0x110ad00 2024-09-13T18:53:23.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fcded7fa700 1 -- 172.21.15.7:0/139575973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde00036c0 con 0x110ad00 2024-09-13T18:53:23.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fcded7fa700 1 -- 172.21.15.7:0/139575973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde001f070 con 0x110ad00 2024-09-13T18:53:23.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fce01395740 1 -- 172.21.15.7:0/139575973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x110ad00 msgr2=0x10f4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fce01395740 1 --2- 172.21.15.7:0/139575973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x110ad00 0x10f4890 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcde0000c00 tx=0x7fcde0010000 comp rx=0 tx=0).stop 2024-09-13T18:53:23.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fce01395740 1 -- 172.21.15.7:0/139575973 shutdown_connections 2024-09-13T18:53:23.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fce01395740 1 --2- 172.21.15.7:0/139575973 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x112c6c0 0x10f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fce01395740 1 --2- 172.21.15.7:0/139575973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x110ad00 0x10f4890 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fce01395740 1 --2- 172.21.15.7:0/139575973 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x112bda0 0x112c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fce01395740 1 -- 172.21.15.7:0/139575973 >> 172.21.15.7:0/139575973 conn(0xe744d0 msgr2=0x110dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:23.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fce01395740 1 -- 172.21.15.7:0/139575973 shutdown_connections 2024-09-13T18:53:23.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fce01395740 1 -- 172.21.15.7:0/139575973 wait complete. 2024-09-13T18:53:23.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fce01395740 1 -- start start 2024-09-13T18:53:23.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fce01395740 1 Processor -- start 2024-09-13T18:53:23.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fce01395740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x110ad00 0x112c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fce01395740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x112bda0 0x112caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fce01395740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x112c6c0 0x1106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fce01395740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x100a730 con 0x112bda0 2024-09-13T18:53:23.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fce01395740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x10105b0 con 0x110ad00 2024-09-13T18:53:23.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fce01395740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0xef8aa0 con 0x112c6c0 2024-09-13T18:53:23.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fce01395740 10 client.0 _ll_register_callbacks cb 0x1046610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:53:23.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fcdeffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x112bda0 0x112caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fcdeffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x112bda0 0x112caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53330/0 (socket says 172.21.15.7:53330) 2024-09-13T18:53:23.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fcdeffff700 1 -- 172.21.15.7:0/3095501507 learned_addr learned my addr 172.21.15.7:0/3095501507 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:23.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fcdf77fe700 1 --2- 172.21.15.7:0/3095501507 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x110ad00 0x112c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fcdf7fff700 1 --2- 172.21.15.7:0/3095501507 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x112c6c0 0x1106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fcdeffff700 1 -- 172.21.15.7:0/3095501507 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x112c6c0 msgr2=0x1106fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:23.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fcdeffff700 1 --2- 172.21.15.7:0/3095501507 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x112c6c0 0x1106fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fcdeffff700 1 -- 172.21.15.7:0/3095501507 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x110ad00 msgr2=0x112c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:23.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.684+0000 7fcdeffff700 1 --2- 172.21.15.7:0/3095501507 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x110ad00 0x112c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fcdeffff700 1 -- 172.21.15.7:0/3095501507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf80ad1f0 con 0x112bda0 2024-09-13T18:53:23.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fcdeffff700 1 --2- 172.21.15.7:0/3095501507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x112bda0 0x112caa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcde0004740 tx=0x7fcde00047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2a19d4e3642939 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:23.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fcdd67fc700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-13T18:53:23.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fcdd67fc700 1 -- 172.21.15.7:0/3095501507 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde002b040 con 0x112bda0 2024-09-13T18:53:23.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fcdd67fc700 1 -- 172.21.15.7:0/3095501507 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcde0003d80 con 0x112bda0 2024-09-13T18:53:23.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 1 -- 172.21.15.7:0/3095501507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xe811e0 con 0x112bda0 2024-09-13T18:53:23.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fcdd67fc700 1 -- 172.21.15.7:0/3095501507 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcde0030470 con 0x112bda0 2024-09-13T18:53:23.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 10 client.5418 fetch_fsmap learned FSMap version 16 2024-09-13T18:53:23.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 1 -- 172.21.15.7:0/3095501507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x1013220 con 0x112bda0 2024-09-13T18:53:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fcdd67fc700 1 -- 172.21.15.7:0/3095501507 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xe811e0 con 0x112bda0 2024-09-13T18:53:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fcdd67fc700 1 -- 172.21.15.7:0/3095501507 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcde0037070 con 0x112bda0 2024-09-13T18:53:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 10 client.5418 fetch_fsmap finished waiting for FSMap version 16 2024-09-13T18:53:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 10 client.5418 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T18:53:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 20 client.5418 populate_metadata read hostname 'smithi007' 2024-09-13T18:53:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 1 --2- 172.21.15.7:0/3095501507 >> [v2:172.21.15.7:6834/2843879273,v1:172.21.15.7:6835/2843879273] conn(0x10e5560 0x10e5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 4 client.5418 mds_command: new command op to 5232 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:53:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 1 -- 172.21.15.7:0/3095501507 --> [v2:172.21.15.7:6834/2843879273,v1:172.21.15.7:6835/2843879273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xecb460 con 0x10e5560 2024-09-13T18:53:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fcdf77fe700 1 --2- 172.21.15.7:0/3095501507 >> [v2:172.21.15.7:6834/2843879273,v1:172.21.15.7:6835/2843879273] conn(0x10e5560 0x10e5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fcdf77fe700 1 --2- 172.21.15.7:0/3095501507 >> [v2:172.21.15.7:6834/2843879273,v1:172.21.15.7:6835/2843879273] conn(0x10e5560 0x10e5940 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5028486548b236a9 server_cookie=b455938eeb419a81 in_seq=0 out_seq=0 2024-09-13T18:53:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fcdd67fc700 10 client.5418 ms_handle_connect on v2:172.21.15.7:6834/2843879273 2024-09-13T18:53:23.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fcdd67fc700 1 -- 172.21.15.7:0/3095501507 <== mds.0 v2:172.21.15.7:6834/2843879273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0xecb460 con 0x10e5560 2024-09-13T18:53:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fcdd67fc700 10 client.5418 handle_command_reply: tid=0 2024-09-13T18:53:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 1 client.5418 shutdown 2024-09-13T18:53:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 1 -- 172.21.15.7:0/3095501507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x112bda0 msgr2=0x112caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 1 --2- 172.21.15.7:0/3095501507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x112bda0 0x112caa0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcde0004740 tx=0x7fcde00047e0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 1 -- 172.21.15.7:0/3095501507 shutdown_connections 2024-09-13T18:53:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 1 --2- 172.21.15.7:0/3095501507 >> [v2:172.21.15.7:6834/2843879273,v1:172.21.15.7:6835/2843879273] conn(0x10e5560 0x10e5940 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 1 --2- 172.21.15.7:0/3095501507 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x112c6c0 0x1106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 1 --2- 172.21.15.7:0/3095501507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x112bda0 0x112caa0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 1 --2- 172.21.15.7:0/3095501507 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x110ad00 0x112c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fcdd67fc700 0 client.5418 ms_handle_reset on v2:172.21.15.7:6834/2843879273 2024-09-13T18:53:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 1 -- 172.21.15.7:0/3095501507 >> 172.21.15.7:0/3095501507 conn(0xe744d0 msgr2=0x1107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:23.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 1 -- 172.21.15.7:0/3095501507 shutdown_connections 2024-09-13T18:53:23.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.688+0000 7fce01395740 1 -- 172.21.15.7:0/3095501507 wait complete. 2024-09-13T18:53:23.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.692+0000 7fce01395740 20 client.5418 trim_cache size 0 max 16384 2024-09-13T18:53:23.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.712+0000 7fce01395740 1 Processor -- start 2024-09-13T18:53:23.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.712+0000 7fce01395740 1 -- start start 2024-09-13T18:53:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.712+0000 7fce01395740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x110ad00 0x112b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.712+0000 7fce01395740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x112bda0 0x1142a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.712+0000 7fce01395740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x112c6c0 0x1142f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.712+0000 7fce01395740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0xff3950 con 0x112c6c0 2024-09-13T18:53:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.712+0000 7fce01395740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x104c610 con 0x112bda0 2024-09-13T18:53:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.712+0000 7fce01395740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0xff5020 con 0x110ad00 2024-09-13T18:53:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fcdf7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x112c6c0 0x1142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fcdf7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x112c6c0 0x1142f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53344/0 (socket says 172.21.15.7:53344) 2024-09-13T18:53:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fcdf7fff700 1 -- 172.21.15.7:0/3188330401 learned_addr learned my addr 172.21.15.7:0/3188330401 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fcdeffff700 1 --2- 172.21.15.7:0/3188330401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x112bda0 0x1142a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fcdf77fe700 1 --2- 172.21.15.7:0/3188330401 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x110ad00 0x112b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fcdf7fff700 1 -- 172.21.15.7:0/3188330401 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x110ad00 msgr2=0x112b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:23.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fcdf7fff700 1 --2- 172.21.15.7:0/3188330401 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x110ad00 0x112b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fcdf7fff700 1 -- 172.21.15.7:0/3188330401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x112bda0 msgr2=0x1142a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:23.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fcdf7fff700 1 --2- 172.21.15.7:0/3188330401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x112bda0 0x1142a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fcdf7fff700 1 -- 172.21.15.7:0/3188330401 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xff9450 con 0x112c6c0 2024-09-13T18:53:23.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fcdf7fff700 1 --2- 172.21.15.7:0/3188330401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x112c6c0 0x1142f80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00076e0 tx=0x7fcdf0001bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f71bd5e2afa6bd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:23.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fcdedffb700 1 -- 172.21.15.7:0/3188330401 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0015a30 con 0x112c6c0 2024-09-13T18:53:23.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fcdedffb700 1 -- 172.21.15.7:0/3188330401 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf0015b90 con 0x112c6c0 2024-09-13T18:53:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fcdedffb700 1 -- 172.21.15.7:0/3188330401 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0015d00 con 0x112c6c0 2024-09-13T18:53:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fce01395740 1 -- 172.21.15.7:0/3188330401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x112c6c0 msgr2=0x1142f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fce01395740 1 --2- 172.21.15.7:0/3188330401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x112c6c0 0x1142f80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00076e0 tx=0x7fcdf0001bb0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fce01395740 1 -- 172.21.15.7:0/3188330401 shutdown_connections 2024-09-13T18:53:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fce01395740 1 --2- 172.21.15.7:0/3188330401 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x112c6c0 0x1142f80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fce01395740 1 --2- 172.21.15.7:0/3188330401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x112bda0 0x1142a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fce01395740 1 --2- 172.21.15.7:0/3188330401 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x110ad00 0x112b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fce01395740 1 -- 172.21.15.7:0/3188330401 >> 172.21.15.7:0/3188330401 conn(0xe744d0 msgr2=0x10e2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fce01395740 1 -- 172.21.15.7:0/3188330401 shutdown_connections 2024-09-13T18:53:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fce01395740 1 -- 172.21.15.7:0/3188330401 wait complete. 2024-09-13T18:53:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.716+0000 7fce01395740 1 -- start start 2024-09-13T18:53:23.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fce01395740 1 Processor -- start 2024-09-13T18:53:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fce01395740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x110ad00 0x1194bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fce01395740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x112bda0 0x110a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fce01395740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x112c6c0 0x110a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fce01395740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x1153350 con 0x110ad00 2024-09-13T18:53:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fce01395740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x10e3760 con 0x112c6c0 2024-09-13T18:53:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fce01395740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x10453d0 con 0x112bda0 2024-09-13T18:53:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fcdf77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x110ad00 0x1194bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fcdf77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x110ad00 0x1194bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53352/0 (socket says 172.21.15.7:53352) 2024-09-13T18:53:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fce01395740 10 client.0 _ll_register_callbacks cb 0x10a3710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:53:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fcdf77fe700 1 -- 172.21.15.7:0/3484479563 learned_addr learned my addr 172.21.15.7:0/3484479563 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fcdeffff700 1 --2- 172.21.15.7:0/3484479563 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x112bda0 0x110a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fcdf7fff700 1 --2- 172.21.15.7:0/3484479563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x112c6c0 0x110a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fcdf77fe700 1 -- 172.21.15.7:0/3484479563 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x112bda0 msgr2=0x110a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:23.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fcdf77fe700 1 --2- 172.21.15.7:0/3484479563 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x112bda0 0x110a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fcdf77fe700 1 -- 172.21.15.7:0/3484479563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x112c6c0 msgr2=0x110a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:23.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fcdf77fe700 1 --2- 172.21.15.7:0/3484479563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x112c6c0 0x110a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fcdf77fe700 1 -- 172.21.15.7:0/3484479563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf0007360 con 0x110ad00 2024-09-13T18:53:23.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fcdf7fff700 1 --2- 172.21.15.7:0/3484479563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x112c6c0 0x110a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fcdf77fe700 1 --2- 172.21.15.7:0/3484479563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x110ad00 0x1194bf0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcde8082930 tx=0x7fcde80517a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d587ce47b7d3e455 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fcdd67fc700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-13T18:53:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fcdd67fc700 1 -- 172.21.15.7:0/3484479563 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde805d7a0 con 0x110ad00 2024-09-13T18:53:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fcdd67fc700 1 -- 172.21.15.7:0/3484479563 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcde8051ac0 con 0x110ad00 2024-09-13T18:53:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.720+0000 7fce01395740 1 -- 172.21.15.7:0/3484479563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcde00036c0 con 0x110ad00 2024-09-13T18:53:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdd67fc700 1 -- 172.21.15.7:0/3484479563 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcde8051c80 con 0x110ad00 2024-09-13T18:53:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 10 client.5424 fetch_fsmap learned FSMap version 16 2024-09-13T18:53:23.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 1 -- 172.21.15.7:0/3484479563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0xfed720 con 0x110ad00 2024-09-13T18:53:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdd67fc700 1 -- 172.21.15.7:0/3484479563 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde00036c0 con 0x110ad00 2024-09-13T18:53:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdd67fc700 1 -- 172.21.15.7:0/3484479563 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcde80840b0 con 0x110ad00 2024-09-13T18:53:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 10 client.5424 fetch_fsmap finished waiting for FSMap version 16 2024-09-13T18:53:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 10 client.5424 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T18:53:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 20 client.5424 populate_metadata read hostname 'smithi007' 2024-09-13T18:53:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 1 --2- 172.21.15.7:0/3484479563 >> [v2:172.21.15.7:6834/2843879273,v1:172.21.15.7:6835/2843879273] conn(0x11422a0 0x1142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 4 client.5424 mds_command: new command op to 5232 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:53:23.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 1 -- 172.21.15.7:0/3484479563 --> [v2:172.21.15.7:6834/2843879273,v1:172.21.15.7:6835/2843879273] -- command(tid 0: {"prefix": "status"}) v1 -- 0xef01e0 con 0x11422a0 2024-09-13T18:53:23.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdeffff700 1 --2- 172.21.15.7:0/3484479563 >> [v2:172.21.15.7:6834/2843879273,v1:172.21.15.7:6835/2843879273] conn(0x11422a0 0x1142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdd67fc700 10 client.5424 ms_handle_connect on v2:172.21.15.7:6834/2843879273 2024-09-13T18:53:23.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdeffff700 1 --2- 172.21.15.7:0/3484479563 >> [v2:172.21.15.7:6834/2843879273,v1:172.21.15.7:6835/2843879273] conn(0x11422a0 0x1142680 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e071944cffbfd66 server_cookie=208d0279910a5a8 in_seq=0 out_seq=0 2024-09-13T18:53:23.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdd67fc700 1 -- 172.21.15.7:0/3484479563 <== mds.0 v2:172.21.15.7:6834/2843879273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0xef01e0 con 0x11422a0 2024-09-13T18:53:23.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdd67fc700 10 client.5424 handle_command_reply: tid=0 2024-09-13T18:53:23.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 1 client.5424 shutdown 2024-09-13T18:53:23.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 1 -- 172.21.15.7:0/3484479563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x110ad00 msgr2=0x1194bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 1 --2- 172.21.15.7:0/3484479563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x110ad00 0x1194bf0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcde8082930 tx=0x7fcde80517a0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 1 -- 172.21.15.7:0/3484479563 shutdown_connections 2024-09-13T18:53:23.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 1 --2- 172.21.15.7:0/3484479563 >> [v2:172.21.15.7:6834/2843879273,v1:172.21.15.7:6835/2843879273] conn(0x11422a0 0x1142680 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 1 --2- 172.21.15.7:0/3484479563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x112c6c0 0x110a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 1 --2- 172.21.15.7:0/3484479563 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x112bda0 0x110a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdd67fc700 0 client.5424 ms_handle_reset on v2:172.21.15.7:6834/2843879273 2024-09-13T18:53:23.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 1 --2- 172.21.15.7:0/3484479563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x110ad00 0x1194bf0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 1 -- 172.21.15.7:0/3484479563 >> 172.21.15.7:0/3484479563 conn(0xe744d0 msgr2=0x112ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:23.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 1 -- 172.21.15.7:0/3484479563 shutdown_connections 2024-09-13T18:53:23.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 1 -- 172.21.15.7:0/3484479563 wait complete. 2024-09-13T18:53:23.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fce01395740 20 client.5424 trim_cache size 0 max 16384 2024-09-13T18:53:23.737 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:53:23.737 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "33406d92-f401-438a-876e-a56de3962d8e", 2024-09-13T18:53:23.737 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-13T18:53:23.737 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5232, 2024-09-13T18:53:23.738 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-13T18:53:23.738 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-13T18:53:23.738 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-13T18:53:23.738 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 5.8284053470000003, 2024-09-13T18:53:23.738 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 16, 2024-09-13T18:53:23.738 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 36, 2024-09-13T18:53:23.738 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 36, 2024-09-13T18:53:23.738 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 13.116596502 2024-09-13T18:53:23.738 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:53:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdfe0e4700 1 -- 172.21.15.7:0/2761887652 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcdd8066760 msgr2=0x7fcdd8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdfe0e4700 1 --2- 172.21.15.7:0/2761887652 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcdd8066760 0x7fcdd8068c20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80ae140 tx=0x7fcde0008030 comp rx=0 tx=0).stop 2024-09-13T18:53:23.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdfe0e4700 1 -- 172.21.15.7:0/2761887652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf8071c80 msgr2=0x7fcdf81517a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdfe0e4700 1 --2- 172.21.15.7:0/2761887652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf8071c80 0x7fcdf81517a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcde800e900 tx=0x7fcde800ec10 comp rx=0 tx=0).stop 2024-09-13T18:53:23.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdfe0e4700 1 -- 172.21.15.7:0/2761887652 shutdown_connections 2024-09-13T18:53:23.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdfe0e4700 1 --2- 172.21.15.7:0/2761887652 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcdd8066760 0x7fcdd8068c20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdfe0e4700 1 --2- 172.21.15.7:0/2761887652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf815ced0 0x7fcdf8160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdfe0e4700 1 --2- 172.21.15.7:0/2761887652 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcdf815caf0 0x7fcdf8152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdfe0e4700 1 --2- 172.21.15.7:0/2761887652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcdf8071c80 0x7fcdf81517a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdfe0e4700 1 -- 172.21.15.7:0/2761887652 >> 172.21.15.7:0/2761887652 conn(0x7fcdf80b1810 msgr2=0x7fcdf80b0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:23.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdfe0e4700 1 -- 172.21.15.7:0/2761887652 shutdown_connections 2024-09-13T18:53:23.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.724+0000 7fcdfe0e4700 1 -- 172.21.15.7:0/2761887652 wait complete. 2024-09-13T18:53:23.749 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T18:53:23.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.860+0000 7f17cd767700 1 Processor -- start 2024-09-13T18:53:23.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.860+0000 7f17cd767700 1 -- start start 2024-09-13T18:53:23.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.860+0000 7f17cd767700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17c806bb90 0x7f17c80cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.860+0000 7f17cd767700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c806a780 con 0x7f17c806fb30 2024-09-13T18:53:23.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.860+0000 7f17cd767700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c8159470 con 0x7f17c806d150 2024-09-13T18:53:23.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.860+0000 7f17cd767700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c806cd30 con 0x7f17c806bb90 2024-09-13T18:53:23.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.860+0000 7f17c77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17c806bb90 0x7f17c80cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.860+0000 7f17c7fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f17c806fb30 0x7f17c80adbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39224/0 (socket says 172.21.15.7:39224) 2024-09-13T18:53:23.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.860+0000 7f17c7fff700 1 -- 172.21.15.7:0/2943005465 learned_addr learned my addr 172.21.15.7:0/2943005465 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:23.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.864+0000 7f17c77fe700 1 -- 172.21.15.7:0/2943005465 >> v1:172.21.15.171:6789/0 conn(0x7f17c806d150 legacy=0x7f17c806d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.864+0000 7f17c77fe700 1 -- 172.21.15.7:0/2943005465 >> v1:172.21.15.7:6789/0 conn(0x7f17c806fb30 legacy=0x7f17c80adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.864+0000 7f17c77fe700 1 -- 172.21.15.7:0/2943005465 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c806c0b0 con 0x7f17c806bb90 2024-09-13T18:53:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.864+0000 7f17c77fe700 1 --2- 172.21.15.7:0/2943005465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c806bb90 0x7f17c80cbf00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009850 tx=0x7f17b0009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8604b2fca4f54780 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.864+0000 7f17c6ffd700 1 -- 172.21.15.7:0/2943005465 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b000f940 con 0x7f17c806bb90 2024-09-13T18:53:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.864+0000 7f17c6ffd700 1 -- 172.21.15.7:0/2943005465 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b000faa0 con 0x7f17c806bb90 2024-09-13T18:53:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.864+0000 7f17c6ffd700 1 -- 172.21.15.7:0/2943005465 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b000fe30 con 0x7f17c806bb90 2024-09-13T18:53:23.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.864+0000 7f17cd767700 1 -- 172.21.15.7:0/2943005465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c806bb90 msgr2=0x7f17c80cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.864+0000 7f17cd767700 1 --2- 172.21.15.7:0/2943005465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c806bb90 0x7f17c80cbf00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009850 tx=0x7f17b0009b60 comp rx=0 tx=0).stop 2024-09-13T18:53:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.864+0000 7f17cd767700 1 -- 172.21.15.7:0/2943005465 shutdown_connections 2024-09-13T18:53:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.864+0000 7f17cd767700 1 --2- 172.21.15.7:0/2943005465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c806bb90 0x7f17c80cbf00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.864+0000 7f17cd767700 1 -- 172.21.15.7:0/2943005465 >> 172.21.15.7:0/2943005465 conn(0x7f17c80b0f00 msgr2=0x7f17c80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.864+0000 7f17cd767700 1 -- 172.21.15.7:0/2943005465 shutdown_connections 2024-09-13T18:53:23.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.864+0000 7f17cd767700 1 -- 172.21.15.7:0/2943005465 wait complete. 2024-09-13T18:53:23.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.864+0000 7f17cd767700 1 Processor -- start 2024-09-13T18:53:23.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.864+0000 7f17cd767700 1 -- start start 2024-09-13T18:53:23.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17cd767700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17c806bb90 0x7f17c8150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17cd767700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17c806d150 0x7f17c8150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17cd767700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c806fb30 0x7f17c8150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17cd767700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c814ee10 con 0x7f17c806fb30 2024-09-13T18:53:23.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17cd767700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c814e4e0 con 0x7f17c806d150 2024-09-13T18:53:23.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17cd767700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c81512f0 con 0x7f17c806bb90 2024-09-13T18:53:23.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17ccf66700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c806fb30 0x7f17c8150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17ccf66700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c806fb30 0x7f17c8150db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53366/0 (socket says 172.21.15.7:53366) 2024-09-13T18:53:23.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17ccf66700 1 -- 172.21.15.7:0/52815371 learned_addr learned my addr 172.21.15.7:0/52815371 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:23.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17c77fe700 1 --2- 172.21.15.7:0/52815371 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17c806d150 0x7f17c8150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17c7fff700 1 --2- 172.21.15.7:0/52815371 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17c806bb90 0x7f17c8150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17ccf66700 1 -- 172.21.15.7:0/52815371 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17c806bb90 msgr2=0x7f17c8150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17ccf66700 1 --2- 172.21.15.7:0/52815371 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17c806bb90 0x7f17c8150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17ccf66700 1 -- 172.21.15.7:0/52815371 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17c806d150 msgr2=0x7f17c8150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17ccf66700 1 --2- 172.21.15.7:0/52815371 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17c806d150 0x7f17c8150870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17ccf66700 1 -- 172.21.15.7:0/52815371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b00094b0 con 0x7f17c806fb30 2024-09-13T18:53:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17ccf66700 1 --2- 172.21.15.7:0/52815371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c806fb30 0x7f17c8150db0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f17c000cc80 tx=0x7f17c000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17c4ff9700 1 -- 172.21.15.7:0/52815371 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0003c20 con 0x7f17c806fb30 2024-09-13T18:53:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17cd767700 1 -- 172.21.15.7:0/52815371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c814e750 con 0x7f17c806fb30 2024-09-13T18:53:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17c4ff9700 1 -- 172.21.15.7:0/52815371 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c0003d80 con 0x7f17c806fb30 2024-09-13T18:53:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17c4ff9700 1 -- 172.21.15.7:0/52815371 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c000aaa0 con 0x7f17c806fb30 2024-09-13T18:53:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17cd767700 1 -- 172.21.15.7:0/52815371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c81ee190 con 0x7f17c806fb30 2024-09-13T18:53:23.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.868+0000 7f17cd767700 1 -- 172.21.15.7:0/52815371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c814ef50 con 0x7f17c806fb30 2024-09-13T18:53:23.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.872+0000 7f17c4ff9700 1 -- 172.21.15.7:0/52815371 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f17c000ac00 con 0x7f17c806fb30 2024-09-13T18:53:23.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.872+0000 7f17c4ff9700 1 --2- 172.21.15.7:0/52815371 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f17a8066730 0x7f17a8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:23.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.872+0000 7f17c4ff9700 1 -- 172.21.15.7:0/52815371 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17c0084a60 con 0x7f17c806fb30 2024-09-13T18:53:23.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.872+0000 7f17c7fff700 1 --2- 172.21.15.7:0/52815371 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f17a8066730 0x7f17a8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:23.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.872+0000 7f17c7fff700 1 --2- 172.21.15.7:0/52815371 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f17a8066730 0x7f17a8068bf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f17b8005930 tx=0x7f17b8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:23.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:23.880+0000 7f17c4ff9700 1 -- 172.21.15.7:0/52815371 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f17c0051dc0 con 0x7f17c806fb30 2024-09-13T18:53:24.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:24.088+0000 7f17cd767700 1 -- 172.21.15.7:0/52815371 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f17c814ef50 con 0x7f17c806fb30 2024-09-13T18:53:24.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:24.088+0000 7f17c4ff9700 1 -- 172.21.15.7:0/52815371 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f17c814ef50 con 0x7f17c806fb30 2024-09-13T18:53:24.097 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T18:53:24.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:24.092+0000 7f17cd767700 1 -- 172.21.15.7:0/52815371 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f17a8066730 msgr2=0x7f17a8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:24.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:24.092+0000 7f17cd767700 1 --2- 172.21.15.7:0/52815371 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f17a8066730 0x7f17a8068bf0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f17b8005930 tx=0x7f17b8009040 comp rx=0 tx=0).stop 2024-09-13T18:53:24.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:24.092+0000 7f17cd767700 1 -- 172.21.15.7:0/52815371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c806fb30 msgr2=0x7f17c8150db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:24.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:24.092+0000 7f17cd767700 1 --2- 172.21.15.7:0/52815371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c806fb30 0x7f17c8150db0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f17c000cc80 tx=0x7f17c000cf90 comp rx=0 tx=0).stop 2024-09-13T18:53:24.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:24.092+0000 7f17cd767700 1 -- 172.21.15.7:0/52815371 shutdown_connections 2024-09-13T18:53:24.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:24.092+0000 7f17cd767700 1 --2- 172.21.15.7:0/52815371 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f17a8066730 0x7f17a8068bf0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:24.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:24.092+0000 7f17cd767700 1 --2- 172.21.15.7:0/52815371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17c806fb30 0x7f17c8150db0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:24.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:24.092+0000 7f17cd767700 1 --2- 172.21.15.7:0/52815371 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17c806d150 0x7f17c8150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:24.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:24.092+0000 7f17cd767700 1 --2- 172.21.15.7:0/52815371 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17c806bb90 0x7f17c8150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:24.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:24.092+0000 7f17cd767700 1 -- 172.21.15.7:0/52815371 >> 172.21.15.7:0/52815371 conn(0x7f17c80b0f00 msgr2=0x7f17c80cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:24.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:24.092+0000 7f17cd767700 1 -- 172.21.15.7:0/52815371 shutdown_connections 2024-09-13T18:53:24.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:24.092+0000 7f17cd767700 1 -- 172.21.15.7:0/52815371 wait complete. 2024-09-13T18:53:24.112 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T18:53:24.112 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T18:53:24.112 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-13T18:53:24.113 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T18:53:24.113 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T18:53:24.113 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T18:53:24.113 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T18:53:24.113 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T18:53:24.113 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T18:53:24.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:24.113 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-13T18:53:24.118 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T18:53:24.119 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T18:53:24.119 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T18:53:24.119 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:53:24.119 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-13T18:53:24.119 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:53:24.119 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:53:24.119 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-13T18:53:24.119 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:53:24.119 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-13T18:53:24.119 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T18:53:24.119 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-13T18:53:24.119 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T18:53:24.119 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42508sec preferred_lft 42508sec 2024-09-13T18:53:24.120 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-13T18:53:24.120 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:53:24.120 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:53:24.120 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-13T18:53:24.120 INFO:teuthology.orchestra.run.smithi007.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T18:53:24.120 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 02:42:8e:73:06:85 brd ff:ff:ff:ff:ff:ff 2024-09-13T18:53:24.120 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T18:53:24.120 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:53:24.120 INFO:teuthology.orchestra.run.smithi007.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T18:53:24.120 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 72:5b:90:5f:a5:2f brd ff:ff:ff:ff:ff:ff 2024-09-13T18:53:24.120 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T18:53:24.120 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:53:24.120 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::9896:a1ff:fe28:24ca/64 scope link 2024-09-13T18:53:24.121 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:53:24.121 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T18:53:24.121 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 72:5b:90:5f:a5:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T18:53:24.121 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::705b:90ff:fe5f:a52f/64 scope link 2024-09-13T18:53:24.121 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:53:24.121 INFO:teuthology.orchestra.run.smithi007.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T18:53:24.121 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ea:d0:e0:a8:33:f2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T18:53:24.121 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::e8d0:e0ff:fea8:33f2/64 scope link 2024-09-13T18:53:24.121 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:53:24.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:24.121 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T18:53:24.135 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-13T18:53:24.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:24.137 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-13T18:53:24.145 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-13T18:53:24.145 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T18:53:24.145 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T18:53:24.145 INFO:teuthology.orchestra.run.smithi007.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T18:53:24.145 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T18:53:24.145 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T18:53:24.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:24.145 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-09-13T18:53:24.145 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-13T18:53:24.146 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T18:53:24.146 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T18:53:24.146 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T18:53:24.146 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-13T18:53:24.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:24.228 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-13T18:53:24.235 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-13T18:53:24.235 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T18:53:24.236 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T18:53:24.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:24.236 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-13T18:53:24.249 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-13T18:53:24.249 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:53:24.249 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T18:53:24.254 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T18:53:24.254 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:53:24.254 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T18:53:24.303 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-09-13T18:53:24.321 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:53:24.321 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:53:24.332 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:53:24.333 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:53:24.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:24.334 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-09-13T18:53:24.339 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T18:53:24.339 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-13T18:53:24.383 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-09-13T18:53:24.400 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:53:24.401 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:53:24.410 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:53:24.411 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:53:24.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:24.412 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-09-13T18:53:24.473 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:2024-09-13T18:53:24.464+0000 7f7f38176640 -1 init, newargv = 0x55ef76e7b870 newargc=15 2024-09-13T18:53:24.473 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[35609]: starting ceph client 2024-09-13T18:53:24.482 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[35609]: starting fuse 2024-09-13T18:53:25.518 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-09-13T18:53:25.539 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:53:25.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:53:25.596 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:53:25.597 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:53:25.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:25.598 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-09-13T18:53:25.605 INFO:teuthology.orchestra.run.smithi007.stdout:52 2024-09-13T18:53:25.605 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-09-13T18:53:25.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:53:25.606 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2024-09-13T18:53:25.606 DEBUG:teuthology.orchestra.run.smithi007:> import re 2024-09-13T18:53:25.606 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-13T18:53:25.606 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-09-13T18:53:25.606 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:53:25.606 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2024-09-13T18:53:25.606 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T18:53:25.606 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2024-09-13T18:53:25.606 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T18:53:25.606 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:53:25.606 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2024-09-13T18:53:25.606 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2024-09-13T18:53:25.606 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2024-09-13T18:53:25.607 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2024-09-13T18:53:25.607 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:53:25.607 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2024-09-13T18:53:25.607 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T18:53:25.607 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T18:53:25.607 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T18:53:25.607 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2024-09-13T18:53:25.607 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2024-09-13T18:53:25.607 DEBUG:teuthology.orchestra.run.smithi007:> return f 2024-09-13T18:53:25.607 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T18:53:25.607 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:53:25.607 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.0")) 2024-09-13T18:53:25.607 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-13T18:53:25.693 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.0.35609.asok 2024-09-13T18:53:25.876 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35609.asok 2024-09-13T18:53:25.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:25.876 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35609.asok status 2024-09-13T18:53:25.979 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:53:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2024-09-13T18:53:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T18:53:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T18:53:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "0", 2024-09-13T18:53:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2024-09-13T18:53:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T18:53:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "35609", 2024-09-13T18:53:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2024-09-13T18:53:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:25.980 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2024-09-13T18:53:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2024-09-13T18:53:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5436, 2024-09-13T18:53:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2024-09-13T18:53:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2024-09-13T18:53:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-09-13T18:53:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 5436 2024-09-13T18:53:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-09-13T18:53:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-13T18:53:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2024-09-13T18:53:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2188038762 2024-09-13T18:53:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:53:25.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-09-13T18:53:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-13T18:53:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2024-09-13T18:53:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2188038762 2024-09-13T18:53:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:53:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.5436 192.168.0.1:0/2188038762", 2024-09-13T18:53:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.1:0/2188038762", 2024-09-13T18:53:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2024-09-13T18:53:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 16, 2024-09-13T18:53:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 36, 2024-09-13T18:53:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2024-09-13T18:53:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2024-09-13T18:53:25.982 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2024-09-13T18:53:25.983 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:53:25.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:25.989 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2024-09-13T18:53:25.990 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T18:53:25.995 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2024-09-13T18:53:25.996 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T18:53:25.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:25.996 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T18:53:26.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T18:53:26.010 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T18:53:26.040 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-09-13T18:53:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-09-13T18:53:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T18:53:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-09-13T18:53:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-09-13T18:53:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-09-13T18:53:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-09-13T18:53:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-09-13T18:53:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-09-13T18:53:26.041 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-09-13T18:53:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T18:53:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-09-13T18:53:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T18:53:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T18:53:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-09-13T18:53:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-09-13T18:53:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-09-13T18:53:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-09-13T18:53:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-09-13T18:53:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-09-13T18:53:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-09-13T18:53:26.042 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-09-13T18:53:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T18:53:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T18:53:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-09-13T18:53:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T18:53:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-09-13T18:53:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-09-13T18:53:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-09-13T18:53:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-09-13T18:53:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-09-13T18:53:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.171,[v2:172.21.15.171:3301,v1:172.21.15.171:6790]" 2024-09-13T18:53:26.043 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 33406d92-f401-438a-876e-a56de3962d8e 2024-09-13T18:53:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.044 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-09-13T18:53:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-09-13T18:53:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-09-13T18:53:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-09-13T18:53:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-09-13T18:53:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-09-13T18:53:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-09-13T18:53:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-09-13T18:53:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-09-13T18:53:26.044 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-09-13T18:53:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-09-13T18:53:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-09-13T18:53:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-09-13T18:53:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-09-13T18:53:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-09-13T18:53:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-09-13T18:53:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-09-13T18:53:26.045 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-09-13T18:53:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-09-13T18:53:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: bdev async discard = True 2024-09-13T18:53:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: bdev enable discard = True 2024-09-13T18:53:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore allocator = bitmap 2024-09-13T18:53:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-09-13T18:53:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-09-13T18:53:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 1/20 2024-09-13T18:53:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 1/20 2024-09-13T18:53:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:53:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-09-13T18:53:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 4/10 2024-09-13T18:53:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T18:53:26.046 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-09-13T18:53:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-09-13T18:53:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2024-09-13T18:53:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2024-09-13T18:53:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-09-13T18:53:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.047 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-09-13T18:53:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:53:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-09-13T18:53:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-13T18:53:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-13T18:53:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:53:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:53:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-09-13T18:53:26.047 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-09-13T18:53:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.048 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-09-13T18:53:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:53:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-13T18:53:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-09-13T18:53:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-13T18:53:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-09-13T18:53:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-09-13T18:53:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:53:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-09-13T18:53:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-09-13T18:53:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:53:26.048 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T18:53:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T18:53:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T18:53:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T18:53:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T18:53:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T18:53:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-09-13T18:53:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-09-13T18:53:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-09-13T18:53:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: mon op complaint time = 120 2024-09-13T18:53:26.049 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on pool no app = False 2024-09-13T18:53:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.050 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-09-13T18:53:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-09-13T18:53:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-09-13T18:53:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-09-13T18:53:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T18:53:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T18:53:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 600 2024-09-13T18:53:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-09-13T18:53:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:53:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = "" 2024-09-13T18:53:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-09-13T18:53:26.050 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-09-13T18:53:26.050 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-09-13T18:53:26.051 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-09-13T18:53:26.051 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-09-13T18:53:26.051 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-09-13T18:53:26.051 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-09-13T18:53:26.051 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-09-13T18:53:26.051 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-09-13T18:53:26.051 INFO:teuthology.orchestra.run.smithi007.stdout:[mds] 2024-09-13T18:53:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds = 20 2024-09-13T18:53:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds balancer = 20 2024-09-13T18:53:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:53:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug frag = True 2024-09-13T18:53:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug scatterstat = True 2024-09-13T18:53:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: mds op complaint time = 180 2024-09-13T18:53:26.051 INFO:teuthology.orchestra.run.smithi007.stdout: mds verify scatter = True 2024-09-13T18:53:26.052 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-09-13T18:53:26.052 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-09-13T18:53:26.052 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-09-13T18:53:26.052 INFO:teuthology.orchestra.run.smithi007.stdout: osd_deep_scrub_large_omap_object_key_threshold = 5 2024-09-13T18:53:26.052 INFO:teuthology.orchestra.run.smithi171.stdout:[global] 2024-09-13T18:53:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: chdir = "" 2024-09-13T18:53:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T18:53:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: auth supported = cephx 2024-09-13T18:53:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: filestore xattr use omap = true 2024-09-13T18:53:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: mon clock drift allowed = 1.000 2024-09-13T18:53:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: osd crush chooseleaf type = 0 2024-09-13T18:53:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: auth debug = true 2024-09-13T18:53:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on old message = true 2024-09-13T18:53:26.053 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on bug = true 2024-09-13T18:53:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T18:53:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: mon pg warn max object skew = 0 2024-09-13T18:53:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T18:53:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T18:53:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default size = 2 2024-09-13T18:53:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow primary affinity = true 2024-09-13T18:53:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow pg remap = true 2024-09-13T18:53:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on legacy crush tunables = false 2024-09-13T18:53:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on crush straw calc version zero = false 2024-09-13T18:53:26.054 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on no sortbitwise = false 2024-09-13T18:53:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on osd down out interval zero = false 2024-09-13T18:53:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on too few osds = false 2024-09-13T18:53:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T18:53:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T18:53:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: mon_allow_pool_size_one = true 2024-09-13T18:53:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T18:53:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: osd default data pool replay window = 5 2024-09-13T18:53:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: mon allow pool delete = true 2024-09-13T18:53:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: mon cluster log file level = debug 2024-09-13T18:53:26.055 INFO:teuthology.orchestra.run.smithi171.stdout: debug asserts on shutdown = true 2024-09-13T18:53:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: mon health detail to clog = false 2024-09-13T18:53:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: mon host = "172.21.15.7,172.21.15.171,[v2:172.21.15.171:3301,v1:172.21.15.171:6790]" 2024-09-13T18:53:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: fsid = 33406d92-f401-438a-876e-a56de3962d8e 2024-09-13T18:53:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.056 INFO:teuthology.orchestra.run.smithi171.stdout:[osd] 2024-09-13T18:53:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: osd journal size = 100 2024-09-13T18:53:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub load threshold = 5.0 2024-09-13T18:53:26.056 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub max interval = 600 2024-09-13T18:53:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: osd mclock profile = high_recovery_ops 2024-09-13T18:53:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: osd recover clone overlap = true 2024-09-13T18:53:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: osd recovery max chunk = 1048576 2024-09-13T18:53:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug shutdown = true 2024-09-13T18:53:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug op order = true 2024-09-13T18:53:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug verify stray on activate = true 2024-09-13T18:53:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: osd open classes on start = true 2024-09-13T18:53:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug pg log writeout = true 2024-09-13T18:53:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.057 INFO:teuthology.orchestra.run.smithi171.stdout: osd deep scrub update digest min age = 30 2024-09-13T18:53:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: osd map max advance = 10 2024-09-13T18:53:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: journal zero on create = true 2024-09-13T18:53:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: filestore ondisk finisher threads = 3 2024-09-13T18:53:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: filestore apply finisher threads = 3 2024-09-13T18:53:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: bdev debug aio = true 2024-09-13T18:53:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug misdirected ops = true 2024-09-13T18:53:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: bdev async discard = True 2024-09-13T18:53:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: bdev enable discard = True 2024-09-13T18:53:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore allocator = bitmap 2024-09-13T18:53:26.058 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore block size = 96636764160 2024-09-13T18:53:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore fsck on mount = True 2024-09-13T18:53:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluefs = 1/20 2024-09-13T18:53:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluestore = 1/20 2024-09-13T18:53:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:53:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: debug osd = 20 2024-09-13T18:53:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: debug rocksdb = 4/10 2024-09-13T18:53:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T18:53:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd full ratio = 0.9 2024-09-13T18:53:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd nearfull ratio = 0.8 2024-09-13T18:53:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: osd failsafe full ratio = 0.95 2024-09-13T18:53:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: osd objectstore = bluestore 2024-09-13T18:53:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: osd op complaint time = 180 2024-09-13T18:53:26.059 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.059 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr] 2024-09-13T18:53:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:53:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: debug mgr = 20 2024-09-13T18:53:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-09-13T18:53:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-09-13T18:53:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:53:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:53:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: mgr/telemetry/nag = false 2024-09-13T18:53:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: debug client = 20 2024-09-13T18:53:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.060 INFO:teuthology.orchestra.run.smithi171.stdout:[mon] 2024-09-13T18:53:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:53:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-09-13T18:53:26.060 INFO:teuthology.orchestra.run.smithi171.stdout: debug paxos = 20 2024-09-13T18:53:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-09-13T18:53:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: mon data avail warn = 5 2024-09-13T18:53:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: mon mgr mkfs grace = 240 2024-09-13T18:53:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:53:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd reporter subtree level = osd 2024-09-13T18:53:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd prime pg temp = true 2024-09-13T18:53:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:53:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T18:53:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T18:53:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T18:53:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T18:53:26.061 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T18:53:26.062 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T18:53:26.062 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.062 INFO:teuthology.orchestra.run.smithi171.stdout: # 1m isn't quite enough 2024-09-13T18:53:26.062 INFO:teuthology.orchestra.run.smithi171.stdout: mon_down_mkfs_grace = 2m 2024-09-13T18:53:26.062 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.062 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_filestore_osds = false 2024-09-13T18:53:26.062 INFO:teuthology.orchestra.run.smithi171.stdout: mon op complaint time = 120 2024-09-13T18:53:26.062 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on pool no app = False 2024-09-13T18:53:26.062 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:53:26.062 INFO:teuthology.orchestra.run.smithi171.stdout:[client] 2024-09-13T18:53:26.062 INFO:teuthology.orchestra.run.smithi171.stdout: rgw cache enabled = true 2024-09-13T18:53:26.062 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable ops log = true 2024-09-13T18:53:26.062 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable usage log = true 2024-09-13T18:53:26.063 INFO:teuthology.orchestra.run.smithi171.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T18:53:26.063 INFO:teuthology.orchestra.run.smithi171.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T18:53:26.063 INFO:teuthology.orchestra.run.smithi171.stdout: client mount timeout = 600 2024-09-13T18:53:26.063 INFO:teuthology.orchestra.run.smithi171.stdout: debug client = 20 2024-09-13T18:53:26.063 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:53:26.063 INFO:teuthology.orchestra.run.smithi171.stdout: pid file = "" 2024-09-13T18:53:26.063 INFO:teuthology.orchestra.run.smithi171.stdout: rados mon op timeout = 900 2024-09-13T18:53:26.063 INFO:teuthology.orchestra.run.smithi171.stdout: rados osd op timeout = 900 2024-09-13T18:53:26.063 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.a] 2024-09-13T18:53:26.063 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.b] 2024-09-13T18:53:26.063 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.c] 2024-09-13T18:53:26.063 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.a] 2024-09-13T18:53:26.063 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.c] 2024-09-13T18:53:26.063 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.b] 2024-09-13T18:53:26.064 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.d] 2024-09-13T18:53:26.064 INFO:teuthology.orchestra.run.smithi171.stdout:[mds] 2024-09-13T18:53:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: debug mds = 20 2024-09-13T18:53:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: debug mds balancer = 20 2024-09-13T18:53:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:53:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: mds debug frag = True 2024-09-13T18:53:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: mds debug scatterstat = True 2024-09-13T18:53:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: mds op complaint time = 180 2024-09-13T18:53:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: mds verify scatter = True 2024-09-13T18:53:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: osd op complaint time = 180 2024-09-13T18:53:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: rados mon op timeout = 900 2024-09-13T18:53:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: rados osd op timeout = 900 2024-09-13T18:53:26.064 INFO:teuthology.orchestra.run.smithi171.stdout: osd_deep_scrub_large_omap_object_key_threshold = 5 2024-09-13T18:53:26.066 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T18:53:26.066 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T18:53:26.066 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T18:53:26.066 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:53:26.067 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T18:53:26.067 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-13T18:53:26.067 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T18:53:26.067 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:53:26.067 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T18:53:26.067 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T18:53:26.067 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T18:53:26.067 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:53:26.068 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T18:53:26.068 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-13T18:53:26.068 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T18:53:26.068 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:53:26.083 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T18:53:26.084 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T18:53:26.084 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T18:53:26.084 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T18:53:26.127 INFO:tasks.ceph.mds.d:Started 2024-09-13T18:53:26.127 DEBUG:teuthology.parallel:result is None 2024-09-13T18:53:26.128 INFO:tasks.ceph.mds.b:Started 2024-09-13T18:53:26.128 DEBUG:teuthology.parallel:result is None 2024-09-13T18:53:26.129 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T18:53:26.129 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T18:53:26.130 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T18:53:26.130 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T18:53:26.131 INFO:tasks.ceph.mds.a:Started 2024-09-13T18:53:26.131 DEBUG:teuthology.parallel:result is None 2024-09-13T18:53:26.133 INFO:tasks.ceph.mds.c:Started 2024-09-13T18:53:26.133 DEBUG:teuthology.parallel:result is None 2024-09-13T18:53:26.133 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:26.206 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-09-13T18:53:26.209 INFO:tasks.ceph.mds.b.smithi171.stdout:starting mds.b at 2024-09-13T18:53:26.210 INFO:tasks.ceph.mds.d.smithi171.stdout:starting mds.d at 2024-09-13T18:53:26.215 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-13T18:53:26.231 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-13T18:53:26.224+0000 7f28f0ff8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T18:53:26.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.240+0000 7f362bfff700 1 Processor -- start 2024-09-13T18:53:26.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.240+0000 7f362bfff700 1 -- start start 2024-09-13T18:53:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.244+0000 7f362bfff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f362c0b0610 0x7f362c0ce130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.244+0000 7f362bfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c0cea60 con 0x7f362c0cdcc0 2024-09-13T18:53:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.244+0000 7f362bfff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c0b4b10 con 0x7f362c1596b0 2024-09-13T18:53:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.244+0000 7f362bfff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362c0ce670 con 0x7f362c0b0610 2024-09-13T18:53:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.244+0000 7f362a7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f362c0cdcc0 0x7f362c0aff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39230/0 (socket says 172.21.15.7:39230) 2024-09-13T18:53:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.244+0000 7f362a7fc700 1 -- 172.21.15.7:0/3065195427 learned_addr learned my addr 172.21.15.7:0/3065195427 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:26.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.244+0000 7f362b7fe700 1 --2- 172.21.15.7:0/3065195427 >> v2:172.21.15.171:3300/0 conn(0x7f362c0b0610 0x7f362c0ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:26.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.244+0000 7f362b7fe700 1 -- 172.21.15.7:0/3065195427 >> v1:172.21.15.171:6789/0 conn(0x7f362c1596b0 legacy=0x7f362c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:26.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.244+0000 7f362b7fe700 1 -- 172.21.15.7:0/3065195427 >> v1:172.21.15.7:6789/0 conn(0x7f362c0cdcc0 legacy=0x7f362c0aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:26.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.244+0000 7f362b7fe700 1 -- 172.21.15.7:0/3065195427 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c0b4df0 con 0x7f362c0b0610 2024-09-13T18:53:26.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.244+0000 7f362b7fe700 1 --2- 172.21.15.7:0/3065195427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f362c0b0610 0x7f362c0ce130 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f361c009850 tx=0x7f361c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1145dfb03faa4de4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:26.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.244+0000 7f3629ffb700 1 -- 172.21.15.7:0/3065195427 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c011830 con 0x7f362c0b0610 2024-09-13T18:53:26.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.244+0000 7f3629ffb700 1 -- 172.21.15.7:0/3065195427 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c011990 con 0x7f362c0b0610 2024-09-13T18:53:26.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.244+0000 7f3629ffb700 1 -- 172.21.15.7:0/3065195427 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c011d20 con 0x7f362c0b0610 2024-09-13T18:53:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362bfff700 1 -- 172.21.15.7:0/3065195427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f362c0b0610 msgr2=0x7f362c0ce130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362bfff700 1 --2- 172.21.15.7:0/3065195427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f362c0b0610 0x7f362c0ce130 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f361c009850 tx=0x7f361c009b60 comp rx=0 tx=0).stop 2024-09-13T18:53:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362bfff700 1 -- 172.21.15.7:0/3065195427 shutdown_connections 2024-09-13T18:53:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362bfff700 1 --2- 172.21.15.7:0/3065195427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f362c0b0610 0x7f362c0ce130 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362bfff700 1 -- 172.21.15.7:0/3065195427 >> 172.21.15.7:0/3065195427 conn(0x7f362c0aeae0 msgr2=0x7f362c072140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362bfff700 1 -- 172.21.15.7:0/3065195427 shutdown_connections 2024-09-13T18:53:26.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362bfff700 1 -- 172.21.15.7:0/3065195427 wait complete. 2024-09-13T18:53:26.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362bfff700 1 Processor -- start 2024-09-13T18:53:26.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362bfff700 1 -- start start 2024-09-13T18:53:26.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f362c0b0610 0x7f362c1509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:26.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c0cdcc0 0x7f362c13e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:26.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362bfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f362c1596b0 0x7f362c06c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:26.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362bfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c14fd80 con 0x7f362c0cdcc0 2024-09-13T18:53:26.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362bfff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c1522d0 con 0x7f362c0b0610 2024-09-13T18:53:26.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362bfff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c152500 con 0x7f362c1596b0 2024-09-13T18:53:26.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c0cdcc0 0x7f362c13e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:26.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c0cdcc0 0x7f362c13e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53412/0 (socket says 172.21.15.7:53412) 2024-09-13T18:53:26.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362a7fc700 1 -- 172.21.15.7:0/1955852515 learned_addr learned my addr 172.21.15.7:0/1955852515 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:26.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362affd700 1 --2- 172.21.15.7:0/1955852515 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f362c0b0610 0x7f362c1509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:26.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362b7fe700 1 --2- 172.21.15.7:0/1955852515 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f362c1596b0 0x7f362c06c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:26.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362a7fc700 1 -- 172.21.15.7:0/1955852515 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f362c1596b0 msgr2=0x7f362c06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:26.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362a7fc700 1 --2- 172.21.15.7:0/1955852515 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f362c1596b0 0x7f362c06c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:26.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362a7fc700 1 -- 172.21.15.7:0/1955852515 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f362c0b0610 msgr2=0x7f362c1509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:26.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362a7fc700 1 --2- 172.21.15.7:0/1955852515 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f362c0b0610 0x7f362c1509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:26.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.248+0000 7f362a7fc700 1 -- 172.21.15.7:0/1955852515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c0094b0 con 0x7f362c0cdcc0 2024-09-13T18:53:26.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.252+0000 7f362a7fc700 1 --2- 172.21.15.7:0/1955852515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c0cdcc0 0x7f362c13e230 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3620001eb0 tx=0x7f362000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:26.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.252+0000 7f36137fe700 1 -- 172.21.15.7:0/1955852515 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620005d90 con 0x7f362c0cdcc0 2024-09-13T18:53:26.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.252+0000 7f36137fe700 1 -- 172.21.15.7:0/1955852515 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36200070d0 con 0x7f362c0cdcc0 2024-09-13T18:53:26.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.252+0000 7f362bfff700 1 -- 172.21.15.7:0/1955852515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c1527e0 con 0x7f362c0cdcc0 2024-09-13T18:53:26.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.252+0000 7f36137fe700 1 -- 172.21.15.7:0/1955852515 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620007440 con 0x7f362c0cdcc0 2024-09-13T18:53:26.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.252+0000 7f362bfff700 1 -- 172.21.15.7:0/1955852515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c152da0 con 0x7f362c0cdcc0 2024-09-13T18:53:26.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.252+0000 7f362bfff700 1 -- 172.21.15.7:0/1955852515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360c005310 con 0x7f362c0cdcc0 2024-09-13T18:53:26.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.252+0000 7f36137fe700 1 -- 172.21.15.7:0/1955852515 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f36200075a0 con 0x7f362c0cdcc0 2024-09-13T18:53:26.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.252+0000 7f36137fe700 1 --2- 172.21.15.7:0/1955852515 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3618066730 0x7f3618068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:26.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.256+0000 7f36137fe700 1 -- 172.21.15.7:0/1955852515 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3620083360 con 0x7f362c0cdcc0 2024-09-13T18:53:26.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.256+0000 7f362affd700 1 --2- 172.21.15.7:0/1955852515 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3618066730 0x7f3618068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:26.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.256+0000 7f362affd700 1 --2- 172.21.15.7:0/1955852515 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3618066730 0x7f3618068bf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f36240012e0 tx=0x7f3624009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:26.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.260+0000 7f36137fe700 1 -- 172.21.15.7:0/1955852515 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3620050720 con 0x7f362c0cdcc0 2024-09-13T18:53:26.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.464+0000 7f362bfff700 1 -- 172.21.15.7:0/1955852515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f360c0055c0 con 0x7f362c0cdcc0 2024-09-13T18:53:26.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.464+0000 7f36137fe700 1 -- 172.21.15.7:0/1955852515 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4732 (secure 0 0 0) 0x7f3620053d40 con 0x7f362c0cdcc0 2024-09-13T18:53:26.473 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:26.474 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2458672850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2458672850},{"type":"v1","addr":"172.21.15.171:6835","nonce":2458672850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3381042298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3381042298},{"type":"v1","addr":"172.21.15.171:6837","nonce":3381042298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2068823578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2068823578},{"type":"v1","addr":"172.21.15.7:6833","nonce":2068823578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:53:17.897317+0000","modified":"2024-09-13T18:53:26.239022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5463":{"gid":5463,"name":"c","rank":0,"incarnation":18,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6835/127797392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":127797392},{"type":"v1","addr":"172.21.15.7:6835","nonce":127797392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T18:53:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.468+0000 7f362bfff700 1 -- 172.21.15.7:0/1955852515 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3618066730 msgr2=0x7f3618068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.468+0000 7f362bfff700 1 --2- 172.21.15.7:0/1955852515 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3618066730 0x7f3618068bf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f36240012e0 tx=0x7f3624009040 comp rx=0 tx=0).stop 2024-09-13T18:53:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.468+0000 7f362bfff700 1 -- 172.21.15.7:0/1955852515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c0cdcc0 msgr2=0x7f362c13e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.468+0000 7f362bfff700 1 --2- 172.21.15.7:0/1955852515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c0cdcc0 0x7f362c13e230 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3620001eb0 tx=0x7f362000c7b0 comp rx=0 tx=0).stop 2024-09-13T18:53:26.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.468+0000 7f362bfff700 1 -- 172.21.15.7:0/1955852515 shutdown_connections 2024-09-13T18:53:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.468+0000 7f362bfff700 1 --2- 172.21.15.7:0/1955852515 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3618066730 0x7f3618068bf0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.468+0000 7f362bfff700 1 --2- 172.21.15.7:0/1955852515 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f362c1596b0 0x7f362c06c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.468+0000 7f362bfff700 1 --2- 172.21.15.7:0/1955852515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c0cdcc0 0x7f362c13e230 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.468+0000 7f362bfff700 1 --2- 172.21.15.7:0/1955852515 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f362c0b0610 0x7f362c1509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.468+0000 7f362bfff700 1 -- 172.21.15.7:0/1955852515 >> 172.21.15.7:0/1955852515 conn(0x7f362c0aeae0 msgr2=0x7f362c0b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.468+0000 7f362bfff700 1 -- 172.21.15.7:0/1955852515 shutdown_connections 2024-09-13T18:53:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:26.468+0000 7f362bfff700 1 -- 172.21.15.7:0/1955852515 wait complete. 2024-09-13T18:53:26.477 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 18 2024-09-13T18:53:26.891 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T18:53:17.897317+0000', 'modified': '2024-09-13T18:53:26.239022+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 39, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5463}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5463': {'gid': 5463, 'name': 'c', 'rank': 0, 'incarnation': 18, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.7:6835/127797392', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 127797392}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 127797392}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T18:53:26.891 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5463:up:replay 2024-09-13T18:53:27.892 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:28.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.008+0000 7f9905fa4700 1 Processor -- start 2024-09-13T18:53:28.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.008+0000 7f9905fa4700 1 -- start start 2024-09-13T18:53:28.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f9905fa4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f99000cdb70 0x7f99000cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f9905fa4700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f99000afaa0 0x7f99000cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f9905fa4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99000ad860 con 0x7f990015abb0 2024-09-13T18:53:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f9905fa4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99000b01a0 con 0x7f99000cdb70 2024-09-13T18:53:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f9905fa4700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99000aff50 con 0x7f99000afaa0 2024-09-13T18:53:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f98fffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f99000cdb70 0x7f99000cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f98fffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f99000cdb70 0x7f99000cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53418/0 (socket says 172.21.15.7:53418) 2024-09-13T18:53:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f9904fa2700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f990015abb0 0x7f99000cd460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39238/0 (socket says 172.21.15.7:39238) 2024-09-13T18:53:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f98fffff700 1 -- 172.21.15.7:0/1927203490 learned_addr learned my addr 172.21.15.7:0/1927203490 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f99057a3700 1 --2- 172.21.15.7:0/1927203490 >> v2:172.21.15.171:3300/0 conn(0x7f99000afaa0 0x7f99000cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f98fffff700 1 -- 172.21.15.7:0/1927203490 >> v2:172.21.15.171:3300/0 conn(0x7f99000afaa0 msgr2=0x7f99000cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f98fffff700 1 --2- 172.21.15.7:0/1927203490 >> v2:172.21.15.171:3300/0 conn(0x7f99000afaa0 0x7f99000cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f98fffff700 1 -- 172.21.15.7:0/1927203490 >> v1:172.21.15.7:6789/0 conn(0x7f990015abb0 legacy=0x7f99000cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f98ff7fe700 1 -- 172.21.15.7:0/1927203490 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792625071 0 0) 0x7f99000ad860 con 0x7f990015abb0 2024-09-13T18:53:28.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f98fffff700 1 -- 172.21.15.7:0/1927203490 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99000cb540 con 0x7f99000cdb70 2024-09-13T18:53:28.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f98fffff700 1 --2- 172.21.15.7:0/1927203490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99000cdb70 0x7f99000cceb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9900071120 tx=0x7f98f0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abef599dae0e3d94 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:28.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f98ff7fe700 1 -- 172.21.15.7:0/1927203490 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f000b8a0 con 0x7f99000cdb70 2024-09-13T18:53:28.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f98ff7fe700 1 -- 172.21.15.7:0/1927203490 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f000ba00 con 0x7f99000cdb70 2024-09-13T18:53:28.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f98ff7fe700 1 -- 172.21.15.7:0/1927203490 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f000bd90 con 0x7f99000cdb70 2024-09-13T18:53:28.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f9905fa4700 1 -- 172.21.15.7:0/1927203490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99000cdb70 msgr2=0x7f99000cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:28.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f9905fa4700 1 --2- 172.21.15.7:0/1927203490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99000cdb70 0x7f99000cceb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9900071120 tx=0x7f98f0009bf0 comp rx=0 tx=0).stop 2024-09-13T18:53:28.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f9905fa4700 1 -- 172.21.15.7:0/1927203490 shutdown_connections 2024-09-13T18:53:28.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f9905fa4700 1 --2- 172.21.15.7:0/1927203490 >> v2:172.21.15.171:3300/0 conn(0x7f99000afaa0 0x7f99000cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:28.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f9905fa4700 1 --2- 172.21.15.7:0/1927203490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99000cdb70 0x7f99000cceb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:28.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f9905fa4700 1 -- 172.21.15.7:0/1927203490 >> 172.21.15.7:0/1927203490 conn(0x7f99000b2580 msgr2=0x7f99000b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:28.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f9905fa4700 1 -- 172.21.15.7:0/1927203490 shutdown_connections 2024-09-13T18:53:28.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f9905fa4700 1 -- 172.21.15.7:0/1927203490 wait complete. 2024-09-13T18:53:28.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.012+0000 7f9905fa4700 1 Processor -- start 2024-09-13T18:53:28.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f9905fa4700 1 -- start start 2024-09-13T18:53:28.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f9905fa4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99000afaa0 0x7f99001f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:28.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f9905fa4700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f99000cdb70 0x7f9900153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:28.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f9905fa4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990015abb0 0x7f9900153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:28.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f9905fa4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99001f3b00 con 0x7f990015abb0 2024-09-13T18:53:28.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f9905fa4700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9900157720 con 0x7f99000afaa0 2024-09-13T18:53:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f9905fa4700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99001579c0 con 0x7f99000cdb70 2024-09-13T18:53:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f99057a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990015abb0 0x7f9900153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f99057a3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990015abb0 0x7f9900153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53434/0 (socket says 172.21.15.7:53434) 2024-09-13T18:53:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f99057a3700 1 -- 172.21.15.7:0/3045922065 learned_addr learned my addr 172.21.15.7:0/3045922065 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f9904fa2700 1 --2- 172.21.15.7:0/3045922065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99000afaa0 0x7f99001f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f98fffff700 1 --2- 172.21.15.7:0/3045922065 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f99000cdb70 0x7f9900153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f99057a3700 1 -- 172.21.15.7:0/3045922065 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f99000cdb70 msgr2=0x7f9900153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f99057a3700 1 --2- 172.21.15.7:0/3045922065 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f99000cdb70 0x7f9900153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f99057a3700 1 -- 172.21.15.7:0/3045922065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99000afaa0 msgr2=0x7f99001f35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f99057a3700 1 --2- 172.21.15.7:0/3045922065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99000afaa0 0x7f99001f35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f99057a3700 1 -- 172.21.15.7:0/3045922065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f0009570 con 0x7f990015abb0 2024-09-13T18:53:28.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f99057a3700 1 --2- 172.21.15.7:0/3045922065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990015abb0 0x7f9900153550 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f98f8007e60 tx=0x7f98f800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:28.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f98fd7fa700 1 -- 172.21.15.7:0/3045922065 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f800ed30 con 0x7f990015abb0 2024-09-13T18:53:28.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f98fd7fa700 1 -- 172.21.15.7:0/3045922065 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f8014420 con 0x7f990015abb0 2024-09-13T18:53:28.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f9905fa4700 1 -- 172.21.15.7:0/3045922065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9900157ca0 con 0x7f990015abb0 2024-09-13T18:53:28.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f98fd7fa700 1 -- 172.21.15.7:0/3045922065 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f80147d0 con 0x7f990015abb0 2024-09-13T18:53:28.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.016+0000 7f9905fa4700 1 -- 172.21.15.7:0/3045922065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9900158260 con 0x7f990015abb0 2024-09-13T18:53:28.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.020+0000 7f9905fa4700 1 -- 172.21.15.7:0/3045922065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9900158450 con 0x7f990015abb0 2024-09-13T18:53:28.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.020+0000 7f98fd7fa700 1 -- 172.21.15.7:0/3045922065 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f98f8014930 con 0x7f990015abb0 2024-09-13T18:53:28.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.020+0000 7f98fd7fa700 1 --2- 172.21.15.7:0/3045922065 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f98ec066730 0x7f98ec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:28.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.020+0000 7f98fd7fa700 1 -- 172.21.15.7:0/3045922065 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f98f8084800 con 0x7f990015abb0 2024-09-13T18:53:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.020+0000 7f9904fa2700 1 --2- 172.21.15.7:0/3045922065 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f98ec066730 0x7f98ec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.024+0000 7f9904fa2700 1 --2- 172.21.15.7:0/3045922065 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f98ec066730 0x7f98ec068bf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f98f4007c60 tx=0x7f98f4007bb0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:28.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.028+0000 7f98fd7fa700 1 -- 172.21.15.7:0/3045922065 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f98f8051b00 con 0x7f990015abb0 2024-09-13T18:53:28.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.240+0000 7f9905fa4700 1 -- 172.21.15.7:0/3045922065 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99000cceb0 con 0x7f990015abb0 2024-09-13T18:53:28.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.240+0000 7f98fd7fa700 1 -- 172.21.15.7:0/3045922065 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f98f8055120 con 0x7f990015abb0 2024-09-13T18:53:28.247 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:28.248 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2458672850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2458672850},{"type":"v1","addr":"172.21.15.171:6835","nonce":2458672850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3381042298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3381042298},{"type":"v1","addr":"172.21.15.171:6837","nonce":3381042298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2068823578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2068823578},{"type":"v1","addr":"172.21.15.7:6833","nonce":2068823578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:53:17.897317+0000","modified":"2024-09-13T18:53:27.247047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5463":{"gid":5463,"name":"c","rank":0,"incarnation":18,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.7:6835/127797392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":127797392},{"type":"v1","addr":"172.21.15.7:6835","nonce":127797392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T18:53:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.244+0000 7f9905fa4700 1 -- 172.21.15.7:0/3045922065 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f98ec066730 msgr2=0x7f98ec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.244+0000 7f9905fa4700 1 --2- 172.21.15.7:0/3045922065 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f98ec066730 0x7f98ec068bf0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f98f4007c60 tx=0x7f98f4007bb0 comp rx=0 tx=0).stop 2024-09-13T18:53:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.244+0000 7f9905fa4700 1 -- 172.21.15.7:0/3045922065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990015abb0 msgr2=0x7f9900153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.244+0000 7f9905fa4700 1 --2- 172.21.15.7:0/3045922065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990015abb0 0x7f9900153550 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f98f8007e60 tx=0x7f98f800e3c0 comp rx=0 tx=0).stop 2024-09-13T18:53:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.244+0000 7f9905fa4700 1 -- 172.21.15.7:0/3045922065 shutdown_connections 2024-09-13T18:53:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.244+0000 7f9905fa4700 1 --2- 172.21.15.7:0/3045922065 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f98ec066730 0x7f98ec068bf0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.244+0000 7f9905fa4700 1 --2- 172.21.15.7:0/3045922065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f990015abb0 0x7f9900153550 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.244+0000 7f9905fa4700 1 --2- 172.21.15.7:0/3045922065 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f99000cdb70 0x7f9900153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.244+0000 7f9905fa4700 1 --2- 172.21.15.7:0/3045922065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99000afaa0 0x7f99001f35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.244+0000 7f9905fa4700 1 -- 172.21.15.7:0/3045922065 >> 172.21.15.7:0/3045922065 conn(0x7f99000b2580 msgr2=0x7f99000ae600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.244+0000 7f9905fa4700 1 -- 172.21.15.7:0/3045922065 shutdown_connections 2024-09-13T18:53:28.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:28.244+0000 7f9905fa4700 1 -- 172.21.15.7:0/3045922065 wait complete. 2024-09-13T18:53:28.254 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 19 2024-09-13T18:53:28.672 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T18:53:17.897317+0000', 'modified': '2024-09-13T18:53:27.247047+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 39, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5463}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5463': {'gid': 5463, 'name': 'c', 'rank': 0, 'incarnation': 18, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.7:6835/127797392', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 127797392}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 127797392}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T18:53:28.672 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5463:up:reconnect 2024-09-13T18:53:29.217 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:53:29.219 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:53:29.673 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:29.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb7d1fb700 1 Processor -- start 2024-09-13T18:53:29.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb7d1fb700 1 -- start start 2024-09-13T18:53:29.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb7d1fb700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7feb780cceb0 0x7feb780afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:29.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb7d1fb700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb780ad860 con 0x7feb7815abb0 2024-09-13T18:53:29.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb7d1fb700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb780cb540 con 0x7feb780cdb70 2024-09-13T18:53:29.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb7d1fb700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb780b01a0 con 0x7feb780cceb0 2024-09-13T18:53:29.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb77fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7feb7815abb0 0x7feb780cd460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39248/0 (socket says 172.21.15.7:39248) 2024-09-13T18:53:29.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb77fff700 1 -- 172.21.15.7:0/2814155506 learned_addr learned my addr 172.21.15.7:0/2814155506 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:29.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb7c9fa700 1 --2- 172.21.15.7:0/2814155506 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7feb780cceb0 0x7feb780afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:29.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb76ffd700 1 -- 172.21.15.7:0/2814155506 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609941169 0 0) 0x7feb780ad860 con 0x7feb7815abb0 2024-09-13T18:53:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb76ffd700 1 -- 172.21.15.7:0/2814155506 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb640035d0 con 0x7feb7815abb0 2024-09-13T18:53:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb76ffd700 1 -- 172.21.15.7:0/2814155506 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302587041 0 0) 0x7feb780cb540 con 0x7feb780cdb70 2024-09-13T18:53:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb76ffd700 1 -- 172.21.15.7:0/2814155506 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb780ad860 con 0x7feb780cdb70 2024-09-13T18:53:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb7c9fa700 1 -- 172.21.15.7:0/2814155506 >> v1:172.21.15.171:6789/0 conn(0x7feb780cdb70 legacy=0x7feb780cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb7c9fa700 1 -- 172.21.15.7:0/2814155506 >> v1:172.21.15.7:6789/0 conn(0x7feb7815abb0 legacy=0x7feb780cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb7c9fa700 1 -- 172.21.15.7:0/2814155506 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb780cfa00 con 0x7feb780cceb0 2024-09-13T18:53:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb7c9fa700 1 --2- 172.21.15.7:0/2814155506 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7feb780cceb0 0x7feb780afaa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feb70009dc0 tx=0x7feb7000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f2a186eccaca12d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb76ffd700 1 -- 172.21.15.7:0/2814155506 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7000e070 con 0x7feb780cceb0 2024-09-13T18:53:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb76ffd700 1 -- 172.21.15.7:0/2814155506 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7000bb20 con 0x7feb780cceb0 2024-09-13T18:53:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb76ffd700 1 -- 172.21.15.7:0/2814155506 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb70012430 con 0x7feb780cceb0 2024-09-13T18:53:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb7d1fb700 1 -- 172.21.15.7:0/2814155506 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7feb780cceb0 msgr2=0x7feb780afaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb7d1fb700 1 --2- 172.21.15.7:0/2814155506 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7feb780cceb0 0x7feb780afaa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feb70009dc0 tx=0x7feb7000b330 comp rx=0 tx=0).stop 2024-09-13T18:53:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb7d1fb700 1 -- 172.21.15.7:0/2814155506 shutdown_connections 2024-09-13T18:53:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb7d1fb700 1 --2- 172.21.15.7:0/2814155506 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7feb780cceb0 0x7feb780afaa0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.792+0000 7feb7d1fb700 1 -- 172.21.15.7:0/2814155506 >> 172.21.15.7:0/2814155506 conn(0x7feb780b2580 msgr2=0x7feb780b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb7d1fb700 1 -- 172.21.15.7:0/2814155506 shutdown_connections 2024-09-13T18:53:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb7d1fb700 1 -- 172.21.15.7:0/2814155506 wait complete. 2024-09-13T18:53:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb7d1fb700 1 Processor -- start 2024-09-13T18:53:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb7d1fb700 1 -- start start 2024-09-13T18:53:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb7d1fb700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb780cceb0 0x7feb7815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb7d1fb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb780cdb70 0x7feb780b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb7d1fb700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7feb7815abb0 0x7feb780b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb7d1fb700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7814dcd0 con 0x7feb780cdb70 2024-09-13T18:53:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb7d1fb700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7815c430 con 0x7feb780cceb0 2024-09-13T18:53:29.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb7d1fb700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7815c710 con 0x7feb7815abb0 2024-09-13T18:53:29.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb780cdb70 0x7feb780b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:29.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb780cdb70 0x7feb780b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53442/0 (socket says 172.21.15.7:53442) 2024-09-13T18:53:29.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb777fe700 1 -- 172.21.15.7:0/1039332547 learned_addr learned my addr 172.21.15.7:0/1039332547 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:29.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb77fff700 1 --2- 172.21.15.7:0/1039332547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb780cceb0 0x7feb7815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:29.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb7c9fa700 1 --2- 172.21.15.7:0/1039332547 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7feb7815abb0 0x7feb780b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:29.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb777fe700 1 -- 172.21.15.7:0/1039332547 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7feb7815abb0 msgr2=0x7feb780b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:29.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb777fe700 1 --2- 172.21.15.7:0/1039332547 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7feb7815abb0 0x7feb780b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:29.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb777fe700 1 -- 172.21.15.7:0/1039332547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb780cceb0 msgr2=0x7feb7815d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:29.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb777fe700 1 --2- 172.21.15.7:0/1039332547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb780cceb0 0x7feb7815d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:29.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb777fe700 1 -- 172.21.15.7:0/1039332547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb70009a70 con 0x7feb780cdb70 2024-09-13T18:53:29.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb777fe700 1 --2- 172.21.15.7:0/1039332547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb780cdb70 0x7feb780b3880 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7feb6000c730 tx=0x7feb6000ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:29.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb74ff9700 1 -- 172.21.15.7:0/1039332547 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb600052d0 con 0x7feb780cdb70 2024-09-13T18:53:29.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb74ff9700 1 -- 172.21.15.7:0/1039332547 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb60005450 con 0x7feb780cdb70 2024-09-13T18:53:29.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb7d1fb700 1 -- 172.21.15.7:0/1039332547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7815c9f0 con 0x7feb780cdb70 2024-09-13T18:53:29.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb74ff9700 1 -- 172.21.15.7:0/1039332547 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb600038e0 con 0x7feb780cdb70 2024-09-13T18:53:29.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.796+0000 7feb7d1fb700 1 -- 172.21.15.7:0/1039332547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7815cfb0 con 0x7feb780cdb70 2024-09-13T18:53:29.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.800+0000 7feb7d1fb700 1 -- 172.21.15.7:0/1039332547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb780afaa0 con 0x7feb780cdb70 2024-09-13T18:53:29.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.800+0000 7feb74ff9700 1 -- 172.21.15.7:0/1039332547 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7feb60003a40 con 0x7feb780cdb70 2024-09-13T18:53:29.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.800+0000 7feb74ff9700 1 --2- 172.21.15.7:0/1039332547 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7feb58066730 0x7feb58068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:29.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.800+0000 7feb74ff9700 1 -- 172.21.15.7:0/1039332547 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7feb60083570 con 0x7feb780cdb70 2024-09-13T18:53:29.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.800+0000 7feb77fff700 1 --2- 172.21.15.7:0/1039332547 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7feb58066730 0x7feb58068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:29.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.804+0000 7feb77fff700 1 --2- 172.21.15.7:0/1039332547 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7feb58066730 0x7feb58068bf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7feb68007fd0 tx=0x7feb68007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:29.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:29.808+0000 7feb74ff9700 1 -- 172.21.15.7:0/1039332547 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7feb60050910 con 0x7feb780cdb70 2024-09-13T18:53:30.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:30.020+0000 7feb7d1fb700 1 -- 172.21.15.7:0/1039332547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb780b69d0 con 0x7feb780cdb70 2024-09-13T18:53:30.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:30.020+0000 7feb74ff9700 1 -- 172.21.15.7:0/1039332547 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4738 (secure 0 0 0) 0x7feb60081e60 con 0x7feb780cdb70 2024-09-13T18:53:30.027 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:30.027 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2458672850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2458672850},{"type":"v1","addr":"172.21.15.171:6835","nonce":2458672850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3381042298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3381042298},{"type":"v1","addr":"172.21.15.171:6837","nonce":3381042298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2068823578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2068823578},{"type":"v1","addr":"172.21.15.7:6833","nonce":2068823578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:53:17.897317+0000","modified":"2024-09-13T18:53:29.251152+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5463":{"gid":5463,"name":"c","rank":0,"incarnation":18,"state":"up:clientreplay","state_seq":4,"addr":"172.21.15.7:6835/127797392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":127797392},{"type":"v1","addr":"172.21.15.7:6835","nonce":127797392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T18:53:30.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:30.024+0000 7feb7d1fb700 1 -- 172.21.15.7:0/1039332547 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7feb58066730 msgr2=0x7feb58068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:30.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:30.024+0000 7feb7d1fb700 1 --2- 172.21.15.7:0/1039332547 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7feb58066730 0x7feb58068bf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7feb68007fd0 tx=0x7feb68007de0 comp rx=0 tx=0).stop 2024-09-13T18:53:30.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:30.024+0000 7feb7d1fb700 1 -- 172.21.15.7:0/1039332547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb780cdb70 msgr2=0x7feb780b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:30.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:30.024+0000 7feb7d1fb700 1 --2- 172.21.15.7:0/1039332547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb780cdb70 0x7feb780b3880 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7feb6000c730 tx=0x7feb6000ca40 comp rx=0 tx=0).stop 2024-09-13T18:53:30.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:30.024+0000 7feb7d1fb700 1 -- 172.21.15.7:0/1039332547 shutdown_connections 2024-09-13T18:53:30.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:30.024+0000 7feb7d1fb700 1 --2- 172.21.15.7:0/1039332547 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7feb58066730 0x7feb58068bf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:30.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:30.024+0000 7feb7d1fb700 1 --2- 172.21.15.7:0/1039332547 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7feb7815abb0 0x7feb780b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:30.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:30.024+0000 7feb7d1fb700 1 --2- 172.21.15.7:0/1039332547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb780cdb70 0x7feb780b3880 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:30.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:30.024+0000 7feb7d1fb700 1 --2- 172.21.15.7:0/1039332547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feb780cceb0 0x7feb7815d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:30.024+0000 7feb7d1fb700 1 -- 172.21.15.7:0/1039332547 >> 172.21.15.7:0/1039332547 conn(0x7feb780b2580 msgr2=0x7feb780cd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:30.024+0000 7feb7d1fb700 1 -- 172.21.15.7:0/1039332547 shutdown_connections 2024-09-13T18:53:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:30.024+0000 7feb7d1fb700 1 -- 172.21.15.7:0/1039332547 wait complete. 2024-09-13T18:53:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 21 2024-09-13T18:53:30.446 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T18:53:17.897317+0000', 'modified': '2024-09-13T18:53:29.251152+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 39, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5463}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5463': {'gid': 5463, 'name': 'c', 'rank': 0, 'incarnation': 18, 'state': 'up:clientreplay', 'state_seq': 4, 'addr': '172.21.15.7:6835/127797392', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 127797392}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 127797392}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T18:53:30.446 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5463:up:clientreplay 2024-09-13T18:53:31.251 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-09-13T18:53:31.244+0000 7fa8b0cc3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T18:53:31.447 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:31.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253f35d700 1 Processor -- start 2024-09-13T18:53:31.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253f35d700 1 -- start start 2024-09-13T18:53:31.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253f35d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f25380d0e80 0x7f25380b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:31.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253f35d700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f25380d3950 0x7f25380b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:31.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253f35d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f25380b4240 0x7f25380b3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:31.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253f35d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25380d17e0 con 0x7f25380d0e80 2024-09-13T18:53:31.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253f35d700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25380d1ac0 con 0x7f25380d3950 2024-09-13T18:53:31.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253f35d700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25380af9e0 con 0x7f25380b4240 2024-09-13T18:53:31.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253d0f9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f25380d0e80 0x7f25380b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:31.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253d0f9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f25380d0e80 0x7f25380b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34908/0 (socket says 172.21.15.7:34908) 2024-09-13T18:53:31.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253d0f9700 1 -- 172.21.15.7:0/4029242152 learned_addr learned my addr 172.21.15.7:0/4029242152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:31.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253c8f8700 1 --2- 172.21.15.7:0/4029242152 >> v2:172.21.15.171:3300/0 conn(0x7f25380d3950 0x7f25380b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:31.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253d8fa700 1 --2- 172.21.15.7:0/4029242152 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f25380b4240 0x7f25380b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:31.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253d0f9700 1 -- 172.21.15.7:0/4029242152 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f25380b4240 msgr2=0x7f25380b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:31.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253d0f9700 1 --2- 172.21.15.7:0/4029242152 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f25380b4240 0x7f25380b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:31.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253d0f9700 1 -- 172.21.15.7:0/4029242152 >> v2:172.21.15.171:3300/0 conn(0x7f25380d3950 msgr2=0x7f25380b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:31.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253d0f9700 1 --2- 172.21.15.7:0/4029242152 >> v2:172.21.15.171:3300/0 conn(0x7f25380d3950 0x7f25380b2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:31.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253d0f9700 1 -- 172.21.15.7:0/4029242152 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25380afcc0 con 0x7f25380d0e80 2024-09-13T18:53:31.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f253d0f9700 1 --2- 172.21.15.7:0/4029242152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25380d0e80 0x7f25380b26a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2530009910 tx=0x7f2530009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3813cc9a5b751c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:31.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f2537fff700 1 -- 172.21.15.7:0/4029242152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253000ba20 con 0x7f25380d0e80 2024-09-13T18:53:31.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f2537fff700 1 -- 172.21.15.7:0/4029242152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253000bb80 con 0x7f25380d0e80 2024-09-13T18:53:31.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.564+0000 7f2537fff700 1 -- 172.21.15.7:0/4029242152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530009e60 con 0x7f25380d0e80 2024-09-13T18:53:31.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253f35d700 1 -- 172.21.15.7:0/4029242152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25380d0e80 msgr2=0x7f25380b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:31.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253f35d700 1 --2- 172.21.15.7:0/4029242152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25380d0e80 0x7f25380b26a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2530009910 tx=0x7f2530009c20 comp rx=0 tx=0).stop 2024-09-13T18:53:31.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253f35d700 1 -- 172.21.15.7:0/4029242152 shutdown_connections 2024-09-13T18:53:31.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253f35d700 1 --2- 172.21.15.7:0/4029242152 >> v2:172.21.15.171:3300/0 conn(0x7f25380d3950 0x7f25380b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:31.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253f35d700 1 --2- 172.21.15.7:0/4029242152 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f25380b4240 0x7f25380b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:31.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253f35d700 1 --2- 172.21.15.7:0/4029242152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25380d0e80 0x7f25380b26a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:31.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253f35d700 1 -- 172.21.15.7:0/4029242152 >> 172.21.15.7:0/4029242152 conn(0x7f25380d2e30 msgr2=0x7f25380d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:31.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253f35d700 1 -- 172.21.15.7:0/4029242152 shutdown_connections 2024-09-13T18:53:31.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253f35d700 1 -- 172.21.15.7:0/4029242152 wait complete. 2024-09-13T18:53:31.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253f35d700 1 Processor -- start 2024-09-13T18:53:31.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253f35d700 1 -- start start 2024-09-13T18:53:31.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253f35d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25380b4240 0x7f253815d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:31.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253f35d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25380d0e80 0x7f25380cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:31.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253f35d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f25380d3950 0x7f25380b8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:31.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253f35d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25380cb0f0 con 0x7f25380b4240 2024-09-13T18:53:31.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253f35d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25380cb3d0 con 0x7f25380d0e80 2024-09-13T18:53:31.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253f35d700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25380cb6b0 con 0x7f25380d3950 2024-09-13T18:53:31.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253d0f9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25380b4240 0x7f253815d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253d0f9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25380b4240 0x7f253815d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34924/0 (socket says 172.21.15.7:34924) 2024-09-13T18:53:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.568+0000 7f253d0f9700 1 -- 172.21.15.7:0/278505470 learned_addr learned my addr 172.21.15.7:0/278505470 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.572+0000 7f253c8f8700 1 --2- 172.21.15.7:0/278505470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25380d0e80 0x7f25380cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.572+0000 7f253d8fa700 1 --2- 172.21.15.7:0/278505470 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f25380d3950 0x7f25380b8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.572+0000 7f253d0f9700 1 -- 172.21.15.7:0/278505470 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f25380d3950 msgr2=0x7f25380b8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.572+0000 7f253d0f9700 1 --2- 172.21.15.7:0/278505470 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f25380d3950 0x7f25380b8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.572+0000 7f253d0f9700 1 -- 172.21.15.7:0/278505470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25380d0e80 msgr2=0x7f25380cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.572+0000 7f253d0f9700 1 --2- 172.21.15.7:0/278505470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25380d0e80 0x7f25380cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.572+0000 7f253d0f9700 1 -- 172.21.15.7:0/278505470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2530009570 con 0x7f25380b4240 2024-09-13T18:53:31.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.572+0000 7f253d0f9700 1 --2- 172.21.15.7:0/278505470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25380b4240 0x7f253815d130 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f253808ba30 tx=0x7f2530004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:31.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.572+0000 7f2535ffb700 1 -- 172.21.15.7:0/278505470 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530004450 con 0x7f25380b4240 2024-09-13T18:53:31.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.572+0000 7f2535ffb700 1 -- 172.21.15.7:0/278505470 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253001f070 con 0x7f25380b4240 2024-09-13T18:53:31.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.572+0000 7f2535ffb700 1 -- 172.21.15.7:0/278505470 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530003900 con 0x7f25380b4240 2024-09-13T18:53:31.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.572+0000 7f253f35d700 1 -- 172.21.15.7:0/278505470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25380cb990 con 0x7f25380b4240 2024-09-13T18:53:31.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.572+0000 7f253f35d700 1 -- 172.21.15.7:0/278505470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25380cc6d0 con 0x7f25380b4240 2024-09-13T18:53:31.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.572+0000 7f253f35d700 1 -- 172.21.15.7:0/278505470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25380d1e50 con 0x7f25380b4240 2024-09-13T18:53:31.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.572+0000 7f2535ffb700 1 -- 172.21.15.7:0/278505470 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2530003a60 con 0x7f25380b4240 2024-09-13T18:53:31.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.576+0000 7f2535ffb700 1 --2- 172.21.15.7:0/278505470 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2518066730 0x7f2518068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:31.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.576+0000 7f2535ffb700 1 -- 172.21.15.7:0/278505470 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2530021070 con 0x7f25380b4240 2024-09-13T18:53:31.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.576+0000 7f253c8f8700 1 --2- 172.21.15.7:0/278505470 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2518066730 0x7f2518068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:31.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.576+0000 7f253c8f8700 1 --2- 172.21.15.7:0/278505470 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2518066730 0x7f2518068bf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f25380b6330 tx=0x7f2520008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:31.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.584+0000 7f2535ffb700 1 -- 172.21.15.7:0/278505470 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2530052910 con 0x7f25380b4240 2024-09-13T18:53:31.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.796+0000 7f253f35d700 1 -- 172.21.15.7:0/278505470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25380b26a0 con 0x7f25380b4240 2024-09-13T18:53:31.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.796+0000 7f2535ffb700 1 -- 172.21.15.7:0/278505470 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4732 (secure 0 0 0) 0x7f25300045d0 con 0x7f25380b4240 2024-09-13T18:53:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:31.805 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2458672850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2458672850},{"type":"v1","addr":"172.21.15.171:6835","nonce":2458672850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3381042298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3381042298},{"type":"v1","addr":"172.21.15.171:6837","nonce":3381042298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2068823578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2068823578},{"type":"v1","addr":"172.21.15.7:6833","nonce":2068823578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:53:17.897317+0000","modified":"2024-09-13T18:53:30.253515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5463":{"gid":5463,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/127797392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":127797392},{"type":"v1","addr":"172.21.15.7:6835","nonce":127797392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T18:53:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.800+0000 7f253f35d700 1 -- 172.21.15.7:0/278505470 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2518066730 msgr2=0x7f2518068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.800+0000 7f253f35d700 1 --2- 172.21.15.7:0/278505470 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2518066730 0x7f2518068bf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f25380b6330 tx=0x7f2520008040 comp rx=0 tx=0).stop 2024-09-13T18:53:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.800+0000 7f253f35d700 1 -- 172.21.15.7:0/278505470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25380b4240 msgr2=0x7f253815d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.800+0000 7f253f35d700 1 --2- 172.21.15.7:0/278505470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25380b4240 0x7f253815d130 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f253808ba30 tx=0x7f2530004770 comp rx=0 tx=0).stop 2024-09-13T18:53:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.800+0000 7f253f35d700 1 -- 172.21.15.7:0/278505470 shutdown_connections 2024-09-13T18:53:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.800+0000 7f253f35d700 1 --2- 172.21.15.7:0/278505470 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2518066730 0x7f2518068bf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.800+0000 7f253f35d700 1 --2- 172.21.15.7:0/278505470 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f25380d3950 0x7f25380b8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.800+0000 7f253f35d700 1 --2- 172.21.15.7:0/278505470 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25380d0e80 0x7f25380cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.800+0000 7f253f35d700 1 --2- 172.21.15.7:0/278505470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25380b4240 0x7f253815d130 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.800+0000 7f253f35d700 1 -- 172.21.15.7:0/278505470 >> 172.21.15.7:0/278505470 conn(0x7f25380d2e30 msgr2=0x7f25380b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.800+0000 7f253f35d700 1 -- 172.21.15.7:0/278505470 shutdown_connections 2024-09-13T18:53:31.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:31.800+0000 7f253f35d700 1 -- 172.21.15.7:0/278505470 wait complete. 2024-09-13T18:53:31.809 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-09-13T18:53:32.222 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T18:53:17.897317+0000', 'modified': '2024-09-13T18:53:30.253515+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 39, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5463}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5463': {'gid': 5463, 'name': 'c', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.7:6835/127797392', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 127797392}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 127797392}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T18:53:32.365 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T18:53:32.366 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-13T18:53:32.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f6159ad6700 1 Processor -- start 2024-09-13T18:53:32.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f6159ad6700 1 -- start start 2024-09-13T18:53:32.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f6159ad6700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f61541606a0 0x7f61540d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f6159ad6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61540d1ca0 con 0x7f6154160b10 2024-09-13T18:53:32.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f6159ad6700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61540cdd00 con 0x7f61540d17a0 2024-09-13T18:53:32.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f6159ad6700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61540cdfe0 con 0x7f61541606a0 2024-09-13T18:53:32.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f61592d5700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6154160b10 0x7f61540b5b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53616/0 (socket says 172.21.15.7:53616) 2024-09-13T18:53:32.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f61592d5700 1 -- 172.21.15.7:0/2571573068 learned_addr learned my addr 172.21.15.7:0/2571573068 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:32.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f6158ad4700 1 --2- 172.21.15.7:0/2571573068 >> v2:172.21.15.171:3300/0 conn(0x7f61541606a0 0x7f61540d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f61537fe700 1 -- 172.21.15.7:0/2571573068 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3667584999 0 0) 0x7f61540d1ca0 con 0x7f6154160b10 2024-09-13T18:53:32.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f61537fe700 1 -- 172.21.15.7:0/2571573068 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f613c0035d0 con 0x7f6154160b10 2024-09-13T18:53:32.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f61537fe700 1 -- 172.21.15.7:0/2571573068 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114368434 0 0) 0x7f61540cdd00 con 0x7f61540d17a0 2024-09-13T18:53:32.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f61537fe700 1 -- 172.21.15.7:0/2571573068 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61540d1ca0 con 0x7f61540d17a0 2024-09-13T18:53:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f61537fe700 1 -- 172.21.15.7:0/2571573068 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 67728203 0 0) 0x7f613c0035d0 con 0x7f6154160b10 2024-09-13T18:53:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f61537fe700 1 -- 172.21.15.7:0/2571573068 >> v2:172.21.15.171:3300/0 conn(0x7f61541606a0 msgr2=0x7f61540d3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f61537fe700 1 --2- 172.21.15.7:0/2571573068 >> v2:172.21.15.171:3300/0 conn(0x7f61541606a0 0x7f61540d3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f61537fe700 1 -- 172.21.15.7:0/2571573068 >> v1:172.21.15.171:6789/0 conn(0x7f61540d17a0 legacy=0x7f61540ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f61537fe700 1 -- 172.21.15.7:0/2571573068 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61540ce2c0 con 0x7f6154160b10 2024-09-13T18:53:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.488+0000 7f61537fe700 1 -- 172.21.15.7:0/2571573068 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7f614c0019f0 con 0x7f6154160b10 2024-09-13T18:53:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f61537fe700 1 -- 172.21.15.7:0/2571573068 >> v1:172.21.15.7:6789/0 conn(0x7f6154160b10 legacy=0x7f61540b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f61537fe700 1 --2- 172.21.15.7:0/2571573068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f613c003bb0 0x7f613c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f61537fe700 1 --2- 172.21.15.7:0/2571573068 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f613c004870 0x7f613c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f61537fe700 1 --2- 172.21.15.7:0/2571573068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f613c005230 0x7f613c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f61537fe700 1 -- 172.21.15.7:0/2571573068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61540cdfe0 con 0x7f613c003bb0 2024-09-13T18:53:32.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f61537fe700 1 -- 172.21.15.7:0/2571573068 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c00c0b0 con 0x7f613c005230 2024-09-13T18:53:32.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f61537fe700 1 -- 172.21.15.7:0/2571573068 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613c00c270 con 0x7f613c004870 2024-09-13T18:53:32.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f6158ad4700 1 --2- 172.21.15.7:0/2571573068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f613c003bb0 0x7f613c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f61592d5700 1 --2- 172.21.15.7:0/2571573068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f613c005230 0x7f613c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f6153fff700 1 --2- 172.21.15.7:0/2571573068 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f613c004870 0x7f613c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f6158ad4700 1 -- 172.21.15.7:0/2571573068 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f613c004870 msgr2=0x7f613c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:32.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f6158ad4700 1 --2- 172.21.15.7:0/2571573068 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f613c004870 0x7f613c004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f6158ad4700 1 -- 172.21.15.7:0/2571573068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f613c005230 msgr2=0x7f613c00ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:32.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f6158ad4700 1 --2- 172.21.15.7:0/2571573068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f613c005230 0x7f613c00ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f6158ad4700 1 -- 172.21.15.7:0/2571573068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c0035d0 con 0x7f613c003bb0 2024-09-13T18:53:32.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f61592d5700 1 --2- 172.21.15.7:0/2571573068 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f613c005230 0x7f613c00ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:32.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f61592d5700 1 -- 172.21.15.7:0/2571573068 reap_dead start 2024-09-13T18:53:32.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f6158ad4700 1 --2- 172.21.15.7:0/2571573068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f613c003bb0 0x7f613c003fb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6148009b70 tx=0x7f613c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be3a061cae049a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:32.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f61537fe700 1 -- 172.21.15.7:0/2571573068 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614800e6c0 con 0x7f613c003bb0 2024-09-13T18:53:32.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f61537fe700 1 -- 172.21.15.7:0/2571573068 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614800e820 con 0x7f613c003bb0 2024-09-13T18:53:32.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f61537fe700 1 -- 172.21.15.7:0/2571573068 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614800eb90 con 0x7f613c003bb0 2024-09-13T18:53:32.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f6159ad6700 1 -- 172.21.15.7:0/2571573068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f613c003bb0 msgr2=0x7f613c003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:32.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f6159ad6700 1 --2- 172.21.15.7:0/2571573068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f613c003bb0 0x7f613c003fb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6148009b70 tx=0x7f613c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f6159ad6700 1 -- 172.21.15.7:0/2571573068 shutdown_connections 2024-09-13T18:53:32.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f6159ad6700 1 --2- 172.21.15.7:0/2571573068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f613c003bb0 0x7f613c003fb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f6159ad6700 1 -- 172.21.15.7:0/2571573068 >> 172.21.15.7:0/2571573068 conn(0x7f61540d2d10 msgr2=0x7f61540d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:32.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f6159ad6700 1 -- 172.21.15.7:0/2571573068 shutdown_connections 2024-09-13T18:53:32.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.492+0000 7f6159ad6700 1 -- 172.21.15.7:0/2571573068 wait complete. 2024-09-13T18:53:32.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6159ad6700 1 Processor -- start 2024-09-13T18:53:32.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6159ad6700 1 -- start start 2024-09-13T18:53:32.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6159ad6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f613c003bb0 0x7f61540ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6159ad6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61540cd010 0x7f615406c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6159ad6700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f615406c5c0 0x7f61541470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6159ad6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615406ca70 con 0x7f613c003bb0 2024-09-13T18:53:32.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6159ad6700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61540cc250 con 0x7f61540cd010 2024-09-13T18:53:32.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6159ad6700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61540cc530 con 0x7f615406c5c0 2024-09-13T18:53:32.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6158ad4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f613c003bb0 0x7f61540ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6158ad4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f613c003bb0 0x7f61540ca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34928/0 (socket says 172.21.15.7:34928) 2024-09-13T18:53:32.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6158ad4700 1 -- 172.21.15.7:0/378734327 learned_addr learned my addr 172.21.15.7:0/378734327 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:32.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6153fff700 1 --2- 172.21.15.7:0/378734327 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61540cd010 0x7f615406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f61592d5700 1 --2- 172.21.15.7:0/378734327 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f615406c5c0 0x7f61541470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6158ad4700 1 -- 172.21.15.7:0/378734327 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f615406c5c0 msgr2=0x7f61541470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:32.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6158ad4700 1 --2- 172.21.15.7:0/378734327 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f615406c5c0 0x7f61541470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6158ad4700 1 -- 172.21.15.7:0/378734327 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61540cd010 msgr2=0x7f615406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:32.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6158ad4700 1 --2- 172.21.15.7:0/378734327 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61540cd010 0x7f615406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6158ad4700 1 -- 172.21.15.7:0/378734327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61540cc810 con 0x7f613c003bb0 2024-09-13T18:53:32.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6158ad4700 1 --2- 172.21.15.7:0/378734327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f613c003bb0 0x7f61540ca7f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f613c00c080 tx=0x7f61480047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:32.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f61517fa700 1 -- 172.21.15.7:0/378734327 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614800e6c0 con 0x7f613c003bb0 2024-09-13T18:53:32.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f61517fa700 1 -- 172.21.15.7:0/378734327 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6148003b10 con 0x7f613c003bb0 2024-09-13T18:53:32.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f6159ad6700 1 -- 172.21.15.7:0/378734327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61540ccaf0 con 0x7f613c003bb0 2024-09-13T18:53:32.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.496+0000 7f61517fa700 1 -- 172.21.15.7:0/378734327 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6148013470 con 0x7f613c003bb0 2024-09-13T18:53:32.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.500+0000 7f6159ad6700 1 -- 172.21.15.7:0/378734327 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6154147610 con 0x7f613c003bb0 2024-09-13T18:53:32.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.500+0000 7f61517fa700 1 -- 172.21.15.7:0/378734327 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f61480135d0 con 0x7f613c003bb0 2024-09-13T18:53:32.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.500+0000 7f61517fa700 1 --2- 172.21.15.7:0/378734327 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6140066760 0x7f6140068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.500+0000 7f61517fa700 1 -- 172.21.15.7:0/378734327 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f61480856b0 con 0x7f613c003bb0 2024-09-13T18:53:32.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.500+0000 7f6153fff700 1 --2- 172.21.15.7:0/378734327 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6140066760 0x7f6140068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.500+0000 7f615dfe9740 1 Processor -- start 2024-09-13T18:53:32.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.500+0000 7f615dfe9740 1 -- start start 2024-09-13T18:53:32.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f6153fff700 1 --2- 172.21.15.7:0/378734327 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6140066760 0x7f6140068c20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f61541f61f0 tx=0x7f6144009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:32.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f615dfe9740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x202b230 0x202b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f615dfe9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2009e90 0x1ff3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f615dfe9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x1ff3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f615dfe9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f52d80 con 0x202bb50 2024-09-13T18:53:32.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f615dfe9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f50510 con 0x2009e90 2024-09-13T18:53:32.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f615dfe9740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f4be00 con 0x202b230 2024-09-13T18:53:32.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f61592d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x1ff3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f61592d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x1ff3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34936/0 (socket says 172.21.15.7:34936) 2024-09-13T18:53:32.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f61592d5700 1 -- 172.21.15.7:0/250794258 learned_addr learned my addr 172.21.15.7:0/250794258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:32.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f6158ad4700 1 --2- 172.21.15.7:0/250794258 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x202b230 0x202b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f6153fff700 1 --2- 172.21.15.7:0/250794258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2009e90 0x1ff3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f61592d5700 1 -- 172.21.15.7:0/250794258 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x202b230 msgr2=0x202b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:32.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f61592d5700 1 --2- 172.21.15.7:0/250794258 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x202b230 0x202b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f61592d5700 1 -- 172.21.15.7:0/250794258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2009e90 msgr2=0x1ff3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:32.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f61592d5700 1 --2- 172.21.15.7:0/250794258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2009e90 0x1ff3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f61592d5700 1 -- 172.21.15.7:0/250794258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1dbe960 con 0x202bb50 2024-09-13T18:53:32.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f6153fff700 1 --2- 172.21.15.7:0/250794258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2009e90 0x1ff3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:32.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f61592d5700 1 --2- 172.21.15.7:0/250794258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x1ff3dd0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f61540d2510 tx=0x7f614c0021a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac5f98fcbef9336 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:32.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f613a7fc700 1 -- 172.21.15.7:0/250794258 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c00ab60 con 0x202bb50 2024-09-13T18:53:32.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f613a7fc700 1 -- 172.21.15.7:0/250794258 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c00acc0 con 0x202bb50 2024-09-13T18:53:32.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f613a7fc700 1 -- 172.21.15.7:0/250794258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c00ae30 con 0x202bb50 2024-09-13T18:53:32.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f615dfe9740 1 -- 172.21.15.7:0/250794258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 msgr2=0x1ff3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:32.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f615dfe9740 1 --2- 172.21.15.7:0/250794258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x1ff3dd0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f61540d2510 tx=0x7f614c0021a0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f615dfe9740 1 -- 172.21.15.7:0/250794258 shutdown_connections 2024-09-13T18:53:32.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f615dfe9740 1 --2- 172.21.15.7:0/250794258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x1ff3dd0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f615dfe9740 1 --2- 172.21.15.7:0/250794258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2009e90 0x1ff3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f615dfe9740 1 --2- 172.21.15.7:0/250794258 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x202b230 0x202b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f615dfe9740 1 -- 172.21.15.7:0/250794258 >> 172.21.15.7:0/250794258 conn(0x1d734d0 msgr2=0x200ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:32.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f615dfe9740 1 -- 172.21.15.7:0/250794258 shutdown_connections 2024-09-13T18:53:32.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.504+0000 7f615dfe9740 1 -- 172.21.15.7:0/250794258 wait complete. 2024-09-13T18:53:32.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f615dfe9740 1 -- start start 2024-09-13T18:53:32.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f615dfe9740 1 Processor -- start 2024-09-13T18:53:32.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f615dfe9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2009e90 0x2005c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f615dfe9740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x202b230 0x1fe38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f615dfe9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x1fe5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f615dfe9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f09730 con 0x202bb50 2024-09-13T18:53:32.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f615dfe9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f0f5b0 con 0x2009e90 2024-09-13T18:53:32.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f615dfe9740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x1df7aa0 con 0x202b230 2024-09-13T18:53:32.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f615dfe9740 10 client.0 _ll_register_callbacks cb 0x1f45610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:53:32.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f61592d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x1fe5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f61592d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x1fe5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34946/0 (socket says 172.21.15.7:34946) 2024-09-13T18:53:32.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f61592d5700 1 -- 172.21.15.7:0/717558305 learned_addr learned my addr 172.21.15.7:0/717558305 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:32.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f6153fff700 1 --2- 172.21.15.7:0/717558305 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x202b230 0x1fe38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f6158ad4700 1 --2- 172.21.15.7:0/717558305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2009e90 0x2005c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f61592d5700 1 -- 172.21.15.7:0/717558305 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x202b230 msgr2=0x1fe38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:32.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f61592d5700 1 --2- 172.21.15.7:0/717558305 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x202b230 0x1fe38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f61592d5700 1 -- 172.21.15.7:0/717558305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2009e90 msgr2=0x2005c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:32.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f61592d5700 1 --2- 172.21.15.7:0/717558305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2009e90 0x2005c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f61592d5700 1 -- 172.21.15.7:0/717558305 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61540ce2c0 con 0x202bb50 2024-09-13T18:53:32.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f61592d5700 1 --2- 172.21.15.7:0/717558305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x1fe5880 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f614c018040 tx=0x7f614c001d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374e79c52500c4a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:32.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f61337fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-13T18:53:32.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f61337fe700 1 -- 172.21.15.7:0/717558305 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c00ab60 con 0x202bb50 2024-09-13T18:53:32.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f61337fe700 1 -- 172.21.15.7:0/717558305 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f614c015c20 con 0x202bb50 2024-09-13T18:53:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.508+0000 7f615dfe9740 1 -- 172.21.15.7:0/717558305 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1d801e0 con 0x202bb50 2024-09-13T18:53:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f61337fe700 1 -- 172.21.15.7:0/717558305 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f614c015e20 con 0x202bb50 2024-09-13T18:53:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 10 client.5499 fetch_fsmap learned FSMap version 22 2024-09-13T18:53:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 1 -- 172.21.15.7:0/717558305 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x1f12220 con 0x202bb50 2024-09-13T18:53:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f61337fe700 1 -- 172.21.15.7:0/717558305 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1d801e0 con 0x202bb50 2024-09-13T18:53:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f61337fe700 1 -- 172.21.15.7:0/717558305 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 22) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f614c012070 con 0x202bb50 2024-09-13T18:53:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 10 client.5499 fetch_fsmap finished waiting for FSMap version 22 2024-09-13T18:53:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 10 client.5499 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T18:53:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 20 client.5499 populate_metadata read hostname 'smithi007' 2024-09-13T18:53:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 1 --2- 172.21.15.7:0/717558305 >> [v2:172.21.15.7:6834/127797392,v1:172.21.15.7:6835/127797392] conn(0x2027210 0x203ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 4 client.5499 mds_command: new command op to 5463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:53:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 1 -- 172.21.15.7:0/717558305 --> [v2:172.21.15.7:6834/127797392,v1:172.21.15.7:6835/127797392] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1dca460 con 0x2027210 2024-09-13T18:53:32.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f6158ad4700 1 --2- 172.21.15.7:0/717558305 >> [v2:172.21.15.7:6834/127797392,v1:172.21.15.7:6835/127797392] conn(0x2027210 0x203ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f6158ad4700 1 --2- 172.21.15.7:0/717558305 >> [v2:172.21.15.7:6834/127797392,v1:172.21.15.7:6835/127797392] conn(0x2027210 0x203ec90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9b95244e4add23b server_cookie=f8b14a729c5bede in_seq=0 out_seq=0 2024-09-13T18:53:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f61337fe700 10 client.5499 ms_handle_connect on v2:172.21.15.7:6834/127797392 2024-09-13T18:53:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f61337fe700 1 -- 172.21.15.7:0/717558305 <== mds.0 v2:172.21.15.7:6834/127797392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x1dca460 con 0x2027210 2024-09-13T18:53:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f61337fe700 10 client.5499 handle_command_reply: tid=0 2024-09-13T18:53:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 1 client.5499 shutdown 2024-09-13T18:53:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 1 -- 172.21.15.7:0/717558305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 msgr2=0x1fe5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 1 --2- 172.21.15.7:0/717558305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x1fe5880 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f614c018040 tx=0x7f614c001d70 comp rx=0 tx=0).stop 2024-09-13T18:53:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 1 -- 172.21.15.7:0/717558305 shutdown_connections 2024-09-13T18:53:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 1 --2- 172.21.15.7:0/717558305 >> [v2:172.21.15.7:6834/127797392,v1:172.21.15.7:6835/127797392] conn(0x2027210 0x203ec90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 1 --2- 172.21.15.7:0/717558305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x1fe5880 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 1 --2- 172.21.15.7:0/717558305 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x202b230 0x1fe38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 1 --2- 172.21.15.7:0/717558305 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2009e90 0x2005c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f61337fe700 0 client.5499 ms_handle_reset on v2:172.21.15.7:6834/127797392 2024-09-13T18:53:32.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 1 -- 172.21.15.7:0/717558305 >> 172.21.15.7:0/717558305 conn(0x1d734d0 msgr2=0x2006470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:32.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 1 -- 172.21.15.7:0/717558305 shutdown_connections 2024-09-13T18:53:32.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 1 -- 172.21.15.7:0/717558305 wait complete. 2024-09-13T18:53:32.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.512+0000 7f615dfe9740 20 client.5499 trim_cache size 0 max 16384 2024-09-13T18:53:32.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.536+0000 7f615dfe9740 1 Processor -- start 2024-09-13T18:53:32.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.536+0000 7f615dfe9740 1 -- start start 2024-09-13T18:53:32.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.536+0000 7f615dfe9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2009e90 0x202ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.536+0000 7f615dfe9740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x202b230 0x20304a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.536+0000 7f615dfe9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x203aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.536+0000 7f615dfe9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x1fe3ca0 con 0x202bb50 2024-09-13T18:53:32.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.536+0000 7f615dfe9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x1eea470 con 0x2009e90 2024-09-13T18:53:32.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.536+0000 7f615dfe9740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f7aec0 con 0x202b230 2024-09-13T18:53:32.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.536+0000 7f61592d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x203aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.536+0000 7f61592d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x203aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34952/0 (socket says 172.21.15.7:34952) 2024-09-13T18:53:32.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.536+0000 7f61592d5700 1 -- 172.21.15.7:0/3623220895 learned_addr learned my addr 172.21.15.7:0/3623220895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:32.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.536+0000 7f6158ad4700 1 --2- 172.21.15.7:0/3623220895 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2009e90 0x202ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.536+0000 7f6153fff700 1 --2- 172.21.15.7:0/3623220895 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x202b230 0x20304a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.536+0000 7f61592d5700 1 -- 172.21.15.7:0/3623220895 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x202b230 msgr2=0x20304a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.536+0000 7f61592d5700 1 --2- 172.21.15.7:0/3623220895 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x202b230 0x20304a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.536+0000 7f61592d5700 1 -- 172.21.15.7:0/3623220895 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2009e90 msgr2=0x202ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.536+0000 7f61592d5700 1 --2- 172.21.15.7:0/3623220895 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2009e90 0x202ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.536+0000 7f61592d5700 1 -- 172.21.15.7:0/3623220895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1ef8450 con 0x202bb50 2024-09-13T18:53:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.540+0000 7f6153fff700 1 --2- 172.21.15.7:0/3623220895 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x202b230 0x20304a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.540+0000 7f61592d5700 1 --2- 172.21.15.7:0/3623220895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x203aa80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f614c0157d0 tx=0x7f614c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c5394a2554b66d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.540+0000 7f613affd700 1 -- 172.21.15.7:0/3623220895 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c001880 con 0x202bb50 2024-09-13T18:53:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.540+0000 7f613affd700 1 -- 172.21.15.7:0/3623220895 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c029470 con 0x202bb50 2024-09-13T18:53:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.540+0000 7f613affd700 1 -- 172.21.15.7:0/3623220895 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c00ab60 con 0x202bb50 2024-09-13T18:53:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.540+0000 7f615dfe9740 1 -- 172.21.15.7:0/3623220895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 msgr2=0x203aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:32.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.540+0000 7f615dfe9740 1 --2- 172.21.15.7:0/3623220895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x203aa80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f614c0157d0 tx=0x7f614c022000 comp rx=0 tx=0).stop 2024-09-13T18:53:32.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.540+0000 7f615dfe9740 1 -- 172.21.15.7:0/3623220895 shutdown_connections 2024-09-13T18:53:32.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.540+0000 7f615dfe9740 1 --2- 172.21.15.7:0/3623220895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x203aa80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.540+0000 7f615dfe9740 1 --2- 172.21.15.7:0/3623220895 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x202b230 0x20304a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.540+0000 7f615dfe9740 1 --2- 172.21.15.7:0/3623220895 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2009e90 0x202ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.540+0000 7f615dfe9740 1 -- 172.21.15.7:0/3623220895 >> 172.21.15.7:0/3623220895 conn(0x1d734d0 msgr2=0x1fe0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:32.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.540+0000 7f615dfe9740 1 -- 172.21.15.7:0/3623220895 shutdown_connections 2024-09-13T18:53:32.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.540+0000 7f615dfe9740 1 -- 172.21.15.7:0/3623220895 wait complete. 2024-09-13T18:53:32.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.540+0000 7f615dfe9740 1 -- start start 2024-09-13T18:53:32.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.540+0000 7f615dfe9740 1 Processor -- start 2024-09-13T18:53:32.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f615dfe9740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2009e90 0x208b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f615dfe9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x202b230 0x208bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f615dfe9740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x208c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f615dfe9740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x1f45bd0 con 0x202bb50 2024-09-13T18:53:32.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f615dfe9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x20498a0 con 0x202b230 2024-09-13T18:53:32.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f615dfe9740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x2030530 con 0x2009e90 2024-09-13T18:53:32.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f615dfe9740 10 client.0 _ll_register_callbacks cb 0x1efdce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:53:32.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f61592d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x208c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f6158ad4700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2009e90 0x208b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f6153fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x202b230 0x208bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f61592d5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x208c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34968/0 (socket says 172.21.15.7:34968) 2024-09-13T18:53:32.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f61592d5700 1 -- 172.21.15.7:0/893722195 learned_addr learned my addr 172.21.15.7:0/893722195 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:32.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f6153fff700 1 -- 172.21.15.7:0/893722195 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2009e90 msgr2=0x208b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:32.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f6153fff700 1 --2- 172.21.15.7:0/893722195 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2009e90 0x208b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f6153fff700 1 -- 172.21.15.7:0/893722195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 msgr2=0x208c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:32.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f6153fff700 1 --2- 172.21.15.7:0/893722195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x208c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f6153fff700 1 -- 172.21.15.7:0/893722195 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1f12220 con 0x202b230 2024-09-13T18:53:32.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f6153fff700 1 --2- 172.21.15.7:0/893722195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x202b230 0x208bf20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f61541559a0 tx=0x7f6144007d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fecf175aa0d428d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:32.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f61337fe700 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-13T18:53:32.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f61337fe700 1 -- 172.21.15.7:0/893722195 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6144028070 con 0x202b230 2024-09-13T18:53:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f61337fe700 1 -- 172.21.15.7:0/893722195 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f614402eb30 con 0x202b230 2024-09-13T18:53:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f615dfe9740 1 -- 172.21.15.7:0/893722195 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f614c00acc0 con 0x202b230 2024-09-13T18:53:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f61337fe700 1 -- 172.21.15.7:0/893722195 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f614402ecf0 con 0x202b230 2024-09-13T18:53:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f615dfe9740 10 client.5437 fetch_fsmap learned FSMap version 22 2024-09-13T18:53:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f615dfe9740 1 -- 172.21.15.7:0/893722195 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x1eec720 con 0x202b230 2024-09-13T18:53:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f61337fe700 1 -- 172.21.15.7:0/893722195 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c00acc0 con 0x202b230 2024-09-13T18:53:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f61337fe700 1 -- 172.21.15.7:0/893722195 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 22) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6144022600 con 0x202b230 2024-09-13T18:53:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f615dfe9740 10 client.5437 fetch_fsmap finished waiting for FSMap version 22 2024-09-13T18:53:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f615dfe9740 10 client.5437 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T18:53:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.544+0000 7f615dfe9740 20 client.5437 populate_metadata read hostname 'smithi007' 2024-09-13T18:53:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f615dfe9740 1 --2- 172.21.15.7:0/893722195 >> [v2:172.21.15.7:6834/127797392,v1:172.21.15.7:6835/127797392] conn(0x202df40 0x202e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:32.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f615dfe9740 4 client.5437 mds_command: new command op to 5463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:53:32.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f615dfe9740 1 -- 172.21.15.7:0/893722195 --> [v2:172.21.15.7:6834/127797392,v1:172.21.15.7:6835/127797392] -- command(tid 0: {"prefix": "status"}) v1 -- 0x1def1e0 con 0x202df40 2024-09-13T18:53:32.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f6158ad4700 1 --2- 172.21.15.7:0/893722195 >> [v2:172.21.15.7:6834/127797392,v1:172.21.15.7:6835/127797392] conn(0x202df40 0x202e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:32.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f6158ad4700 1 --2- 172.21.15.7:0/893722195 >> [v2:172.21.15.7:6834/127797392,v1:172.21.15.7:6835/127797392] conn(0x202df40 0x202e320 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44b0034cec9266aa server_cookie=be40294853aed8cc in_seq=0 out_seq=0 2024-09-13T18:53:32.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f61337fe700 10 client.5437 ms_handle_connect on v2:172.21.15.7:6834/127797392 2024-09-13T18:53:32.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f61337fe700 1 -- 172.21.15.7:0/893722195 <== mds.0 v2:172.21.15.7:6834/127797392 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x1def1e0 con 0x202df40 2024-09-13T18:53:32.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f61337fe700 10 client.5437 handle_command_reply: tid=0 2024-09-13T18:53:32.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f615dfe9740 1 client.5437 shutdown 2024-09-13T18:53:32.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f615dfe9740 1 -- 172.21.15.7:0/893722195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x202b230 msgr2=0x208bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:32.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f615dfe9740 1 --2- 172.21.15.7:0/893722195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x202b230 0x208bf20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f61541559a0 tx=0x7f6144007d70 comp rx=0 tx=0).stop 2024-09-13T18:53:32.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f615dfe9740 1 -- 172.21.15.7:0/893722195 shutdown_connections 2024-09-13T18:53:32.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f615dfe9740 1 --2- 172.21.15.7:0/893722195 >> [v2:172.21.15.7:6834/127797392,v1:172.21.15.7:6835/127797392] conn(0x202df40 0x202e320 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f615dfe9740 1 --2- 172.21.15.7:0/893722195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x202bb50 0x208c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f615dfe9740 1 --2- 172.21.15.7:0/893722195 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x202b230 0x208bf20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f61337fe700 0 client.5437 ms_handle_reset on v2:172.21.15.7:6834/127797392 2024-09-13T18:53:32.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f615dfe9740 1 --2- 172.21.15.7:0/893722195 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2009e90 0x208b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f615dfe9740 1 -- 172.21.15.7:0/893722195 >> 172.21.15.7:0/893722195 conn(0x1d734d0 msgr2=0x203aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:32.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f615dfe9740 1 -- 172.21.15.7:0/893722195 shutdown_connections 2024-09-13T18:53:32.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f615dfe9740 1 -- 172.21.15.7:0/893722195 wait complete. 2024-09-13T18:53:32.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f615dfe9740 20 client.5437 trim_cache size 0 max 16384 2024-09-13T18:53:32.558 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:53:32.558 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "33406d92-f401-438a-876e-a56de3962d8e", 2024-09-13T18:53:32.558 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-13T18:53:32.559 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5463, 2024-09-13T18:53:32.559 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-13T18:53:32.559 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-13T18:53:32.559 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-13T18:53:32.559 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.3048374569999996, 2024-09-13T18:53:32.559 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 22, 2024-09-13T18:53:32.559 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 40, 2024-09-13T18:53:32.559 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 40, 2024-09-13T18:53:32.559 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 6.3367247930000001 2024-09-13T18:53:32.559 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:53:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f6159ad6700 1 -- 172.21.15.7:0/378734327 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6140066760 msgr2=0x7f6140068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f6159ad6700 1 --2- 172.21.15.7:0/378734327 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6140066760 0x7f6140068c20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f61541f61f0 tx=0x7f6144009040 comp rx=0 tx=0).stop 2024-09-13T18:53:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f6159ad6700 1 -- 172.21.15.7:0/378734327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f613c003bb0 msgr2=0x7f61540ca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f6159ad6700 1 --2- 172.21.15.7:0/378734327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f613c003bb0 0x7f61540ca7f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f613c00c080 tx=0x7f61480047e0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f6159ad6700 1 -- 172.21.15.7:0/378734327 shutdown_connections 2024-09-13T18:53:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f6159ad6700 1 --2- 172.21.15.7:0/378734327 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6140066760 0x7f6140068c20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f6159ad6700 1 --2- 172.21.15.7:0/378734327 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f615406c5c0 0x7f61541470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f6159ad6700 1 --2- 172.21.15.7:0/378734327 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f61540cd010 0x7f615406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f6159ad6700 1 --2- 172.21.15.7:0/378734327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f613c003bb0 0x7f61540ca7f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:32.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f6159ad6700 1 -- 172.21.15.7:0/378734327 >> 172.21.15.7:0/378734327 conn(0x7f61540d2d10 msgr2=0x7f61540b5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:32.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f6159ad6700 1 -- 172.21.15.7:0/378734327 shutdown_connections 2024-09-13T18:53:32.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:32.548+0000 7f6159ad6700 1 -- 172.21.15.7:0/378734327 wait complete. 2024-09-13T18:53:32.575 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:53:32.575 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:32.582 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:53:32.582 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-09-13T18:53:32.583 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:53:32.583 DEBUG:teuthology.orchestra.run.smithi007:> with open("/home/ubuntu/cephtest/mnt.0/open_file0", '"'"'w'"'"') as f: 2024-09-13T18:53:32.583 DEBUG:teuthology.orchestra.run.smithi007:> f.write("content") 2024-09-13T18:53:32.583 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2024-09-13T18:53:32.583 DEBUG:teuthology.orchestra.run.smithi007:> while True: 2024-09-13T18:53:32.583 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(1) 2024-09-13T18:53:32.583 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-13T18:53:32.585 DEBUG:teuthology.orchestra.run.smithi007:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file0 2024-09-13T18:53:32.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:53:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/open_file0': No such file or directory 2024-09-13T18:53:33.592 DEBUG:teuthology.orchestra.run.smithi007:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file0 2024-09-13T18:53:33.600 INFO:teuthology.orchestra.run.smithi007.stdout:7 2024-09-13T18:53:33.600 INFO:tasks.cephfs.mount:File open_file0 became visible with size 7 from 0 after 1s 2024-09-13T18:53:33.600 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:53:33.601 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:33.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:33.608 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file0 bs=1M conv=fdatasync count=1 seek=0) 2024-09-13T18:53:33.690 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-13T18:53:33.690 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-13T18:53:33.691 INFO:teuthology.orchestra.run.smithi007.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221866 s, 47.3 MB/s 2024-09-13T18:53:33.691 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:53:33.692 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:33.698 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:53:33.698 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-09-13T18:53:33.698 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:53:33.698 DEBUG:teuthology.orchestra.run.smithi007:> with open("/home/ubuntu/cephtest/mnt.0/open_file1", '"'"'w'"'"') as f: 2024-09-13T18:53:33.698 DEBUG:teuthology.orchestra.run.smithi007:> f.write("content") 2024-09-13T18:53:33.698 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2024-09-13T18:53:33.698 DEBUG:teuthology.orchestra.run.smithi007:> while True: 2024-09-13T18:53:33.699 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(1) 2024-09-13T18:53:33.699 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-13T18:53:33.701 DEBUG:teuthology.orchestra.run.smithi007:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file1 2024-09-13T18:53:33.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:53:33.708 INFO:teuthology.orchestra.run.smithi007.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/open_file1': No such file or directory 2024-09-13T18:53:34.709 DEBUG:teuthology.orchestra.run.smithi007:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file1 2024-09-13T18:53:34.719 INFO:teuthology.orchestra.run.smithi007.stdout:7 2024-09-13T18:53:34.719 INFO:tasks.cephfs.mount:File open_file1 became visible with size 7 from 0 after 1s 2024-09-13T18:53:34.719 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:53:34.720 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:34.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:34.728 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file1 bs=1M conv=fdatasync count=1 seek=0) 2024-09-13T18:53:34.815 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-13T18:53:34.815 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-13T18:53:34.816 INFO:teuthology.orchestra.run.smithi007.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218329 s, 48.0 MB/s 2024-09-13T18:53:34.817 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:53:34.817 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:34.823 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:53:34.823 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-09-13T18:53:34.823 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:53:34.823 DEBUG:teuthology.orchestra.run.smithi007:> with open("/home/ubuntu/cephtest/mnt.0/open_file2", '"'"'w'"'"') as f: 2024-09-13T18:53:34.824 DEBUG:teuthology.orchestra.run.smithi007:> f.write("content") 2024-09-13T18:53:34.824 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2024-09-13T18:53:34.824 DEBUG:teuthology.orchestra.run.smithi007:> while True: 2024-09-13T18:53:34.824 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(1) 2024-09-13T18:53:34.824 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-13T18:53:34.826 DEBUG:teuthology.orchestra.run.smithi007:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file2 2024-09-13T18:53:34.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:53:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/open_file2': No such file or directory 2024-09-13T18:53:35.836 DEBUG:teuthology.orchestra.run.smithi007:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file2 2024-09-13T18:53:35.845 INFO:teuthology.orchestra.run.smithi007.stdout:7 2024-09-13T18:53:35.845 INFO:tasks.cephfs.mount:File open_file2 became visible with size 7 from 0 after 1s 2024-09-13T18:53:35.845 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:53:35.845 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:35.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:35.853 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file2 bs=1M conv=fdatasync count=1 seek=0) 2024-09-13T18:53:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-13T18:53:35.891 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-13T18:53:35.892 INFO:teuthology.orchestra.run.smithi007.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195386 s, 53.7 MB/s 2024-09-13T18:53:35.892 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:53:35.893 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:35.941 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:53:35.942 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-09-13T18:53:35.942 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:53:35.942 DEBUG:teuthology.orchestra.run.smithi007:> with open("/home/ubuntu/cephtest/mnt.0/open_file3", '"'"'w'"'"') as f: 2024-09-13T18:53:35.943 DEBUG:teuthology.orchestra.run.smithi007:> f.write("content") 2024-09-13T18:53:35.943 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2024-09-13T18:53:35.943 DEBUG:teuthology.orchestra.run.smithi007:> while True: 2024-09-13T18:53:35.943 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(1) 2024-09-13T18:53:35.943 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-13T18:53:35.945 DEBUG:teuthology.orchestra.run.smithi007:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file3 2024-09-13T18:53:35.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:53:35.953 INFO:teuthology.orchestra.run.smithi007.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/open_file3': No such file or directory 2024-09-13T18:53:36.954 DEBUG:teuthology.orchestra.run.smithi007:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file3 2024-09-13T18:53:36.962 INFO:teuthology.orchestra.run.smithi007.stdout:7 2024-09-13T18:53:36.963 INFO:tasks.cephfs.mount:File open_file3 became visible with size 7 from 0 after 1s 2024-09-13T18:53:36.963 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:53:36.963 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:36.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:36.970 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file3 bs=1M conv=fdatasync count=1 seek=0) 2024-09-13T18:53:37.006 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-13T18:53:37.006 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-13T18:53:37.006 INFO:teuthology.orchestra.run.smithi007.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184868 s, 56.7 MB/s 2024-09-13T18:53:37.007 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:53:37.007 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:37.013 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:53:37.013 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-09-13T18:53:37.013 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:53:37.014 DEBUG:teuthology.orchestra.run.smithi007:> with open("/home/ubuntu/cephtest/mnt.0/open_file4", '"'"'w'"'"') as f: 2024-09-13T18:53:37.014 DEBUG:teuthology.orchestra.run.smithi007:> f.write("content") 2024-09-13T18:53:37.014 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2024-09-13T18:53:37.014 DEBUG:teuthology.orchestra.run.smithi007:> while True: 2024-09-13T18:53:37.014 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(1) 2024-09-13T18:53:37.014 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-13T18:53:37.059 DEBUG:teuthology.orchestra.run.smithi007:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file4 2024-09-13T18:53:37.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:53:37.069 INFO:teuthology.orchestra.run.smithi007.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/open_file4': No such file or directory 2024-09-13T18:53:38.070 DEBUG:teuthology.orchestra.run.smithi007:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file4 2024-09-13T18:53:38.079 INFO:teuthology.orchestra.run.smithi007.stdout:7 2024-09-13T18:53:38.079 INFO:tasks.cephfs.mount:File open_file4 became visible with size 7 from 0 after 1s 2024-09-13T18:53:38.079 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:53:38.079 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:38.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:38.128 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file4 bs=1M conv=fdatasync count=1 seek=0) 2024-09-13T18:53:38.167 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-13T18:53:38.167 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-13T18:53:38.167 INFO:teuthology.orchestra.run.smithi007.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180041 s, 58.2 MB/s 2024-09-13T18:53:38.168 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:53:38.168 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:38.175 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:53:38.176 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-09-13T18:53:38.176 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:53:38.176 DEBUG:teuthology.orchestra.run.smithi007:> with open("/home/ubuntu/cephtest/mnt.0/open_file5", '"'"'w'"'"') as f: 2024-09-13T18:53:38.176 DEBUG:teuthology.orchestra.run.smithi007:> f.write("content") 2024-09-13T18:53:38.176 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2024-09-13T18:53:38.176 DEBUG:teuthology.orchestra.run.smithi007:> while True: 2024-09-13T18:53:38.176 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(1) 2024-09-13T18:53:38.176 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-13T18:53:38.178 DEBUG:teuthology.orchestra.run.smithi007:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file5 2024-09-13T18:53:38.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:53:38.186 INFO:teuthology.orchestra.run.smithi007.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/open_file5': No such file or directory 2024-09-13T18:53:39.187 DEBUG:teuthology.orchestra.run.smithi007:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file5 2024-09-13T18:53:39.196 INFO:teuthology.orchestra.run.smithi007.stdout:7 2024-09-13T18:53:39.196 INFO:tasks.cephfs.mount:File open_file5 became visible with size 7 from 0 after 1s 2024-09-13T18:53:39.196 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:53:39.196 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:39.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:39.203 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file5 bs=1M conv=fdatasync count=1 seek=0) 2024-09-13T18:53:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-13T18:53:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-13T18:53:39.243 INFO:teuthology.orchestra.run.smithi007.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192674 s, 54.4 MB/s 2024-09-13T18:53:39.244 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:53:39.244 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:39.250 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:53:39.250 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-09-13T18:53:39.250 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:53:39.250 DEBUG:teuthology.orchestra.run.smithi007:> with open("/home/ubuntu/cephtest/mnt.0/open_file6", '"'"'w'"'"') as f: 2024-09-13T18:53:39.250 DEBUG:teuthology.orchestra.run.smithi007:> f.write("content") 2024-09-13T18:53:39.250 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2024-09-13T18:53:39.250 DEBUG:teuthology.orchestra.run.smithi007:> while True: 2024-09-13T18:53:39.250 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(1) 2024-09-13T18:53:39.250 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-13T18:53:39.295 DEBUG:teuthology.orchestra.run.smithi007:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file6 2024-09-13T18:53:39.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:53:39.304 INFO:teuthology.orchestra.run.smithi007.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/open_file6': No such file or directory 2024-09-13T18:53:40.305 DEBUG:teuthology.orchestra.run.smithi007:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file6 2024-09-13T18:53:40.313 INFO:teuthology.orchestra.run.smithi007.stdout:7 2024-09-13T18:53:40.314 INFO:tasks.cephfs.mount:File open_file6 became visible with size 7 from 0 after 1s 2024-09-13T18:53:40.314 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:53:40.314 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:40.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:40.321 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file6 bs=1M conv=fdatasync count=1 seek=0) 2024-09-13T18:53:40.402 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-13T18:53:40.402 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-13T18:53:40.402 INFO:teuthology.orchestra.run.smithi007.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205175 s, 51.1 MB/s 2024-09-13T18:53:40.403 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:53:40.403 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:40.411 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:53:40.411 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-09-13T18:53:40.411 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:53:40.411 DEBUG:teuthology.orchestra.run.smithi007:> with open("/home/ubuntu/cephtest/mnt.0/open_file7", '"'"'w'"'"') as f: 2024-09-13T18:53:40.411 DEBUG:teuthology.orchestra.run.smithi007:> f.write("content") 2024-09-13T18:53:40.411 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2024-09-13T18:53:40.411 DEBUG:teuthology.orchestra.run.smithi007:> while True: 2024-09-13T18:53:40.411 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(1) 2024-09-13T18:53:40.412 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-13T18:53:40.414 DEBUG:teuthology.orchestra.run.smithi007:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file7 2024-09-13T18:53:40.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:53:40.422 INFO:teuthology.orchestra.run.smithi007.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/open_file7': No such file or directory 2024-09-13T18:53:41.423 DEBUG:teuthology.orchestra.run.smithi007:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/open_file7 2024-09-13T18:53:41.432 INFO:teuthology.orchestra.run.smithi007.stdout:7 2024-09-13T18:53:41.432 INFO:tasks.cephfs.mount:File open_file7 became visible with size 7 from 0 after 1s 2024-09-13T18:53:41.432 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:53:41.432 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:41.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:41.481 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=open_file7 bs=1M conv=fdatasync count=1 seek=0) 2024-09-13T18:53:41.514 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-13T18:53:41.514 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-13T18:53:41.514 INFO:teuthology.orchestra.run.smithi007.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016643 s, 63.0 MB/s 2024-09-13T18:53:51.516 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat mds0_openfiles.1 2024-09-13T18:53:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.551+0000 7f2dd4a28140 1 Processor -- start 2024-09-13T18:53:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.551+0000 7f2dd4a28140 1 -- start start 2024-09-13T18:53:51.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd4a28140 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561fdd6f48e0 0x561fdd7760c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd4a28140 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561fdd6f4cc0 0x561fdd798040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd4a28140 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x561fdd77b140 0x561fdd77b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd4a28140 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fdd6542f0 con 0x561fdd6f48e0 2024-09-13T18:53:51.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd4a28140 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fdd6108e0 con 0x561fdd6f4cc0 2024-09-13T18:53:51.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd4a28140 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fdd610580 con 0x561fdd77b140 2024-09-13T18:53:51.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd255d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561fdd6f48e0 0x561fdd7760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd255d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561fdd6f48e0 0x561fdd7760c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33158/0 (socket says 172.21.15.7:33158) 2024-09-13T18:53:51.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd255d700 1 -- 172.21.15.7:0/1057794606 learned_addr learned my addr 172.21.15.7:0/1057794606 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:51.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd1d5c700 1 --2- 172.21.15.7:0/1057794606 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561fdd6f4cc0 0x561fdd798040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd2d5e700 1 --2- 172.21.15.7:0/1057794606 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x561fdd77b140 0x561fdd77b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd255d700 1 -- 172.21.15.7:0/1057794606 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x561fdd77b140 msgr2=0x561fdd77b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:51.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd255d700 1 --2- 172.21.15.7:0/1057794606 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x561fdd77b140 0x561fdd77b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd255d700 1 -- 172.21.15.7:0/1057794606 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561fdd6f4cc0 msgr2=0x561fdd798040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:51.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd255d700 1 --2- 172.21.15.7:0/1057794606 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561fdd6f4cc0 0x561fdd798040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd255d700 1 -- 172.21.15.7:0/1057794606 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc005d20 con 0x561fdd6f48e0 2024-09-13T18:53:51.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd2d5e700 1 --2- 172.21.15.7:0/1057794606 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x561fdd77b140 0x561fdd77b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:51.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd255d700 1 --2- 172.21.15.7:0/1057794606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561fdd6f48e0 0x561fdd7760c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400a810 tx=0x7f2dc4005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1cf05be35cdeb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd0d5a700 1 -- 172.21.15.7:0/1057794606 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4012660 con 0x561fdd6f48e0 2024-09-13T18:53:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd0d5a700 1 -- 172.21.15.7:0/1057794606 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc40127c0 con 0x561fdd6f48e0 2024-09-13T18:53:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd0d5a700 1 -- 172.21.15.7:0/1057794606 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4012b50 con 0x561fdd6f48e0 2024-09-13T18:53:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd4a28140 1 -- 172.21.15.7:0/1057794606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561fdd6f48e0 msgr2=0x561fdd7760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd4a28140 1 --2- 172.21.15.7:0/1057794606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561fdd6f48e0 0x561fdd7760c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400a810 tx=0x7f2dc4005d80 comp rx=0 tx=0).stop 2024-09-13T18:53:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd4a28140 1 -- 172.21.15.7:0/1057794606 shutdown_connections 2024-09-13T18:53:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd4a28140 1 --2- 172.21.15.7:0/1057794606 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x561fdd77b140 0x561fdd77b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd4a28140 1 --2- 172.21.15.7:0/1057794606 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561fdd6f4cc0 0x561fdd798040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd4a28140 1 --2- 172.21.15.7:0/1057794606 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561fdd6f48e0 0x561fdd7760c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd4a28140 1 -- 172.21.15.7:0/1057794606 >> 172.21.15.7:0/1057794606 conn(0x561fdd6f3270 msgr2=0x561fdd77a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd4a28140 1 -- 172.21.15.7:0/1057794606 shutdown_connections 2024-09-13T18:53:51.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.555+0000 7f2dd4a28140 1 -- 172.21.15.7:0/1057794606 wait complete. 2024-09-13T18:53:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd4a28140 1 Processor -- start 2024-09-13T18:53:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd4a28140 1 -- start start 2024-09-13T18:53:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd4a28140 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x561fdd6f48e0 0x561fdd82d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd4a28140 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561fdd6f4cc0 0x561fdd79b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd4a28140 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561fdd77b140 0x561fdd79b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd4a28140 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fdd5e68d0 con 0x561fdd6f4cc0 2024-09-13T18:53:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd4a28140 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fdd77bdd0 con 0x561fdd77b140 2024-09-13T18:53:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd4a28140 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fdd7962c0 con 0x561fdd6f48e0 2024-09-13T18:53:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd1d5c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561fdd6f4cc0 0x561fdd79b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd1d5c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561fdd6f4cc0 0x561fdd79b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33160/0 (socket says 172.21.15.7:33160) 2024-09-13T18:53:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd1d5c700 1 -- 172.21.15.7:0/3803493514 learned_addr learned my addr 172.21.15.7:0/3803493514 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd255d700 1 --2- 172.21.15.7:0/3803493514 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x561fdd6f48e0 0x561fdd82d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd2d5e700 1 --2- 172.21.15.7:0/3803493514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561fdd77b140 0x561fdd79b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd1d5c700 1 -- 172.21.15.7:0/3803493514 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x561fdd6f48e0 msgr2=0x561fdd82d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd1d5c700 1 --2- 172.21.15.7:0/3803493514 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x561fdd6f48e0 0x561fdd82d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd1d5c700 1 -- 172.21.15.7:0/3803493514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561fdd77b140 msgr2=0x561fdd79b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:51.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd1d5c700 1 --2- 172.21.15.7:0/3803493514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561fdd77b140 0x561fdd79b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd1d5c700 1 -- 172.21.15.7:0/3803493514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc400a410 con 0x561fdd6f4cc0 2024-09-13T18:53:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd2d5e700 1 --2- 172.21.15.7:0/3803493514 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561fdd77b140 0x561fdd79b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd1d5c700 1 --2- 172.21.15.7:0/3803493514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561fdd6f4cc0 0x561fdd79b100 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x561fdd70ef40 tx=0x7f2dbc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dcb7fe700 1 -- 172.21.15.7:0/3803493514 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc012610 con 0x561fdd6f4cc0 2024-09-13T18:53:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dcb7fe700 1 -- 172.21.15.7:0/3803493514 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dbc012770 con 0x561fdd6f4cc0 2024-09-13T18:53:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd4a28140 1 -- 172.21.15.7:0/3803493514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561fdd5feb00 con 0x561fdd6f4cc0 2024-09-13T18:53:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dcb7fe700 1 -- 172.21.15.7:0/3803493514 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc012b20 con 0x561fdd6f4cc0 2024-09-13T18:53:51.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.559+0000 7f2dd4a28140 1 -- 172.21.15.7:0/3803493514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561fdd5f1f40 con 0x561fdd6f4cc0 2024-09-13T18:53:51.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.563+0000 7f2dcb7fe700 1 -- 172.21.15.7:0/3803493514 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2dbc012c80 con 0x561fdd6f4cc0 2024-09-13T18:53:51.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.563+0000 7f2dcb7fe700 1 --2- 172.21.15.7:0/3803493514 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2db4065060 0x7f2db4067520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.563+0000 7f2dcb7fe700 1 -- 172.21.15.7:0/3803493514 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2dbc00ae40 con 0x561fdd6f4cc0 2024-09-13T18:53:51.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.563+0000 7f2dd255d700 1 --2- 172.21.15.7:0/3803493514 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2db4065060 0x7f2db4067520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.563+0000 7f2dd4a28140 1 --2- 172.21.15.7:0/3803493514 >> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] conn(0x561fdd706ce0 0x561fdd7070f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.563+0000 7f2dd4a28140 1 -- 172.21.15.7:0/3803493514 --> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] -- osd_op(unknown.0.0:1 4.13 4:c9647462:::mds0_openfiles.1:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x561fdd707630 con 0x561fdd706ce0 2024-09-13T18:53:51.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.567+0000 7f2dd2d5e700 1 --2- 172.21.15.7:0/3803493514 >> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] conn(0x561fdd706ce0 0x561fdd7070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.567+0000 7f2dd255d700 1 --2- 172.21.15.7:0/3803493514 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2db4065060 0x7f2db4067520 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4000f80 tx=0x7f2dc401e040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:51.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.567+0000 7f2dd2d5e700 1 --2- 172.21.15.7:0/3803493514 >> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] conn(0x561fdd706ce0 0x561fdd7070f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:51.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.567+0000 7f2dd2d5e700 1 -- 172.21.15.7:0/3803493514 <== osd.4 v2:172.21.15.171:6816/1731498321 1 ==== osd_op_reply(1 mds0_openfiles.1 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x7f2dcc00f570 con 0x561fdd706ce0 2024-09-13T18:53:51.575 INFO:teuthology.orchestra.run.smithi007.stdout:cephfs_metadata/mds0_openfiles.1 mtime 2024-09-13T18:53:46.000000+0000, size 0 2024-09-13T18:53:51.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.567+0000 7f2dd4a28140 1 -- 172.21.15.7:0/3803493514 >> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] conn(0x561fdd706ce0 msgr2=0x561fdd7070f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:51.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.567+0000 7f2dd4a28140 1 --2- 172.21.15.7:0/3803493514 >> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] conn(0x561fdd706ce0 0x561fdd7070f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.567+0000 7f2dd4a28140 1 -- 172.21.15.7:0/3803493514 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2db4065060 msgr2=0x7f2db4067520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:51.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.567+0000 7f2dd4a28140 1 --2- 172.21.15.7:0/3803493514 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2db4065060 0x7f2db4067520 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4000f80 tx=0x7f2dc401e040 comp rx=0 tx=0).stop 2024-09-13T18:53:51.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.567+0000 7f2dd4a28140 1 -- 172.21.15.7:0/3803493514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561fdd6f4cc0 msgr2=0x561fdd79b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:51.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.567+0000 7f2dd4a28140 1 --2- 172.21.15.7:0/3803493514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561fdd6f4cc0 0x561fdd79b100 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x561fdd70ef40 tx=0x7f2dbc002fb0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.567+0000 7f2dd2d5e700 1 -- 172.21.15.7:0/3803493514 reap_dead start 2024-09-13T18:53:51.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.567+0000 7f2dd4a28140 1 -- 172.21.15.7:0/3803493514 shutdown_connections 2024-09-13T18:53:51.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.567+0000 7f2dd4a28140 1 -- 172.21.15.7:0/3803493514 >> 172.21.15.7:0/3803493514 conn(0x561fdd6f3270 msgr2=0x561fdd77d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:51.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.567+0000 7f2dd4a28140 1 -- 172.21.15.7:0/3803493514 shutdown_connections 2024-09-13T18:53:51.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.567+0000 7f2dd4a28140 1 -- 172.21.15.7:0/3803493514 wait complete. 2024-09-13T18:53:51.633 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T18:53:51.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T18:53:51.652 DEBUG:teuthology.orchestra.run.smithi171:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T18:53:51.673 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T18:53:51.673 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T18:53:51.687 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T18:53:51.747 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T18:53:51.757 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T18:53:51.771 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T18:53:51.784 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T18:53:51.794 INFO:tasks.cephfs.mount:Terminating background process 2024-09-13T18:53:51.804 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:53:51.804 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:51.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:53:51.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926f2bf700 1 Processor -- start 2024-09-13T18:53:51.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926f2bf700 1 -- start start 2024-09-13T18:53:51.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926f2bf700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f92680ad6b0 0x7f926815bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926f2bf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f92680ccbb0 0x7f92680cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926f2bf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926806fc80 con 0x7f92680718d0 2024-09-13T18:53:51.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926f2bf700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926806fed0 con 0x7f92680ccbb0 2024-09-13T18:53:51.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926f2bf700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92680701b0 con 0x7f92680ad6b0 2024-09-13T18:53:51.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926eabe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f92680ccbb0 0x7f92680cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926eabe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f92680ccbb0 0x7f92680cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33162/0 (socket says 172.21.15.7:33162) 2024-09-13T18:53:51.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926eabe700 1 -- 172.21.15.7:0/1138530818 learned_addr learned my addr 172.21.15.7:0/1138530818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:51.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926e2bd700 1 --2- 172.21.15.7:0/1138530818 >> v2:172.21.15.171:3300/0 conn(0x7f92680ad6b0 0x7f926815bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926eabe700 1 -- 172.21.15.7:0/1138530818 >> v2:172.21.15.171:3300/0 conn(0x7f92680ad6b0 msgr2=0x7f926815bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:51.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926eabe700 1 --2- 172.21.15.7:0/1138530818 >> v2:172.21.15.171:3300/0 conn(0x7f92680ad6b0 0x7f926815bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926eabe700 1 -- 172.21.15.7:0/1138530818 >> v1:172.21.15.7:6789/0 conn(0x7f92680718d0 legacy=0x7f92680ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:51.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926eabe700 1 -- 172.21.15.7:0/1138530818 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9268070490 con 0x7f92680ccbb0 2024-09-13T18:53:51.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926eabe700 1 --2- 172.21.15.7:0/1138530818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92680ccbb0 0x7f92680cd040 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f926806f610 tx=0x7f926000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d637fd10960ee3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:51.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926d2bb700 1 -- 172.21.15.7:0/1138530818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926000e070 con 0x7f92680ccbb0 2024-09-13T18:53:51.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926d2bb700 1 -- 172.21.15.7:0/1138530818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926000b9c0 con 0x7f92680ccbb0 2024-09-13T18:53:51.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.927+0000 7f926d2bb700 1 -- 172.21.15.7:0/1138530818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926000bd30 con 0x7f92680ccbb0 2024-09-13T18:53:51.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.931+0000 7f926f2bf700 1 -- 172.21.15.7:0/1138530818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92680ccbb0 msgr2=0x7f92680cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:51.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.931+0000 7f926f2bf700 1 --2- 172.21.15.7:0/1138530818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92680ccbb0 0x7f92680cd040 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f926806f610 tx=0x7f926000b330 comp rx=0 tx=0).stop 2024-09-13T18:53:51.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.931+0000 7f926f2bf700 1 -- 172.21.15.7:0/1138530818 shutdown_connections 2024-09-13T18:53:51.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.931+0000 7f926f2bf700 1 --2- 172.21.15.7:0/1138530818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92680ccbb0 0x7f92680cd040 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.931+0000 7f926f2bf700 1 --2- 172.21.15.7:0/1138530818 >> v2:172.21.15.171:3300/0 conn(0x7f92680ad6b0 0x7f926815bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.931+0000 7f926f2bf700 1 -- 172.21.15.7:0/1138530818 >> 172.21.15.7:0/1138530818 conn(0x7f92680b0570 msgr2=0x7f92680b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:51.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.931+0000 7f926f2bf700 1 -- 172.21.15.7:0/1138530818 shutdown_connections 2024-09-13T18:53:51.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.931+0000 7f926f2bf700 1 -- 172.21.15.7:0/1138530818 wait complete. 2024-09-13T18:53:51.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.931+0000 7f926f2bf700 1 Processor -- start 2024-09-13T18:53:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.931+0000 7f926f2bf700 1 -- start start 2024-09-13T18:53:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.931+0000 7f926f2bf700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92680718d0 0x7f926814d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.931+0000 7f926f2bf700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f92680ad6b0 0x7f92680ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.931+0000 7f926f2bf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92680ccbb0 0x7f92680ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.931+0000 7f926f2bf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92680af7e0 con 0x7f92680ccbb0 2024-09-13T18:53:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.931+0000 7f926f2bf700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92680ae930 con 0x7f92680718d0 2024-09-13T18:53:51.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.931+0000 7f926f2bf700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92680aebd0 con 0x7f92680ad6b0 2024-09-13T18:53:51.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f926eabe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92680ccbb0 0x7f92680ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f926eabe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92680ccbb0 0x7f92680ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33164/0 (socket says 172.21.15.7:33164) 2024-09-13T18:53:51.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f926eabe700 1 -- 172.21.15.7:0/2481923953 learned_addr learned my addr 172.21.15.7:0/2481923953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:51.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f926e2bd700 1 --2- 172.21.15.7:0/2481923953 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92680718d0 0x7f926814d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f926dabc700 1 --2- 172.21.15.7:0/2481923953 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f92680ad6b0 0x7f92680ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f926eabe700 1 -- 172.21.15.7:0/2481923953 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f92680ad6b0 msgr2=0x7f92680ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:51.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f926eabe700 1 --2- 172.21.15.7:0/2481923953 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f92680ad6b0 0x7f92680ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f926eabe700 1 -- 172.21.15.7:0/2481923953 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92680718d0 msgr2=0x7f926814d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:51.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f926eabe700 1 --2- 172.21.15.7:0/2481923953 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92680718d0 0x7f926814d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:51.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f926eabe700 1 -- 172.21.15.7:0/2481923953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9260009b30 con 0x7f92680ccbb0 2024-09-13T18:53:51.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f926dabc700 1 --2- 172.21.15.7:0/2481923953 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f92680ad6b0 0x7f92680ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:51.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f926eabe700 1 --2- 172.21.15.7:0/2481923953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92680ccbb0 0x7f92680ac860 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f926000b730 tx=0x7f9260004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:51.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f925affd700 1 -- 172.21.15.7:0/2481923953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260004250 con 0x7f92680ccbb0 2024-09-13T18:53:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f925affd700 1 -- 172.21.15.7:0/2481923953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92600043b0 con 0x7f92680ccbb0 2024-09-13T18:53:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f926f2bf700 1 -- 172.21.15.7:0/2481923953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92680aeeb0 con 0x7f92680ccbb0 2024-09-13T18:53:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f925affd700 1 -- 172.21.15.7:0/2481923953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260004760 con 0x7f92680ccbb0 2024-09-13T18:53:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f926f2bf700 1 -- 172.21.15.7:0/2481923953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92680af470 con 0x7f92680ccbb0 2024-09-13T18:53:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f926f2bf700 1 -- 172.21.15.7:0/2481923953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92680af600 con 0x7f92680ccbb0 2024-09-13T18:53:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.935+0000 7f925affd700 1 -- 172.21.15.7:0/2481923953 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f926001c440 con 0x7f92680ccbb0 2024-09-13T18:53:51.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.939+0000 7f925affd700 1 --2- 172.21.15.7:0/2481923953 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f924c066730 0x7f924c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.939+0000 7f925affd700 1 -- 172.21.15.7:0/2481923953 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9260085690 con 0x7f92680ccbb0 2024-09-13T18:53:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.939+0000 7f926e2bd700 1 --2- 172.21.15.7:0/2481923953 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f924c066730 0x7f924c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:51.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.939+0000 7f926e2bd700 1 --2- 172.21.15.7:0/2481923953 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f924c066730 0x7f924c068bf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f925c009a00 tx=0x7f925c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:51.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:51.947+0000 7f925affd700 1 -- 172.21.15.7:0/2481923953 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9260052a30 con 0x7f92680ccbb0 2024-09-13T18:53:52.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.119+0000 7f926f2bf700 1 -- 172.21.15.7:0/2481923953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f926815bbe0 con 0x7f92680ccbb0 2024-09-13T18:53:52.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.123+0000 7f925affd700 1 -- 172.21.15.7:0/2481923953 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16648 (secure 0 0 0) 0x7f9260056050 con 0x7f92680ccbb0 2024-09-13T18:53:52.132 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:52.132 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:53:26.238445+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:53:15.862087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T18:53:16.885280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/2843879273":"2024-09-14T18:53:26.228249+0000","172.21.15.7:6834/2843879273":"2024-09-14T18:53:26.228249+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:53:52.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.127+0000 7f926f2bf700 1 -- 172.21.15.7:0/2481923953 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f924c066730 msgr2=0x7f924c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.127+0000 7f926f2bf700 1 --2- 172.21.15.7:0/2481923953 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f924c066730 0x7f924c068bf0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f925c009a00 tx=0x7f925c008040 comp rx=0 tx=0).stop 2024-09-13T18:53:52.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.127+0000 7f926f2bf700 1 -- 172.21.15.7:0/2481923953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92680ccbb0 msgr2=0x7f92680ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.127+0000 7f926f2bf700 1 --2- 172.21.15.7:0/2481923953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92680ccbb0 0x7f92680ac860 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f926000b730 tx=0x7f9260004af0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.127+0000 7f926f2bf700 1 -- 172.21.15.7:0/2481923953 shutdown_connections 2024-09-13T18:53:52.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.127+0000 7f926f2bf700 1 --2- 172.21.15.7:0/2481923953 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f924c066730 0x7f924c068bf0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.127+0000 7f926f2bf700 1 --2- 172.21.15.7:0/2481923953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f92680ccbb0 0x7f92680ac860 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.127+0000 7f926f2bf700 1 --2- 172.21.15.7:0/2481923953 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f92680ad6b0 0x7f92680ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.127+0000 7f926f2bf700 1 --2- 172.21.15.7:0/2481923953 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f92680718d0 0x7f926814d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.127+0000 7f926f2bf700 1 -- 172.21.15.7:0/2481923953 >> 172.21.15.7:0/2481923953 conn(0x7f92680b0570 msgr2=0x7f92680b0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:52.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.127+0000 7f926f2bf700 1 -- 172.21.15.7:0/2481923953 shutdown_connections 2024-09-13T18:53:52.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.127+0000 7f926f2bf700 1 -- 172.21.15.7:0/2481923953 wait complete. 2024-09-13T18:53:52.150 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:53:15.862087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:53:16.885280+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:53:52.151 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:53:52.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.267+0000 7f08f1571700 1 Processor -- start 2024-09-13T18:53:52.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.267+0000 7f08f1571700 1 -- start start 2024-09-13T18:53:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08f1571700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f08ec06db40 0x7f08ec164080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08f1571700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ec0cbd80 con 0x7f08ec06ad40 2024-09-13T18:53:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08f1571700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ec0cc040 con 0x7f08ec1596c0 2024-09-13T18:53:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08f1571700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ec0b5170 con 0x7f08ec06db40 2024-09-13T18:53:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08ea7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f08ec06ad40 0x7f08ec06b140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45494/0 (socket says 172.21.15.7:45494) 2024-09-13T18:53:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08ea7fc700 1 -- 172.21.15.7:0/3691198084 learned_addr learned my addr 172.21.15.7:0/3691198084 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08eb7fe700 1 --2- 172.21.15.7:0/3691198084 >> v2:172.21.15.7:3300/0 conn(0x7f08ec06db40 0x7f08ec164080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08e9ffb700 1 -- 172.21.15.7:0/3691198084 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003684802 0 0) 0x7f08ec0cbd80 con 0x7f08ec06ad40 2024-09-13T18:53:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08eb7fe700 1 -- 172.21.15.7:0/3691198084 >> v1:172.21.15.171:6789/0 conn(0x7f08ec1596c0 legacy=0x7f08ec06a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08eb7fe700 1 -- 172.21.15.7:0/3691198084 >> v1:172.21.15.7:6789/0 conn(0x7f08ec06ad40 legacy=0x7f08ec06b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08eb7fe700 1 -- 172.21.15.7:0/3691198084 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec0cc320 con 0x7f08ec06db40 2024-09-13T18:53:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08eb7fe700 1 --2- 172.21.15.7:0/3691198084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ec06db40 0x7f08ec164080 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009d90 tx=0x7f08e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a9ed3685822a78 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08e9ffb700 1 -- 172.21.15.7:0/3691198084 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e400bdd0 con 0x7f08ec06db40 2024-09-13T18:53:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08e9ffb700 1 -- 172.21.15.7:0/3691198084 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e4009e80 con 0x7f08ec06db40 2024-09-13T18:53:52.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08e9ffb700 1 -- 172.21.15.7:0/3691198084 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e40106c0 con 0x7f08ec06db40 2024-09-13T18:53:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08f1571700 1 -- 172.21.15.7:0/3691198084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ec06db40 msgr2=0x7f08ec164080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08f1571700 1 --2- 172.21.15.7:0/3691198084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ec06db40 0x7f08ec164080 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009d90 tx=0x7f08e400b330 comp rx=0 tx=0).stop 2024-09-13T18:53:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08f1571700 1 -- 172.21.15.7:0/3691198084 shutdown_connections 2024-09-13T18:53:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08f1571700 1 --2- 172.21.15.7:0/3691198084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ec06db40 0x7f08ec164080 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08f1571700 1 -- 172.21.15.7:0/3691198084 >> 172.21.15.7:0/3691198084 conn(0x7f08ec0b3fe0 msgr2=0x7f08ec0b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08f1571700 1 -- 172.21.15.7:0/3691198084 shutdown_connections 2024-09-13T18:53:52.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.271+0000 7f08f1571700 1 -- 172.21.15.7:0/3691198084 wait complete. 2024-09-13T18:53:52.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.275+0000 7f08f1571700 1 Processor -- start 2024-09-13T18:53:52.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.275+0000 7f08f1571700 1 -- start start 2024-09-13T18:53:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.275+0000 7f08f1571700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08ec06ad40 0x7f08ec14f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.275+0000 7f08f1571700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ec06db40 0x7f08ec15a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.275+0000 7f08f1571700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f08ec1596c0 0x7f08ec15a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.275+0000 7f08f1571700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0c8db0 con 0x7f08ec06db40 2024-09-13T18:53:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.275+0000 7f08f1571700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0c9000 con 0x7f08ec06ad40 2024-09-13T18:53:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.275+0000 7f08f1571700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0c92e0 con 0x7f08ec1596c0 2024-09-13T18:53:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.275+0000 7f08ea7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ec06db40 0x7f08ec15a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.275+0000 7f08ea7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ec06db40 0x7f08ec15a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33180/0 (socket says 172.21.15.7:33180) 2024-09-13T18:53:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.275+0000 7f08ea7fc700 1 -- 172.21.15.7:0/3227116442 learned_addr learned my addr 172.21.15.7:0/3227116442 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.275+0000 7f08eaffd700 1 --2- 172.21.15.7:0/3227116442 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08ec06ad40 0x7f08ec14f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.275+0000 7f08eb7fe700 1 --2- 172.21.15.7:0/3227116442 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f08ec1596c0 0x7f08ec15a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.275+0000 7f08ea7fc700 1 -- 172.21.15.7:0/3227116442 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f08ec1596c0 msgr2=0x7f08ec15a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.275+0000 7f08ea7fc700 1 --2- 172.21.15.7:0/3227116442 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f08ec1596c0 0x7f08ec15a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.275+0000 7f08ea7fc700 1 -- 172.21.15.7:0/3227116442 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08ec06ad40 msgr2=0x7f08ec14f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.275+0000 7f08ea7fc700 1 --2- 172.21.15.7:0/3227116442 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08ec06ad40 0x7f08ec14f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.275+0000 7f08ea7fc700 1 -- 172.21.15.7:0/3227116442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e4009a70 con 0x7f08ec06db40 2024-09-13T18:53:52.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.279+0000 7f08ea7fc700 1 --2- 172.21.15.7:0/3227116442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ec06db40 0x7f08ec15a290 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f08e000c550 tx=0x7f08e000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.279+0000 7f08d37fe700 1 -- 172.21.15.7:0/3227116442 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0004ca0 con 0x7f08ec06db40 2024-09-13T18:53:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.279+0000 7f08d37fe700 1 -- 172.21.15.7:0/3227116442 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e0004e00 con 0x7f08ec06db40 2024-09-13T18:53:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.279+0000 7f08f1571700 1 -- 172.21.15.7:0/3227116442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ec0c95c0 con 0x7f08ec06db40 2024-09-13T18:53:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.279+0000 7f08d37fe700 1 -- 172.21.15.7:0/3227116442 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0007310 con 0x7f08ec06db40 2024-09-13T18:53:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.279+0000 7f08f1571700 1 -- 172.21.15.7:0/3227116442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec0c9b80 con 0x7f08ec06db40 2024-09-13T18:53:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.279+0000 7f08f1571700 1 -- 172.21.15.7:0/3227116442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ec066410 con 0x7f08ec06db40 2024-09-13T18:53:52.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.279+0000 7f08d37fe700 1 -- 172.21.15.7:0/3227116442 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f08e0007470 con 0x7f08ec06db40 2024-09-13T18:53:52.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.279+0000 7f08d37fe700 1 --2- 172.21.15.7:0/3227116442 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f08d8066730 0x7f08d8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.279+0000 7f08d37fe700 1 -- 172.21.15.7:0/3227116442 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f08e0083ce0 con 0x7f08ec06db40 2024-09-13T18:53:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.283+0000 7f08eaffd700 1 --2- 172.21.15.7:0/3227116442 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f08d8066730 0x7f08d8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.283+0000 7f08eaffd700 1 --2- 172.21.15.7:0/3227116442 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f08d8066730 0x7f08d8068bf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f08dc000f60 tx=0x7f08dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.287+0000 7f08d37fe700 1 -- 172.21.15.7:0/3227116442 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f08e0051080 con 0x7f08ec06db40 2024-09-13T18:53:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.463+0000 7f08f1571700 1 -- 172.21.15.7:0/3227116442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08ec15cd80 con 0x7f08ec06db40 2024-09-13T18:53:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.463+0000 7f08d37fe700 1 -- 172.21.15.7:0/3227116442 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08e0007720 con 0x7f08ec06db40 2024-09-13T18:53:52.471 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:53:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.463+0000 7f08f1571700 1 -- 172.21.15.7:0/3227116442 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f08d8066730 msgr2=0x7f08d8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.463+0000 7f08f1571700 1 --2- 172.21.15.7:0/3227116442 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f08d8066730 0x7f08d8068bf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f08dc000f60 tx=0x7f08dc009040 comp rx=0 tx=0).stop 2024-09-13T18:53:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.463+0000 7f08f1571700 1 -- 172.21.15.7:0/3227116442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ec06db40 msgr2=0x7f08ec15a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.463+0000 7f08f1571700 1 --2- 172.21.15.7:0/3227116442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ec06db40 0x7f08ec15a290 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f08e000c550 tx=0x7f08e000c860 comp rx=0 tx=0).stop 2024-09-13T18:53:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.463+0000 7f08f1571700 1 -- 172.21.15.7:0/3227116442 shutdown_connections 2024-09-13T18:53:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.463+0000 7f08f1571700 1 --2- 172.21.15.7:0/3227116442 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f08d8066730 0x7f08d8068bf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.463+0000 7f08f1571700 1 --2- 172.21.15.7:0/3227116442 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f08ec1596c0 0x7f08ec15a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.463+0000 7f08f1571700 1 --2- 172.21.15.7:0/3227116442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08ec06db40 0x7f08ec15a290 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.463+0000 7f08f1571700 1 --2- 172.21.15.7:0/3227116442 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f08ec06ad40 0x7f08ec14f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.463+0000 7f08f1571700 1 -- 172.21.15.7:0/3227116442 >> 172.21.15.7:0/3227116442 conn(0x7f08ec0b3fe0 msgr2=0x7f08ec155930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.463+0000 7f08f1571700 1 -- 172.21.15.7:0/3227116442 shutdown_connections 2024-09-13T18:53:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.463+0000 7f08f1571700 1 -- 172.21.15.7:0/3227116442 wait complete. 2024-09-13T18:53:52.487 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:53:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5ffff700 1 Processor -- start 2024-09-13T18:53:52.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5ffff700 1 -- start start 2024-09-13T18:53:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5ffff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f5d60160ff0 0x7f5d600ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5ffff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5d6006cdc0 0x7f5d60164b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5ffff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d600b3b40 con 0x7f5d60070020 2024-09-13T18:53:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5ffff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d60070480 con 0x7f5d60160ff0 2024-09-13T18:53:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5ffff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d600b3860 con 0x7f5d6006cdc0 2024-09-13T18:53:52.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5e7fc700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f5d60070020 0x7f5d6006c6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:57224/0 (socket says 172.21.15.7:57224) 2024-09-13T18:53:52.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5f7fe700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5d6006cdc0 0x7f5d60164b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5e7fc700 1 -- 172.21.15.7:0/1553007907 learned_addr learned my addr 172.21.15.7:0/1553007907 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:52.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5effd700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f5d60160ff0 0x7f5d600ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5dffb700 1 -- 172.21.15.7:0/1553007907 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420725362 0 0) 0x7f5d600b3b40 con 0x7f5d60070020 2024-09-13T18:53:52.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5dffb700 1 -- 172.21.15.7:0/1553007907 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d4c0035d0 con 0x7f5d60070020 2024-09-13T18:53:52.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5f7fe700 1 -- 172.21.15.7:0/1553007907 >> v2:172.21.15.171:3300/0 conn(0x7f5d60160ff0 msgr2=0x7f5d600ada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:52.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5f7fe700 1 --2- 172.21.15.7:0/1553007907 >> v2:172.21.15.171:3300/0 conn(0x7f5d60160ff0 0x7f5d600ada40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5f7fe700 1 -- 172.21.15.7:0/1553007907 >> v1:172.21.15.171:6789/0 conn(0x7f5d60070020 legacy=0x7f5d6006c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5f7fe700 1 -- 172.21.15.7:0/1553007907 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d60071a20 con 0x7f5d6006cdc0 2024-09-13T18:53:52.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5effd700 1 --2- 172.21.15.7:0/1553007907 >> v2:172.21.15.171:3300/0 conn(0x7f5d60160ff0 0x7f5d600ada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:52.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.591+0000 7f5d5f7fe700 1 --2- 172.21.15.7:0/1553007907 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5d6006cdc0 0x7f5d60164b20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009d90 tx=0x7f5d5800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1889ac4b8b20d98 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:52.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.595+0000 7f5d5dffb700 1 -- 172.21.15.7:0/1553007907 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5800e070 con 0x7f5d6006cdc0 2024-09-13T18:53:52.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.595+0000 7f5d5dffb700 1 -- 172.21.15.7:0/1553007907 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d5800bb20 con 0x7f5d6006cdc0 2024-09-13T18:53:52.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.595+0000 7f5d5dffb700 1 -- 172.21.15.7:0/1553007907 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5800be90 con 0x7f5d6006cdc0 2024-09-13T18:53:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.595+0000 7f5d5ffff700 1 -- 172.21.15.7:0/1553007907 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5d6006cdc0 msgr2=0x7f5d60164b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.595+0000 7f5d5ffff700 1 --2- 172.21.15.7:0/1553007907 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5d6006cdc0 0x7f5d60164b20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009d90 tx=0x7f5d5800b330 comp rx=0 tx=0).stop 2024-09-13T18:53:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.595+0000 7f5d5ffff700 1 -- 172.21.15.7:0/1553007907 shutdown_connections 2024-09-13T18:53:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.595+0000 7f5d5ffff700 1 --2- 172.21.15.7:0/1553007907 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5d6006cdc0 0x7f5d60164b20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.595+0000 7f5d5ffff700 1 --2- 172.21.15.7:0/1553007907 >> v2:172.21.15.171:3300/0 conn(0x7f5d60160ff0 0x7f5d600ada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.595+0000 7f5d5ffff700 1 -- 172.21.15.7:0/1553007907 >> 172.21.15.7:0/1553007907 conn(0x7f5d600b0570 msgr2=0x7f5d600b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.595+0000 7f5d5ffff700 1 -- 172.21.15.7:0/1553007907 shutdown_connections 2024-09-13T18:53:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.595+0000 7f5d5ffff700 1 -- 172.21.15.7:0/1553007907 wait complete. 2024-09-13T18:53:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.599+0000 7f5d5ffff700 1 Processor -- start 2024-09-13T18:53:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.599+0000 7f5d5ffff700 1 -- start start 2024-09-13T18:53:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.599+0000 7f5d5ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6006cdc0 0x7f5d600a87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.599+0000 7f5d5ffff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d60070020 0x7f5d6015eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.599+0000 7f5d5ffff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5d60160ff0 0x7f5d6014ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.599+0000 7f5d5ffff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d600a8d10 con 0x7f5d6006cdc0 2024-09-13T18:53:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.599+0000 7f5d5ffff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6015f090 con 0x7f5d60070020 2024-09-13T18:53:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.599+0000 7f5d5ffff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6014b040 con 0x7f5d60160ff0 2024-09-13T18:53:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.599+0000 7f5d5effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6006cdc0 0x7f5d600a87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.599+0000 7f5d5effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6006cdc0 0x7f5d600a87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33182/0 (socket says 172.21.15.7:33182) 2024-09-13T18:53:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.599+0000 7f5d5effd700 1 -- 172.21.15.7:0/3068665128 learned_addr learned my addr 172.21.15.7:0/3068665128 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.599+0000 7f5d5e7fc700 1 --2- 172.21.15.7:0/3068665128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d60070020 0x7f5d6015eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.599+0000 7f5d5f7fe700 1 --2- 172.21.15.7:0/3068665128 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5d60160ff0 0x7f5d6014ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.599+0000 7f5d5effd700 1 -- 172.21.15.7:0/3068665128 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5d60160ff0 msgr2=0x7f5d6014ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.599+0000 7f5d5effd700 1 --2- 172.21.15.7:0/3068665128 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5d60160ff0 0x7f5d6014ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.599+0000 7f5d5effd700 1 -- 172.21.15.7:0/3068665128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d60070020 msgr2=0x7f5d6015eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.599+0000 7f5d5effd700 1 --2- 172.21.15.7:0/3068665128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d60070020 0x7f5d6015eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.599+0000 7f5d5effd700 1 -- 172.21.15.7:0/3068665128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d480094b0 con 0x7f5d6006cdc0 2024-09-13T18:53:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.603+0000 7f5d5effd700 1 --2- 172.21.15.7:0/3068665128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6006cdc0 0x7f5d600a87d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800ebe0 tx=0x7f5d4800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.603+0000 7f5d577fe700 1 -- 172.21.15.7:0/3068665128 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48009770 con 0x7f5d6006cdc0 2024-09-13T18:53:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.603+0000 7f5d577fe700 1 -- 172.21.15.7:0/3068665128 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4800c850 con 0x7f5d6006cdc0 2024-09-13T18:53:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.603+0000 7f5d5ffff700 1 -- 172.21.15.7:0/3068665128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d58009a70 con 0x7f5d6006cdc0 2024-09-13T18:53:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.603+0000 7f5d577fe700 1 -- 172.21.15.7:0/3068665128 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4800cbe0 con 0x7f5d6006cdc0 2024-09-13T18:53:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.603+0000 7f5d5ffff700 1 -- 172.21.15.7:0/3068665128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6015f880 con 0x7f5d6006cdc0 2024-09-13T18:53:52.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.603+0000 7f5d5ffff700 1 -- 172.21.15.7:0/3068665128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d6006f760 con 0x7f5d6006cdc0 2024-09-13T18:53:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.603+0000 7f5d577fe700 1 -- 172.21.15.7:0/3068665128 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5d4800cd40 con 0x7f5d6006cdc0 2024-09-13T18:53:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.607+0000 7f5d577fe700 1 --2- 172.21.15.7:0/3068665128 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5d40066760 0x7f5d40068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.607+0000 7f5d577fe700 1 -- 172.21.15.7:0/3068665128 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5d48084110 con 0x7f5d6006cdc0 2024-09-13T18:53:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.607+0000 7f5d5e7fc700 1 --2- 172.21.15.7:0/3068665128 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5d40066760 0x7f5d40068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.607+0000 7f5d5e7fc700 1 --2- 172.21.15.7:0/3068665128 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5d40066760 0x7f5d40068c20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5d6015da50 tx=0x7f5d50009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.611+0000 7f5d577fe700 1 -- 172.21.15.7:0/3068665128 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5d48051490 con 0x7f5d6006cdc0 2024-09-13T18:53:52.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.783+0000 7f5d5ffff700 1 -- 172.21.15.7:0/3068665128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d60160b30 con 0x7f5d6006cdc0 2024-09-13T18:53:52.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.783+0000 7f5d577fe700 1 -- 172.21.15.7:0/3068665128 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5d48054ab0 con 0x7f5d6006cdc0 2024-09-13T18:53:52.793 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:53:52.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.787+0000 7f5d5ffff700 1 -- 172.21.15.7:0/3068665128 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5d40066760 msgr2=0x7f5d40068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.787+0000 7f5d5ffff700 1 --2- 172.21.15.7:0/3068665128 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5d40066760 0x7f5d40068c20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5d6015da50 tx=0x7f5d50009040 comp rx=0 tx=0).stop 2024-09-13T18:53:52.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.787+0000 7f5d5ffff700 1 -- 172.21.15.7:0/3068665128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6006cdc0 msgr2=0x7f5d600a87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.787+0000 7f5d5ffff700 1 --2- 172.21.15.7:0/3068665128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6006cdc0 0x7f5d600a87d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800ebe0 tx=0x7f5d4800eef0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.787+0000 7f5d5ffff700 1 -- 172.21.15.7:0/3068665128 shutdown_connections 2024-09-13T18:53:52.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.787+0000 7f5d5ffff700 1 --2- 172.21.15.7:0/3068665128 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5d40066760 0x7f5d40068c20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.787+0000 7f5d5ffff700 1 --2- 172.21.15.7:0/3068665128 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5d60160ff0 0x7f5d6014ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.787+0000 7f5d5ffff700 1 --2- 172.21.15.7:0/3068665128 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5d60070020 0x7f5d6015eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.787+0000 7f5d5ffff700 1 --2- 172.21.15.7:0/3068665128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d6006cdc0 0x7f5d600a87d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.787+0000 7f5d5ffff700 1 -- 172.21.15.7:0/3068665128 >> 172.21.15.7:0/3068665128 conn(0x7f5d600b0570 msgr2=0x7f5d600af890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.787+0000 7f5d5ffff700 1 -- 172.21.15.7:0/3068665128 shutdown_connections 2024-09-13T18:53:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.787+0000 7f5d5ffff700 1 -- 172.21.15.7:0/3068665128 wait complete. 2024-09-13T18:53:52.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:53:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.931+0000 7f95549fe700 1 Processor -- start 2024-09-13T18:53:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.931+0000 7f95549fe700 1 -- start start 2024-09-13T18:53:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.931+0000 7f95549fe700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f95500d17a0 0x7f95500d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.931+0000 7f95549fe700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95500d1ca0 con 0x7f95501606a0 2024-09-13T18:53:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.931+0000 7f95549fe700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95500cdd00 con 0x7f9550160b10 2024-09-13T18:53:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.931+0000 7f95549fe700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95500cdfe0 con 0x7f95500d17a0 2024-09-13T18:53:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f954ed9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f95501606a0 0x7f95500ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45506/0 (socket says 172.21.15.7:45506) 2024-09-13T18:53:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f954ed9d700 1 -- 172.21.15.7:0/2967208856 learned_addr learned my addr 172.21.15.7:0/2967208856 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f954e59c700 1 --2- 172.21.15.7:0/2967208856 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f95500d17a0 0x7f95500d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f954dd9b700 1 -- 172.21.15.7:0/2967208856 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923587109 0 0) 0x7f95500d1ca0 con 0x7f95501606a0 2024-09-13T18:53:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f954dd9b700 1 -- 172.21.15.7:0/2967208856 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95380035d0 con 0x7f95501606a0 2024-09-13T18:53:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f954dd9b700 1 -- 172.21.15.7:0/2967208856 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147764795 0 0) 0x7f95500cdd00 con 0x7f9550160b10 2024-09-13T18:53:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f954dd9b700 1 -- 172.21.15.7:0/2967208856 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95500d1ca0 con 0x7f9550160b10 2024-09-13T18:53:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f954e59c700 1 -- 172.21.15.7:0/2967208856 >> v1:172.21.15.171:6789/0 conn(0x7f9550160b10 legacy=0x7f95500b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f954e59c700 1 -- 172.21.15.7:0/2967208856 >> v1:172.21.15.7:6789/0 conn(0x7f95501606a0 legacy=0x7f95500ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f954e59c700 1 -- 172.21.15.7:0/2967208856 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95500ce2c0 con 0x7f95500d17a0 2024-09-13T18:53:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f954e59c700 1 --2- 172.21.15.7:0/2967208856 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f95500d17a0 0x7f95500d3800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9540009880 tx=0x7f9540009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20cf593a16991db8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f954dd9b700 1 -- 172.21.15.7:0/2967208856 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540011830 con 0x7f95500d17a0 2024-09-13T18:53:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f954dd9b700 1 -- 172.21.15.7:0/2967208856 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9540011990 con 0x7f95500d17a0 2024-09-13T18:53:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f954dd9b700 1 -- 172.21.15.7:0/2967208856 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540011d00 con 0x7f95500d17a0 2024-09-13T18:53:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f95549fe700 1 -- 172.21.15.7:0/2967208856 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f95500d17a0 msgr2=0x7f95500d3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f95549fe700 1 --2- 172.21.15.7:0/2967208856 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f95500d17a0 0x7f95500d3800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9540009880 tx=0x7f9540009b90 comp rx=0 tx=0).stop 2024-09-13T18:53:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f95549fe700 1 -- 172.21.15.7:0/2967208856 shutdown_connections 2024-09-13T18:53:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f95549fe700 1 --2- 172.21.15.7:0/2967208856 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f95500d17a0 0x7f95500d3800 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f95549fe700 1 -- 172.21.15.7:0/2967208856 >> 172.21.15.7:0/2967208856 conn(0x7f95500d2d10 msgr2=0x7f95500d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f95549fe700 1 -- 172.21.15.7:0/2967208856 shutdown_connections 2024-09-13T18:53:52.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.935+0000 7f95549fe700 1 -- 172.21.15.7:0/2967208856 wait complete. 2024-09-13T18:53:52.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f95549fe700 1 Processor -- start 2024-09-13T18:53:52.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f95549fe700 1 -- start start 2024-09-13T18:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f95549fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95500d17a0 0x7f95500cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f95549fe700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f95501606a0 0x7f95501f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f95549fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9550160b10 0x7f955014b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f95549fe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955014be70 con 0x7f95500d17a0 2024-09-13T18:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f95549fe700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95500c6ea0 con 0x7f9550160b10 2024-09-13T18:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f95549fe700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95500c82c0 con 0x7f95501606a0 2024-09-13T18:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f954ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95500d17a0 0x7f95500cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f954ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95500d17a0 0x7f95500cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33190/0 (socket says 172.21.15.7:33190) 2024-09-13T18:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f954ed9d700 1 -- 172.21.15.7:0/2987192673 learned_addr learned my addr 172.21.15.7:0/2987192673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f954f59e700 1 --2- 172.21.15.7:0/2987192673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9550160b10 0x7f955014b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f954e59c700 1 --2- 172.21.15.7:0/2987192673 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f95501606a0 0x7f95501f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f954ed9d700 1 -- 172.21.15.7:0/2987192673 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f95501606a0 msgr2=0x7f95501f21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f954ed9d700 1 --2- 172.21.15.7:0/2987192673 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f95501606a0 0x7f95501f21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f954ed9d700 1 -- 172.21.15.7:0/2987192673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9550160b10 msgr2=0x7f955014b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f954ed9d700 1 --2- 172.21.15.7:0/2987192673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9550160b10 0x7f955014b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f954ed9d700 1 -- 172.21.15.7:0/2987192673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95400094b0 con 0x7f95500d17a0 2024-09-13T18:53:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f954ed9d700 1 --2- 172.21.15.7:0/2987192673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95500d17a0 0x7f95500cca40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f954400c520 tx=0x7f954400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f95377fe700 1 -- 172.21.15.7:0/2987192673 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544005d90 con 0x7f95500d17a0 2024-09-13T18:53:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f95377fe700 1 -- 172.21.15.7:0/2987192673 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95440070d0 con 0x7f95500d17a0 2024-09-13T18:53:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f95377fe700 1 -- 172.21.15.7:0/2987192673 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9544007480 con 0x7f95500d17a0 2024-09-13T18:53:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f95549fe700 1 -- 172.21.15.7:0/2987192673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95500c68e0 con 0x7f95500d17a0 2024-09-13T18:53:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.939+0000 7f95549fe700 1 -- 172.21.15.7:0/2987192673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95500c8560 con 0x7f95500d17a0 2024-09-13T18:53:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.943+0000 7f95549fe700 1 -- 172.21.15.7:0/2987192673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955006b140 con 0x7f95500d17a0 2024-09-13T18:53:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.943+0000 7f95377fe700 1 -- 172.21.15.7:0/2987192673 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f95440075e0 con 0x7f95500d17a0 2024-09-13T18:53:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.943+0000 7f95377fe700 1 --2- 172.21.15.7:0/2987192673 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f953c066730 0x7f953c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.943+0000 7f95377fe700 1 -- 172.21.15.7:0/2987192673 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9544083690 con 0x7f95500d17a0 2024-09-13T18:53:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.943+0000 7f954e59c700 1 --2- 172.21.15.7:0/2987192673 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f953c066730 0x7f953c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:52.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.947+0000 7f954e59c700 1 --2- 172.21.15.7:0/2987192673 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f953c066730 0x7f953c068bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f95400090b0 tx=0x7f9540010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:52.951+0000 7f95377fe700 1 -- 172.21.15.7:0/2987192673 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9544050880 con 0x7f95500d17a0 2024-09-13T18:53:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.127+0000 7f95549fe700 1 -- 172.21.15.7:0/2987192673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f95500cdd00 con 0x7f95500d17a0 2024-09-13T18:53:53.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.127+0000 7f95377fe700 1 -- 172.21.15.7:0/2987192673 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9544053ea0 con 0x7f95500d17a0 2024-09-13T18:53:53.135 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.127+0000 7f95549fe700 1 -- 172.21.15.7:0/2987192673 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f953c066730 msgr2=0x7f953c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.127+0000 7f95549fe700 1 --2- 172.21.15.7:0/2987192673 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f953c066730 0x7f953c068bf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f95400090b0 tx=0x7f9540010000 comp rx=0 tx=0).stop 2024-09-13T18:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.127+0000 7f95549fe700 1 -- 172.21.15.7:0/2987192673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95500d17a0 msgr2=0x7f95500cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.127+0000 7f95549fe700 1 --2- 172.21.15.7:0/2987192673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95500d17a0 0x7f95500cca40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f954400c520 tx=0x7f954400c830 comp rx=0 tx=0).stop 2024-09-13T18:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.131+0000 7f95549fe700 1 -- 172.21.15.7:0/2987192673 shutdown_connections 2024-09-13T18:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.131+0000 7f95549fe700 1 --2- 172.21.15.7:0/2987192673 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f953c066730 0x7f953c068bf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.131+0000 7f95549fe700 1 --2- 172.21.15.7:0/2987192673 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9550160b10 0x7f955014b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.131+0000 7f95549fe700 1 --2- 172.21.15.7:0/2987192673 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f95501606a0 0x7f95501f21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.131+0000 7f95549fe700 1 --2- 172.21.15.7:0/2987192673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95500d17a0 0x7f95500cca40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.131+0000 7f95549fe700 1 -- 172.21.15.7:0/2987192673 >> 172.21.15.7:0/2987192673 conn(0x7f95500d2d10 msgr2=0x7f95500ce6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:53.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.131+0000 7f95549fe700 1 -- 172.21.15.7:0/2987192673 shutdown_connections 2024-09-13T18:53:53.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.131+0000 7f95549fe700 1 -- 172.21.15.7:0/2987192673 wait complete. 2024-09-13T18:53:53.155 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:53:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.255+0000 7fcac7fff700 1 Processor -- start 2024-09-13T18:53:53.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.255+0000 7fcac7fff700 1 -- start start 2024-09-13T18:53:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac7fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcac80718d0 0x7fcac80cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac7fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac815c020 con 0x7fcac815bbe0 2024-09-13T18:53:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac7fff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac806fc80 con 0x7fcac80ad6b0 2024-09-13T18:53:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac7fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac806fed0 con 0x7fcac80718d0 2024-09-13T18:53:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcabe7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcac815bbe0 0x7fcac80ccbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45514/0 (socket says 172.21.15.7:45514) 2024-09-13T18:53:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcabe7fc700 1 -- 172.21.15.7:0/2139673242 learned_addr learned my addr 172.21.15.7:0/2139673242 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac77fe700 1 --2- 172.21.15.7:0/2139673242 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcac80718d0 0x7fcac80cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac67fc700 1 -- 172.21.15.7:0/2139673242 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1858145352 0 0) 0x7fcac815c020 con 0x7fcac815bbe0 2024-09-13T18:53:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac67fc700 1 -- 172.21.15.7:0/2139673242 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaa80035d0 con 0x7fcac815bbe0 2024-09-13T18:53:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac67fc700 1 -- 172.21.15.7:0/2139673242 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3962178864 0 0) 0x7fcaa80035d0 con 0x7fcac815bbe0 2024-09-13T18:53:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac67fc700 1 -- 172.21.15.7:0/2139673242 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcac80718d0 msgr2=0x7fcac80cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac67fc700 1 --2- 172.21.15.7:0/2139673242 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcac80718d0 0x7fcac80cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac67fc700 1 -- 172.21.15.7:0/2139673242 >> v1:172.21.15.171:6789/0 conn(0x7fcac80ad6b0 legacy=0x7fcac80ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac67fc700 1 -- 172.21.15.7:0/2139673242 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac80701b0 con 0x7fcac815bbe0 2024-09-13T18:53:53.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac67fc700 1 -- 172.21.15.7:0/2139673242 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7fcab0001670 con 0x7fcac815bbe0 2024-09-13T18:53:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac67fc700 1 -- 172.21.15.7:0/2139673242 >> v1:172.21.15.7:6789/0 conn(0x7fcac815bbe0 legacy=0x7fcac80ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac67fc700 1 --2- 172.21.15.7:0/2139673242 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcac80718d0 0x7fcaa8003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac67fc700 1 --2- 172.21.15.7:0/2139673242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcaa8003f90 0x7fcaa8004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac67fc700 1 --2- 172.21.15.7:0/2139673242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa8004db0 0x7fcaa8005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac67fc700 1 -- 172.21.15.7:0/2139673242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac806fed0 con 0x7fcaa8004db0 2024-09-13T18:53:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac67fc700 1 -- 172.21.15.7:0/2139673242 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa800ba90 con 0x7fcaa8003f90 2024-09-13T18:53:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac67fc700 1 -- 172.21.15.7:0/2139673242 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa800bc50 con 0x7fcac80718d0 2024-09-13T18:53:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac77fe700 1 --2- 172.21.15.7:0/2139673242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa8004db0 0x7fcaa8005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcabe7fc700 1 --2- 172.21.15.7:0/2139673242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcaa8003f90 0x7fcaa8004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac6ffd700 1 --2- 172.21.15.7:0/2139673242 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcac80718d0 0x7fcaa8003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac6ffd700 1 -- 172.21.15.7:0/2139673242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcaa8003f90 msgr2=0x7fcaa8004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac6ffd700 1 --2- 172.21.15.7:0/2139673242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcaa8003f90 0x7fcaa8004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac6ffd700 1 -- 172.21.15.7:0/2139673242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa8004db0 msgr2=0x7fcaa8005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:53.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac6ffd700 1 --2- 172.21.15.7:0/2139673242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcaa8004db0 0x7fcaa8005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac6ffd700 1 -- 172.21.15.7:0/2139673242 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa80035d0 con 0x7fcac80718d0 2024-09-13T18:53:53.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcabe7fc700 1 --2- 172.21.15.7:0/2139673242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcaa8003f90 0x7fcaa8004870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:53.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac6ffd700 1 --2- 172.21.15.7:0/2139673242 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcac80718d0 0x7fcaa8003a50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcab80018f0 tx=0x7fcaa80058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c92d37834a7d73b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:53.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac67fc700 1 -- 172.21.15.7:0/2139673242 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8001390 con 0x7fcac80718d0 2024-09-13T18:53:53.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac67fc700 1 -- 172.21.15.7:0/2139673242 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab8012620 con 0x7fcac80718d0 2024-09-13T18:53:53.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.259+0000 7fcac67fc700 1 -- 172.21.15.7:0/2139673242 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab80129d0 con 0x7fcac80718d0 2024-09-13T18:53:53.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcac7fff700 1 -- 172.21.15.7:0/2139673242 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcac80718d0 msgr2=0x7fcaa8003a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:53.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcac7fff700 1 --2- 172.21.15.7:0/2139673242 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcac80718d0 0x7fcaa8003a50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcab80018f0 tx=0x7fcaa80058a0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcac77fe700 1 -- 172.21.15.7:0/2139673242 reap_dead start 2024-09-13T18:53:53.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcac7fff700 1 -- 172.21.15.7:0/2139673242 shutdown_connections 2024-09-13T18:53:53.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcac7fff700 1 -- 172.21.15.7:0/2139673242 >> 172.21.15.7:0/2139673242 conn(0x7fcac80b0570 msgr2=0x7fcac80b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:53.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcac7fff700 1 -- 172.21.15.7:0/2139673242 shutdown_connections 2024-09-13T18:53:53.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcac7fff700 1 -- 172.21.15.7:0/2139673242 wait complete. 2024-09-13T18:53:53.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcac7fff700 1 Processor -- start 2024-09-13T18:53:53.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcac7fff700 1 -- start start 2024-09-13T18:53:53.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcac7fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcac80718d0 0x7fcac8144380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcac7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcac80af1c0 0x7fcac80af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcac7fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcac814e920 0x7fcac814eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcac7fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac814f2e0 con 0x7fcac80af1c0 2024-09-13T18:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcac7fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac81448c0 con 0x7fcac814e920 2024-09-13T18:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcac7fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac81535c0 con 0x7fcac80718d0 2024-09-13T18:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcabe7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcac80af1c0 0x7fcac80af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcabe7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcac80af1c0 0x7fcac80af640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33210/0 (socket says 172.21.15.7:33210) 2024-09-13T18:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcabe7fc700 1 -- 172.21.15.7:0/170556307 learned_addr learned my addr 172.21.15.7:0/170556307 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.263+0000 7fcac6ffd700 1 --2- 172.21.15.7:0/170556307 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcac80718d0 0x7fcac8144380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.267+0000 7fcac77fe700 1 --2- 172.21.15.7:0/170556307 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcac814e920 0x7fcac814eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:53.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.267+0000 7fcabe7fc700 1 -- 172.21.15.7:0/170556307 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcac80718d0 msgr2=0x7fcac8144380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.267+0000 7fcabe7fc700 1 --2- 172.21.15.7:0/170556307 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcac80718d0 0x7fcac8144380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.267+0000 7fcabe7fc700 1 -- 172.21.15.7:0/170556307 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcac814e920 msgr2=0x7fcac814eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.267+0000 7fcabe7fc700 1 --2- 172.21.15.7:0/170556307 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcac814e920 0x7fcac814eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.267+0000 7fcabe7fc700 1 -- 172.21.15.7:0/170556307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac8152d20 con 0x7fcac80af1c0 2024-09-13T18:53:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.267+0000 7fcabe7fc700 1 --2- 172.21.15.7:0/170556307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcac80af1c0 0x7fcac80af640 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcab0007300 tx=0x7fcab0007610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.267+0000 7fcac4ff9700 1 -- 172.21.15.7:0/170556307 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab000f7d0 con 0x7fcac80af1c0 2024-09-13T18:53:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.267+0000 7fcac4ff9700 1 -- 172.21.15.7:0/170556307 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab000f930 con 0x7fcac80af1c0 2024-09-13T18:53:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.267+0000 7fcac7fff700 1 -- 172.21.15.7:0/170556307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac8153000 con 0x7fcac80af1c0 2024-09-13T18:53:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.267+0000 7fcac4ff9700 1 -- 172.21.15.7:0/170556307 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab000fca0 con 0x7fcac80af1c0 2024-09-13T18:53:53.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.267+0000 7fcac7fff700 1 -- 172.21.15.7:0/170556307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac815c230 con 0x7fcac80af1c0 2024-09-13T18:53:53.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.267+0000 7fcac7fff700 1 -- 172.21.15.7:0/170556307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa0005310 con 0x7fcac80af1c0 2024-09-13T18:53:53.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.271+0000 7fcac4ff9700 1 -- 172.21.15.7:0/170556307 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcab001d460 con 0x7fcac80af1c0 2024-09-13T18:53:53.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.271+0000 7fcac4ff9700 1 --2- 172.21.15.7:0/170556307 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcaac066760 0x7fcaac068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:53.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.271+0000 7fcac4ff9700 1 -- 172.21.15.7:0/170556307 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcab0084d60 con 0x7fcac80af1c0 2024-09-13T18:53:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.271+0000 7fcac6ffd700 1 --2- 172.21.15.7:0/170556307 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcaac066760 0x7fcaac068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:53.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.271+0000 7fcac6ffd700 1 --2- 172.21.15.7:0/170556307 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcaac066760 0x7fcaac068c20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcab8001840 tx=0x7fcab8005040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:53.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.279+0000 7fcac4ff9700 1 -- 172.21.15.7:0/170556307 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcab0052050 con 0x7fcac80af1c0 2024-09-13T18:53:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.459+0000 7fcac7fff700 1 -- 172.21.15.7:0/170556307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcaa0005890 con 0x7fcac80af1c0 2024-09-13T18:53:53.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.463+0000 7fcac4ff9700 1 -- 172.21.15.7:0/170556307 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v40) v1 ==== 80+0+120 (secure 0 0 0) 0x7fcab0055670 con 0x7fcac80af1c0 2024-09-13T18:53:53.470 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6835/2843879273 2024-09-14T18:53:26.228249+0000 2024-09-13T18:53:53.470 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6834/2843879273 2024-09-14T18:53:26.228249+0000 2024-09-13T18:53:53.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.463+0000 7fcac7fff700 1 -- 172.21.15.7:0/170556307 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcaac066760 msgr2=0x7fcaac068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:53.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.463+0000 7fcac7fff700 1 --2- 172.21.15.7:0/170556307 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcaac066760 0x7fcaac068c20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcab8001840 tx=0x7fcab8005040 comp rx=0 tx=0).stop 2024-09-13T18:53:53.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.463+0000 7fcac7fff700 1 -- 172.21.15.7:0/170556307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcac80af1c0 msgr2=0x7fcac80af640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:53.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.463+0000 7fcac7fff700 1 --2- 172.21.15.7:0/170556307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcac80af1c0 0x7fcac80af640 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcab0007300 tx=0x7fcab0007610 comp rx=0 tx=0).stop 2024-09-13T18:53:53.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.463+0000 7fcac7fff700 1 -- 172.21.15.7:0/170556307 shutdown_connections 2024-09-13T18:53:53.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.463+0000 7fcac7fff700 1 --2- 172.21.15.7:0/170556307 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcaac066760 0x7fcaac068c20 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.463+0000 7fcac7fff700 1 --2- 172.21.15.7:0/170556307 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcac814e920 0x7fcac814eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.463+0000 7fcac7fff700 1 --2- 172.21.15.7:0/170556307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcac80af1c0 0x7fcac80af640 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.463+0000 7fcac7fff700 1 --2- 172.21.15.7:0/170556307 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcac80718d0 0x7fcac8144380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.463+0000 7fcac7fff700 1 -- 172.21.15.7:0/170556307 >> 172.21.15.7:0/170556307 conn(0x7fcac80b0570 msgr2=0x7fcac80cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:53.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.463+0000 7fcac7fff700 1 -- 172.21.15.7:0/170556307 shutdown_connections 2024-09-13T18:53:53.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.463+0000 7fcac7fff700 1 -- 172.21.15.7:0/170556307 wait complete. 2024-09-13T18:53:53.474 INFO:teuthology.orchestra.run.smithi007.stderr:listed 2 entries 2024-09-13T18:53:53.485 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2024-09-13T18:53:53.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:53.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T18:53:53.495 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[35609]: fuse finished with error 0 and tester_r 0 2024-09-13T18:53:53.499 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-13T18:53:53.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:53.499 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T18:53:53.623 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:53:53.623 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:53.633 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346336k,nr_inodes=4086584,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18533 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T18:53:53.634 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T18:53:53.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:53.634 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-13T18:53:53.686 INFO:teuthology.orchestra.run.smithi171.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-09-13T18:53:53.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:53:53.688 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:53:53.688 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:53.696 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16349432k,nr_inodes=4087358,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276116k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276112k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276116k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T18:53:53.696 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-13T18:53:53.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:53.696 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-13T18:53:53.744 INFO:teuthology.orchestra.run.smithi007.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-13T18:53:53.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:53:53.745 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:53:53.745 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:53:53.752 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346336k,nr_inodes=4086584,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18533 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T18:53:53.752 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T18:53:53.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:53:53.752 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-13T18:53:53.759 INFO:teuthology.orchestra.run.smithi171.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-13T18:53:53.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:53:53.760 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:53.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.875+0000 7fba44e7a700 1 Processor -- start 2024-09-13T18:53:53.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.875+0000 7fba44e7a700 1 -- start start 2024-09-13T18:53:53.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.875+0000 7fba44e7a700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba4006fee0 0x7fba400702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:53.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.875+0000 7fba44e7a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fba4006cef0 0x7fba40070800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:53.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.875+0000 7fba44e7a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba400b1870 con 0x7fba400adae0 2024-09-13T18:53:53.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.875+0000 7fba44e7a700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba4006d390 con 0x7fba4006cef0 2024-09-13T18:53:53.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.875+0000 7fba44e7a700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba4006d670 con 0x7fba4006fee0 2024-09-13T18:53:53.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.875+0000 7fba3effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fba4006cef0 0x7fba40070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:53.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.875+0000 7fba3ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fba400adae0 0x7fba4015cc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45528/0 (socket says 172.21.15.7:45528) 2024-09-13T18:53:53.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.875+0000 7fba3ffff700 1 -- 172.21.15.7:0/1509319175 learned_addr learned my addr 172.21.15.7:0/1509319175 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:53.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.875+0000 7fba3f7fe700 1 --2- 172.21.15.7:0/1509319175 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba4006fee0 0x7fba400702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba3effd700 1 -- 172.21.15.7:0/1509319175 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba4006fee0 msgr2=0x7fba400702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba3effd700 1 --2- 172.21.15.7:0/1509319175 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba4006fee0 0x7fba400702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba3effd700 1 -- 172.21.15.7:0/1509319175 >> v1:172.21.15.7:6789/0 conn(0x7fba400adae0 legacy=0x7fba4015cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba3effd700 1 -- 172.21.15.7:0/1509319175 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba400b1b10 con 0x7fba4006cef0 2024-09-13T18:53:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba3e7fc700 1 -- 172.21.15.7:0/1509319175 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245953206 0 0) 0x7fba400b1870 con 0x7fba400adae0 2024-09-13T18:53:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba3effd700 1 --2- 172.21.15.7:0/1509319175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba4006cef0 0x7fba40070800 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fba30009dc0 tx=0x7fba3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c05f51d2627516d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba3e7fc700 1 -- 172.21.15.7:0/1509319175 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3000e070 con 0x7fba4006cef0 2024-09-13T18:53:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba3e7fc700 1 -- 172.21.15.7:0/1509319175 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3000bae0 con 0x7fba4006cef0 2024-09-13T18:53:53.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba3e7fc700 1 -- 172.21.15.7:0/1509319175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3000be90 con 0x7fba4006cef0 2024-09-13T18:53:53.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba44e7a700 1 -- 172.21.15.7:0/1509319175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba4006cef0 msgr2=0x7fba40070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:53.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba44e7a700 1 --2- 172.21.15.7:0/1509319175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba4006cef0 0x7fba40070800 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fba30009dc0 tx=0x7fba3000b330 comp rx=0 tx=0).stop 2024-09-13T18:53:53.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba44e7a700 1 -- 172.21.15.7:0/1509319175 shutdown_connections 2024-09-13T18:53:53.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba44e7a700 1 --2- 172.21.15.7:0/1509319175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba4006cef0 0x7fba40070800 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba44e7a700 1 --2- 172.21.15.7:0/1509319175 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba4006fee0 0x7fba400702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba44e7a700 1 -- 172.21.15.7:0/1509319175 >> 172.21.15.7:0/1509319175 conn(0x7fba400b0570 msgr2=0x7fba400b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:53.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba44e7a700 1 -- 172.21.15.7:0/1509319175 shutdown_connections 2024-09-13T18:53:53.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba44e7a700 1 -- 172.21.15.7:0/1509319175 wait complete. 2024-09-13T18:53:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba44e7a700 1 Processor -- start 2024-09-13T18:53:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.879+0000 7fba44e7a700 1 -- start start 2024-09-13T18:53:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba44e7a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba4006cef0 0x7fba40145620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba44e7a700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba4006fee0 0x7fba400cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba44e7a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba400adae0 0x7fba400cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba44e7a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba40154c70 con 0x7fba400adae0 2024-09-13T18:53:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba44e7a700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba40145b60 con 0x7fba4006cef0 2024-09-13T18:53:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba44e7a700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba4015c1e0 con 0x7fba4006fee0 2024-09-13T18:53:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba3ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba400adae0 0x7fba400cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:53.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba3ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba400adae0 0x7fba400cce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33234/0 (socket says 172.21.15.7:33234) 2024-09-13T18:53:53.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba3ffff700 1 -- 172.21.15.7:0/497742666 learned_addr learned my addr 172.21.15.7:0/497742666 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:53.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba3effd700 1 --2- 172.21.15.7:0/497742666 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba4006fee0 0x7fba400cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:53.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba3f7fe700 1 --2- 172.21.15.7:0/497742666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba4006cef0 0x7fba40145620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba3ffff700 1 -- 172.21.15.7:0/497742666 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba4006fee0 msgr2=0x7fba400cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba3ffff700 1 --2- 172.21.15.7:0/497742666 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba4006fee0 0x7fba400cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba3ffff700 1 -- 172.21.15.7:0/497742666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba4006cef0 msgr2=0x7fba40145620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba3ffff700 1 --2- 172.21.15.7:0/497742666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba4006cef0 0x7fba40145620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba3ffff700 1 -- 172.21.15.7:0/497742666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba30009a70 con 0x7fba400adae0 2024-09-13T18:53:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba3ffff700 1 --2- 172.21.15.7:0/497742666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba400adae0 0x7fba400cce80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fba3800c4d0 tx=0x7fba3800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba37fff700 1 -- 172.21.15.7:0/497742666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba38004ca0 con 0x7fba400adae0 2024-09-13T18:53:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba37fff700 1 -- 172.21.15.7:0/497742666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba38004e20 con 0x7fba400adae0 2024-09-13T18:53:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba44e7a700 1 -- 172.21.15.7:0/497742666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba4015c4c0 con 0x7fba400adae0 2024-09-13T18:53:53.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba37fff700 1 -- 172.21.15.7:0/497742666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba38007310 con 0x7fba400adae0 2024-09-13T18:53:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.883+0000 7fba44e7a700 1 -- 172.21.15.7:0/497742666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba40154ec0 con 0x7fba400adae0 2024-09-13T18:53:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.887+0000 7fba44e7a700 1 -- 172.21.15.7:0/497742666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba400711a0 con 0x7fba400adae0 2024-09-13T18:53:53.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.887+0000 7fba37fff700 1 -- 172.21.15.7:0/497742666 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fba38007470 con 0x7fba400adae0 2024-09-13T18:53:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.887+0000 7fba37fff700 1 --2- 172.21.15.7:0/497742666 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fba20066730 0x7fba20068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.887+0000 7fba37fff700 1 -- 172.21.15.7:0/497742666 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fba38083d70 con 0x7fba400adae0 2024-09-13T18:53:53.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.887+0000 7fba3f7fe700 1 --2- 172.21.15.7:0/497742666 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fba20066730 0x7fba20068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:53.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.887+0000 7fba3f7fe700 1 --2- 172.21.15.7:0/497742666 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fba20066730 0x7fba20068bf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fba28009b50 tx=0x7fba28009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:53.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:53.895+0000 7fba37fff700 1 -- 172.21.15.7:0/497742666 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fba38051090 con 0x7fba400adae0 2024-09-13T18:53:54.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.099+0000 7fba44e7a700 1 -- 172.21.15.7:0/497742666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba4006bca0 con 0x7fba400adae0 2024-09-13T18:53:54.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.099+0000 7fba37fff700 1 -- 172.21.15.7:0/497742666 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4732 (secure 0 0 0) 0x7fba38007720 con 0x7fba400adae0 2024-09-13T18:53:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:54.107 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2458672850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2458672850},{"type":"v1","addr":"172.21.15.171:6835","nonce":2458672850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3381042298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3381042298},{"type":"v1","addr":"172.21.15.171:6837","nonce":3381042298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2068823578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2068823578},{"type":"v1","addr":"172.21.15.7:6833","nonce":2068823578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:53:17.897317+0000","modified":"2024-09-13T18:53:30.253515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5463":{"gid":5463,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/127797392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":127797392},{"type":"v1","addr":"172.21.15.7:6835","nonce":127797392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T18:53:54.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.099+0000 7fba44e7a700 1 -- 172.21.15.7:0/497742666 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fba20066730 msgr2=0x7fba20068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:54.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.099+0000 7fba44e7a700 1 --2- 172.21.15.7:0/497742666 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fba20066730 0x7fba20068bf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fba28009b50 tx=0x7fba28009240 comp rx=0 tx=0).stop 2024-09-13T18:53:54.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.099+0000 7fba44e7a700 1 -- 172.21.15.7:0/497742666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba400adae0 msgr2=0x7fba400cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:54.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.099+0000 7fba44e7a700 1 --2- 172.21.15.7:0/497742666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba400adae0 0x7fba400cce80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fba3800c4d0 tx=0x7fba3800c7e0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.099+0000 7fba44e7a700 1 -- 172.21.15.7:0/497742666 shutdown_connections 2024-09-13T18:53:54.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.099+0000 7fba44e7a700 1 --2- 172.21.15.7:0/497742666 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fba20066730 0x7fba20068bf0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.099+0000 7fba44e7a700 1 --2- 172.21.15.7:0/497742666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba400adae0 0x7fba400cce80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.099+0000 7fba44e7a700 1 --2- 172.21.15.7:0/497742666 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fba4006fee0 0x7fba400cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.099+0000 7fba44e7a700 1 --2- 172.21.15.7:0/497742666 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba4006cef0 0x7fba40145620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.099+0000 7fba44e7a700 1 -- 172.21.15.7:0/497742666 >> 172.21.15.7:0/497742666 conn(0x7fba400b0570 msgr2=0x7fba400cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:54.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.099+0000 7fba44e7a700 1 -- 172.21.15.7:0/497742666 shutdown_connections 2024-09-13T18:53:54.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.103+0000 7fba44e7a700 1 -- 172.21.15.7:0/497742666 wait complete. 2024-09-13T18:53:54.111 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-09-13T18:53:54.123 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:53:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9759e700 1 Processor -- start 2024-09-13T18:53:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9759e700 1 -- start start 2024-09-13T18:53:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9759e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcc98160ff0 0x7fcc980ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9759e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcc98070020 0x7fcc9806c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9759e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fcc9806cbf0 0x7fcc9806d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9759e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc98070480 con 0x7fcc98070020 2024-09-13T18:53:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9759e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc980b3b40 con 0x7fcc9806cbf0 2024-09-13T18:53:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9759e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc980b3860 con 0x7fcc98160ff0 2024-09-13T18:53:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc95d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcc98070020 0x7fcc9806c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc95d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcc98070020 0x7fcc9806c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33248/0 (socket says 172.21.15.7:33248) 2024-09-13T18:53:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc95d9b700 1 -- 172.21.15.7:0/3341317719 learned_addr learned my addr 172.21.15.7:0/3341317719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9659c700 1 --2- 172.21.15.7:0/3341317719 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcc98160ff0 0x7fcc980ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc96d9d700 1 --2- 172.21.15.7:0/3341317719 >> v2:172.21.15.171:3300/0 conn(0x7fcc9806cbf0 0x7fcc9806d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc95d9b700 1 -- 172.21.15.7:0/3341317719 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcc98160ff0 msgr2=0x7fcc980ada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc95d9b700 1 --2- 172.21.15.7:0/3341317719 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcc98160ff0 0x7fcc980ada40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc95d9b700 1 -- 172.21.15.7:0/3341317719 >> v2:172.21.15.171:3300/0 conn(0x7fcc9806cbf0 msgr2=0x7fcc9806d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc95d9b700 1 --2- 172.21.15.7:0/3341317719 >> v2:172.21.15.171:3300/0 conn(0x7fcc9806cbf0 0x7fcc9806d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc95d9b700 1 -- 172.21.15.7:0/3341317719 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc98071a20 con 0x7fcc98070020 2024-09-13T18:53:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc95d9b700 1 --2- 172.21.15.7:0/3341317719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc98070020 0x7fcc9806c6b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcc9806b290 tx=0x7fcc9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5429f14bca9b00e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:54.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9559a700 1 -- 172.21.15.7:0/3341317719 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9000e070 con 0x7fcc98070020 2024-09-13T18:53:54.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9559a700 1 -- 172.21.15.7:0/3341317719 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9000b9c0 con 0x7fcc98070020 2024-09-13T18:53:54.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9559a700 1 -- 172.21.15.7:0/3341317719 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9000bd70 con 0x7fcc98070020 2024-09-13T18:53:54.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9759e700 1 -- 172.21.15.7:0/3341317719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc98070020 msgr2=0x7fcc9806c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:54.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9759e700 1 --2- 172.21.15.7:0/3341317719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc98070020 0x7fcc9806c6b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcc9806b290 tx=0x7fcc9000b330 comp rx=0 tx=0).stop 2024-09-13T18:53:54.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9759e700 1 -- 172.21.15.7:0/3341317719 shutdown_connections 2024-09-13T18:53:54.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9759e700 1 --2- 172.21.15.7:0/3341317719 >> v2:172.21.15.171:3300/0 conn(0x7fcc9806cbf0 0x7fcc9806d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9759e700 1 --2- 172.21.15.7:0/3341317719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc98070020 0x7fcc9806c6b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9759e700 1 --2- 172.21.15.7:0/3341317719 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcc98160ff0 0x7fcc980ada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9759e700 1 -- 172.21.15.7:0/3341317719 >> 172.21.15.7:0/3341317719 conn(0x7fcc980b0570 msgr2=0x7fcc980b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:54.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.227+0000 7fcc9759e700 1 -- 172.21.15.7:0/3341317719 shutdown_connections 2024-09-13T18:53:54.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.231+0000 7fcc9759e700 1 -- 172.21.15.7:0/3341317719 wait complete. 2024-09-13T18:53:54.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.231+0000 7fcc9759e700 1 Processor -- start 2024-09-13T18:53:54.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.231+0000 7fcc9759e700 1 -- start start 2024-09-13T18:53:54.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.231+0000 7fcc9759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc9806cbf0 0x7fcc98158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.231+0000 7fcc9759e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcc98070020 0x7fcc98156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.231+0000 7fcc9759e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc98160ff0 0x7fcc98156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.231+0000 7fcc9759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc98159e70 con 0x7fcc9806cbf0 2024-09-13T18:53:54.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc9759e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc981598b0 con 0x7fcc98160ff0 2024-09-13T18:53:54.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc9759e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc98159b90 con 0x7fcc98070020 2024-09-13T18:53:54.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc9659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc9806cbf0 0x7fcc98158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc9659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc9806cbf0 0x7fcc98158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33250/0 (socket says 172.21.15.7:33250) 2024-09-13T18:53:54.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc9659c700 1 -- 172.21.15.7:0/2908020424 learned_addr learned my addr 172.21.15.7:0/2908020424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:54.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc96d9d700 1 --2- 172.21.15.7:0/2908020424 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc98160ff0 0x7fcc98156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc95d9b700 1 --2- 172.21.15.7:0/2908020424 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcc98070020 0x7fcc98156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc9659c700 1 -- 172.21.15.7:0/2908020424 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcc98070020 msgr2=0x7fcc98156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:54.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc9659c700 1 --2- 172.21.15.7:0/2908020424 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcc98070020 0x7fcc98156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc9659c700 1 -- 172.21.15.7:0/2908020424 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc98160ff0 msgr2=0x7fcc98156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:54.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc9659c700 1 --2- 172.21.15.7:0/2908020424 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc98160ff0 0x7fcc98156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc9659c700 1 -- 172.21.15.7:0/2908020424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc90009b30 con 0x7fcc9806cbf0 2024-09-13T18:53:54.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc9659c700 1 --2- 172.21.15.7:0/2908020424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc9806cbf0 0x7fcc98158f40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000d800 tx=0x7fcc8000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:54.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc8effd700 1 -- 172.21.15.7:0/2908020424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc800125f0 con 0x7fcc9806cbf0 2024-09-13T18:53:54.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc8effd700 1 -- 172.21.15.7:0/2908020424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc80012750 con 0x7fcc9806cbf0 2024-09-13T18:53:54.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc9759e700 1 -- 172.21.15.7:0/2908020424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc98161f70 con 0x7fcc9806cbf0 2024-09-13T18:53:54.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc8effd700 1 -- 172.21.15.7:0/2908020424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80012ac0 con 0x7fcc9806cbf0 2024-09-13T18:53:54.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc9759e700 1 -- 172.21.15.7:0/2908020424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc98159660 con 0x7fcc9806cbf0 2024-09-13T18:53:54.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc9759e700 1 -- 172.21.15.7:0/2908020424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc98162400 con 0x7fcc9806cbf0 2024-09-13T18:53:54.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.235+0000 7fcc8effd700 1 -- 172.21.15.7:0/2908020424 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcc80012c20 con 0x7fcc9806cbf0 2024-09-13T18:53:54.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.239+0000 7fcc8effd700 1 --2- 172.21.15.7:0/2908020424 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcc78066730 0x7fcc78068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.239+0000 7fcc8effd700 1 -- 172.21.15.7:0/2908020424 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcc80084190 con 0x7fcc9806cbf0 2024-09-13T18:53:54.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.239+0000 7fcc95d9b700 1 --2- 172.21.15.7:0/2908020424 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcc78066730 0x7fcc78068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.239+0000 7fcc95d9b700 1 --2- 172.21.15.7:0/2908020424 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcc78066730 0x7fcc78068bf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcc9806b290 tx=0x7fcc90009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:54.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.243+0000 7fcc8effd700 1 -- 172.21.15.7:0/2908020424 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcc98162400 con 0x7fcc9806cbf0 2024-09-13T18:53:54.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.423+0000 7fcc9759e700 1 -- 172.21.15.7:0/2908020424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc98162400 con 0x7fcc9806cbf0 2024-09-13T18:53:54.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.423+0000 7fcc8effd700 1 -- 172.21.15.7:0/2908020424 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16648 (secure 0 0 0) 0x7fcc98162400 con 0x7fcc9806cbf0 2024-09-13T18:53:54.434 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:54.434 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:53:26.238445+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:53:15.862087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T18:53:16.885280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/2843879273":"2024-09-14T18:53:26.228249+0000","172.21.15.7:6834/2843879273":"2024-09-14T18:53:26.228249+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:53:54.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.427+0000 7fcc9759e700 1 -- 172.21.15.7:0/2908020424 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcc78066730 msgr2=0x7fcc78068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.427+0000 7fcc9759e700 1 --2- 172.21.15.7:0/2908020424 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcc78066730 0x7fcc78068bf0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fcc9806b290 tx=0x7fcc90009000 comp rx=0 tx=0).stop 2024-09-13T18:53:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.427+0000 7fcc9759e700 1 -- 172.21.15.7:0/2908020424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc9806cbf0 msgr2=0x7fcc98158f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.427+0000 7fcc9759e700 1 --2- 172.21.15.7:0/2908020424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc9806cbf0 0x7fcc98158f40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000d800 tx=0x7fcc8000db10 comp rx=0 tx=0).stop 2024-09-13T18:53:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.427+0000 7fcc9759e700 1 -- 172.21.15.7:0/2908020424 shutdown_connections 2024-09-13T18:53:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.427+0000 7fcc9759e700 1 --2- 172.21.15.7:0/2908020424 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcc78066730 0x7fcc78068bf0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.427+0000 7fcc9759e700 1 --2- 172.21.15.7:0/2908020424 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcc98160ff0 0x7fcc98156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.427+0000 7fcc9759e700 1 --2- 172.21.15.7:0/2908020424 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcc98070020 0x7fcc98156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.427+0000 7fcc9759e700 1 --2- 172.21.15.7:0/2908020424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc9806cbf0 0x7fcc98158f40 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.427+0000 7fcc9759e700 1 -- 172.21.15.7:0/2908020424 >> 172.21.15.7:0/2908020424 conn(0x7fcc980b0570 msgr2=0x7fcc9815f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.427+0000 7fcc9759e700 1 -- 172.21.15.7:0/2908020424 shutdown_connections 2024-09-13T18:53:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.427+0000 7fcc9759e700 1 -- 172.21.15.7:0/2908020424 wait complete. 2024-09-13T18:53:54.449 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:53:15.862087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:53:16.885280+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:53:54.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:53:54.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.559+0000 7fc5e0fa9700 1 Processor -- start 2024-09-13T18:53:54.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.559+0000 7fc5e0fa9700 1 -- start start 2024-09-13T18:53:54.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.559+0000 7fc5e0fa9700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fc5dc0b1560 0x7fc5dc161640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.559+0000 7fc5e0fa9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5dc0b1f20 con 0x7fc5dc0d3a20 2024-09-13T18:53:54.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.559+0000 7fc5e0fa9700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5dc0b2200 con 0x7fc5dc0d1120 2024-09-13T18:53:54.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.559+0000 7fc5e0fa9700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5dc0afa70 con 0x7fc5dc0b1560 2024-09-13T18:53:54.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.559+0000 7fc5d9d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc5dc0d3a20 0x7fc5dc160f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45542/0 (socket says 172.21.15.7:45542) 2024-09-13T18:53:54.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.559+0000 7fc5d9d9b700 1 -- 172.21.15.7:0/941496558 learned_addr learned my addr 172.21.15.7:0/941496558 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.559+0000 7fc5dad9d700 1 --2- 172.21.15.7:0/941496558 >> v2:172.21.15.171:3300/0 conn(0x7fc5dc0b1560 0x7fc5dc161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.559+0000 7fc5d959a700 1 -- 172.21.15.7:0/941496558 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873626155 0 0) 0x7fc5dc0b1f20 con 0x7fc5dc0d3a20 2024-09-13T18:53:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.559+0000 7fc5d959a700 1 -- 172.21.15.7:0/941496558 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5c40035d0 con 0x7fc5dc0d3a20 2024-09-13T18:53:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d959a700 1 -- 172.21.15.7:0/941496558 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1272337677 0 0) 0x7fc5c40035d0 con 0x7fc5dc0d3a20 2024-09-13T18:53:54.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d959a700 1 -- 172.21.15.7:0/941496558 >> v2:172.21.15.171:3300/0 conn(0x7fc5dc0b1560 msgr2=0x7fc5dc161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:54.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d959a700 1 --2- 172.21.15.7:0/941496558 >> v2:172.21.15.171:3300/0 conn(0x7fc5dc0b1560 0x7fc5dc161640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d959a700 1 -- 172.21.15.7:0/941496558 >> v1:172.21.15.171:6789/0 conn(0x7fc5dc0d1120 legacy=0x7fc5dc0d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:54.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d959a700 1 -- 172.21.15.7:0/941496558 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5dc0afd50 con 0x7fc5dc0d3a20 2024-09-13T18:53:54.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d959a700 1 -- 172.21.15.7:0/941496558 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7fc5cc001670 con 0x7fc5dc0d3a20 2024-09-13T18:53:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d959a700 1 -- 172.21.15.7:0/941496558 >> v1:172.21.15.7:6789/0 conn(0x7fc5dc0d3a20 legacy=0x7fc5dc160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d959a700 1 --2- 172.21.15.7:0/941496558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5c4003a90 0x7fc5c4003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d959a700 1 --2- 172.21.15.7:0/941496558 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5c4004870 0x7fc5c4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d959a700 1 --2- 172.21.15.7:0/941496558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5c4005230 0x7fc5c400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d959a700 1 -- 172.21.15.7:0/941496558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc0afa70 con 0x7fc5c4003a90 2024-09-13T18:53:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d959a700 1 -- 172.21.15.7:0/941496558 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c400c160 con 0x7fc5c4005230 2024-09-13T18:53:54.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d959a700 1 -- 172.21.15.7:0/941496558 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5c400c320 con 0x7fc5c4004870 2024-09-13T18:53:54.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5da59c700 1 --2- 172.21.15.7:0/941496558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5c4003a90 0x7fc5c4003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5dad9d700 1 --2- 172.21.15.7:0/941496558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5c4005230 0x7fc5c400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d9d9b700 1 --2- 172.21.15.7:0/941496558 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5c4004870 0x7fc5c4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d9d9b700 1 -- 172.21.15.7:0/941496558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5c4005230 msgr2=0x7fc5c400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:54.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d9d9b700 1 --2- 172.21.15.7:0/941496558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5c4005230 0x7fc5c400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d9d9b700 1 -- 172.21.15.7:0/941496558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5c4003a90 msgr2=0x7fc5c4003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:54.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d9d9b700 1 --2- 172.21.15.7:0/941496558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5c4003a90 0x7fc5c4003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d9d9b700 1 -- 172.21.15.7:0/941496558 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c40035d0 con 0x7fc5c4004870 2024-09-13T18:53:54.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5dad9d700 1 --2- 172.21.15.7:0/941496558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5c4005230 0x7fc5c400ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:54.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5dad9d700 1 -- 172.21.15.7:0/941496558 reap_dead start 2024-09-13T18:53:54.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d9d9b700 1 --2- 172.21.15.7:0/941496558 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5c4004870 0x7fc5c4004cf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00c2a0 tx=0x7fc5c40058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c2aa269f5719c5f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:54.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d959a700 1 -- 172.21.15.7:0/941496558 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc007c30 con 0x7fc5c4004870 2024-09-13T18:53:54.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d959a700 1 -- 172.21.15.7:0/941496558 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5cc00ce00 con 0x7fc5c4004870 2024-09-13T18:53:54.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5d959a700 1 -- 172.21.15.7:0/941496558 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc005bc0 con 0x7fc5c4004870 2024-09-13T18:53:54.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5e0fa9700 1 -- 172.21.15.7:0/941496558 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5c4004870 msgr2=0x7fc5c4004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:54.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5e0fa9700 1 --2- 172.21.15.7:0/941496558 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5c4004870 0x7fc5c4004cf0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00c2a0 tx=0x7fc5c40058a0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5e0fa9700 1 -- 172.21.15.7:0/941496558 shutdown_connections 2024-09-13T18:53:54.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5e0fa9700 1 --2- 172.21.15.7:0/941496558 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5c4004870 0x7fc5c4004cf0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5e0fa9700 1 -- 172.21.15.7:0/941496558 >> 172.21.15.7:0/941496558 conn(0x7fc5dc09c180 msgr2=0x7fc5dc0d2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:54.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5e0fa9700 1 -- 172.21.15.7:0/941496558 shutdown_connections 2024-09-13T18:53:54.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.563+0000 7fc5e0fa9700 1 -- 172.21.15.7:0/941496558 wait complete. 2024-09-13T18:53:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5e0fa9700 1 Processor -- start 2024-09-13T18:53:54.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5e0fa9700 1 -- start start 2024-09-13T18:53:54.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5e0fa9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5c4004870 0x7fc5dc154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5e0fa9700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5dc0b85b0 0x7fc5dc1f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5e0fa9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5dc0b89f0 0x7fc5dc1f6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5e0fa9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc06bc00 con 0x7fc5c4004870 2024-09-13T18:53:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5e0fa9700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc06bee0 con 0x7fc5dc0b89f0 2024-09-13T18:53:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5e0fa9700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc06c1c0 con 0x7fc5dc0b85b0 2024-09-13T18:53:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5da59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5c4004870 0x7fc5dc154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5da59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5c4004870 0x7fc5dc154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33266/0 (socket says 172.21.15.7:33266) 2024-09-13T18:53:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5da59c700 1 -- 172.21.15.7:0/3214893760 learned_addr learned my addr 172.21.15.7:0/3214893760 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5d9d9b700 1 --2- 172.21.15.7:0/3214893760 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5dc0b85b0 0x7fc5dc1f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5dad9d700 1 --2- 172.21.15.7:0/3214893760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5dc0b89f0 0x7fc5dc1f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5da59c700 1 -- 172.21.15.7:0/3214893760 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5dc0b85b0 msgr2=0x7fc5dc1f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5da59c700 1 --2- 172.21.15.7:0/3214893760 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5dc0b85b0 0x7fc5dc1f5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5da59c700 1 -- 172.21.15.7:0/3214893760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5dc0b89f0 msgr2=0x7fc5dc1f6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:54.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5da59c700 1 --2- 172.21.15.7:0/3214893760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5dc0b89f0 0x7fc5dc1f6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5da59c700 1 -- 172.21.15.7:0/3214893760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5dc06c4a0 con 0x7fc5c4004870 2024-09-13T18:53:54.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.567+0000 7fc5dad9d700 1 --2- 172.21.15.7:0/3214893760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5dc0b89f0 0x7fc5dc1f6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:54.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.571+0000 7fc5da59c700 1 --2- 172.21.15.7:0/3214893760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5c4004870 0x7fc5dc154280 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000f7f0 tx=0x7fc5d000fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:54.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.571+0000 7fc5caffd700 1 -- 172.21.15.7:0/3214893760 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0005320 con 0x7fc5c4004870 2024-09-13T18:53:54.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.571+0000 7fc5caffd700 1 -- 172.21.15.7:0/3214893760 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d0005480 con 0x7fc5c4004870 2024-09-13T18:53:54.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.571+0000 7fc5e0fa9700 1 -- 172.21.15.7:0/3214893760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5dc06c780 con 0x7fc5c4004870 2024-09-13T18:53:54.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.571+0000 7fc5caffd700 1 -- 172.21.15.7:0/3214893760 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0006aa0 con 0x7fc5c4004870 2024-09-13T18:53:54.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.571+0000 7fc5e0fa9700 1 -- 172.21.15.7:0/3214893760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5dc1f6940 con 0x7fc5c4004870 2024-09-13T18:53:54.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.571+0000 7fc5e0fa9700 1 -- 172.21.15.7:0/3214893760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5dc0b1f20 con 0x7fc5c4004870 2024-09-13T18:53:54.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.571+0000 7fc5caffd700 1 -- 172.21.15.7:0/3214893760 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc5d0006c00 con 0x7fc5c4004870 2024-09-13T18:53:54.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.571+0000 7fc5caffd700 1 --2- 172.21.15.7:0/3214893760 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc5bc066730 0x7fc5bc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.571+0000 7fc5caffd700 1 -- 172.21.15.7:0/3214893760 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc5d00839d0 con 0x7fc5c4004870 2024-09-13T18:53:54.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.571+0000 7fc5d9d9b700 1 --2- 172.21.15.7:0/3214893760 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc5bc066730 0x7fc5bc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.575+0000 7fc5d9d9b700 1 --2- 172.21.15.7:0/3214893760 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc5bc066730 0x7fc5bc068bf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc000c00 tx=0x7fc5cc007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:54.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.579+0000 7fc5caffd700 1 -- 172.21.15.7:0/3214893760 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc5d0050ce0 con 0x7fc5c4004870 2024-09-13T18:53:54.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.751+0000 7fc5e0fa9700 1 -- 172.21.15.7:0/3214893760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5dc0ce520 con 0x7fc5c4004870 2024-09-13T18:53:54.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.751+0000 7fc5caffd700 1 -- 172.21.15.7:0/3214893760 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5d001a0a0 con 0x7fc5c4004870 2024-09-13T18:53:54.761 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:53:54.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.755+0000 7fc5e0fa9700 1 -- 172.21.15.7:0/3214893760 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc5bc066730 msgr2=0x7fc5bc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:54.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.755+0000 7fc5e0fa9700 1 --2- 172.21.15.7:0/3214893760 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc5bc066730 0x7fc5bc068bf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc000c00 tx=0x7fc5cc007000 comp rx=0 tx=0).stop 2024-09-13T18:53:54.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.755+0000 7fc5e0fa9700 1 -- 172.21.15.7:0/3214893760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5c4004870 msgr2=0x7fc5dc154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:54.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.755+0000 7fc5e0fa9700 1 --2- 172.21.15.7:0/3214893760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5c4004870 0x7fc5dc154280 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000f7f0 tx=0x7fc5d000fb00 comp rx=0 tx=0).stop 2024-09-13T18:53:54.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.755+0000 7fc5e0fa9700 1 -- 172.21.15.7:0/3214893760 shutdown_connections 2024-09-13T18:53:54.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.755+0000 7fc5e0fa9700 1 --2- 172.21.15.7:0/3214893760 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc5bc066730 0x7fc5bc068bf0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.755+0000 7fc5e0fa9700 1 --2- 172.21.15.7:0/3214893760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5dc0b89f0 0x7fc5dc1f6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.755+0000 7fc5e0fa9700 1 --2- 172.21.15.7:0/3214893760 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5dc0b85b0 0x7fc5dc1f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.755+0000 7fc5e0fa9700 1 --2- 172.21.15.7:0/3214893760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5c4004870 0x7fc5dc154280 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.755+0000 7fc5e0fa9700 1 -- 172.21.15.7:0/3214893760 >> 172.21.15.7:0/3214893760 conn(0x7fc5dc09c180 msgr2=0x7fc5dc0cd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:54.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.755+0000 7fc5e0fa9700 1 -- 172.21.15.7:0/3214893760 shutdown_connections 2024-09-13T18:53:54.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.755+0000 7fc5e0fa9700 1 -- 172.21.15.7:0/3214893760 wait complete. 2024-09-13T18:53:54.775 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:53:54.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f270df55700 1 Processor -- start 2024-09-13T18:53:54.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f270df55700 1 -- start start 2024-09-13T18:53:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f270df55700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f27081606a0 0x7f27080d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f270df55700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f2708160b10 0x7f2708160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f270df55700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27080d1ca0 con 0x7f27080d17a0 2024-09-13T18:53:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f270df55700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27081614e0 con 0x7f27081606a0 2024-09-13T18:53:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f270df55700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27080cdd00 con 0x7f2708160b10 2024-09-13T18:53:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f270cf53700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f27081606a0 0x7f27080d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f2707fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f27080d17a0 0x7f27080ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45550/0 (socket says 172.21.15.7:45550) 2024-09-13T18:53:54.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f2707fff700 1 -- 172.21.15.7:0/2245951688 learned_addr learned my addr 172.21.15.7:0/2245951688 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:54.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f270d754700 1 --2- 172.21.15.7:0/2245951688 >> v2:172.21.15.171:3300/0 conn(0x7f2708160b10 0x7f2708160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f27077fe700 1 -- 172.21.15.7:0/2245951688 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3653962990 0 0) 0x7f27080d1ca0 con 0x7f27080d17a0 2024-09-13T18:53:54.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f27077fe700 1 -- 172.21.15.7:0/2245951688 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26f00035d0 con 0x7f27080d17a0 2024-09-13T18:53:54.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f270cf53700 1 -- 172.21.15.7:0/2245951688 >> v2:172.21.15.171:3300/0 conn(0x7f2708160b10 msgr2=0x7f2708160fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:54.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f270cf53700 1 --2- 172.21.15.7:0/2245951688 >> v2:172.21.15.171:3300/0 conn(0x7f2708160b10 0x7f2708160fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f270cf53700 1 -- 172.21.15.7:0/2245951688 >> v1:172.21.15.7:6789/0 conn(0x7f27080d17a0 legacy=0x7f27080ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:54.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f270cf53700 1 -- 172.21.15.7:0/2245951688 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27080cdfe0 con 0x7f27081606a0 2024-09-13T18:53:54.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f270cf53700 1 --2- 172.21.15.7:0/2245951688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27081606a0 0x7f27080d3800 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009d90 tx=0x7f26f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8657e1cdcdf5400 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:54.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f27077fe700 1 -- 172.21.15.7:0/2245951688 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f800e070 con 0x7f27081606a0 2024-09-13T18:53:54.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f27077fe700 1 -- 172.21.15.7:0/2245951688 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f800bae0 con 0x7f27081606a0 2024-09-13T18:53:54.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.887+0000 7f27077fe700 1 -- 172.21.15.7:0/2245951688 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f800be90 con 0x7f27081606a0 2024-09-13T18:53:54.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.891+0000 7f270df55700 1 -- 172.21.15.7:0/2245951688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27081606a0 msgr2=0x7f27080d3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:54.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.891+0000 7f270df55700 1 --2- 172.21.15.7:0/2245951688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27081606a0 0x7f27080d3800 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009d90 tx=0x7f26f800b330 comp rx=0 tx=0).stop 2024-09-13T18:53:54.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.891+0000 7f270df55700 1 -- 172.21.15.7:0/2245951688 shutdown_connections 2024-09-13T18:53:54.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.891+0000 7f270df55700 1 --2- 172.21.15.7:0/2245951688 >> v2:172.21.15.171:3300/0 conn(0x7f2708160b10 0x7f2708160fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.891+0000 7f270df55700 1 --2- 172.21.15.7:0/2245951688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27081606a0 0x7f27080d3800 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.891+0000 7f270df55700 1 -- 172.21.15.7:0/2245951688 >> 172.21.15.7:0/2245951688 conn(0x7f27080d2d10 msgr2=0x7f27080d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:54.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.891+0000 7f270df55700 1 -- 172.21.15.7:0/2245951688 shutdown_connections 2024-09-13T18:53:54.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.891+0000 7f270df55700 1 -- 172.21.15.7:0/2245951688 wait complete. 2024-09-13T18:53:54.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.891+0000 7f270df55700 1 Processor -- start 2024-09-13T18:53:54.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.891+0000 7f270df55700 1 -- start start 2024-09-13T18:53:54.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270df55700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27080d17a0 0x7f27080c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270df55700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f27081606a0 0x7f27080c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270df55700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2708160b10 0x7f27080c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270df55700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27080c9160 con 0x7f2708160b10 2024-09-13T18:53:54.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270df55700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27080c6db0 con 0x7f27080d17a0 2024-09-13T18:53:54.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270df55700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270815fab0 con 0x7f27081606a0 2024-09-13T18:53:54.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270d754700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2708160b10 0x7f27080c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270d754700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2708160b10 0x7f27080c8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33274/0 (socket says 172.21.15.7:33274) 2024-09-13T18:53:54.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270d754700 1 -- 172.21.15.7:0/3246219845 learned_addr learned my addr 172.21.15.7:0/3246219845 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:54.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f2707fff700 1 --2- 172.21.15.7:0/3246219845 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f27081606a0 0x7f27080c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270cf53700 1 --2- 172.21.15.7:0/3246219845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27080d17a0 0x7f27080c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270d754700 1 -- 172.21.15.7:0/3246219845 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f27081606a0 msgr2=0x7f27080c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:54.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270d754700 1 --2- 172.21.15.7:0/3246219845 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f27081606a0 0x7f27080c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270d754700 1 -- 172.21.15.7:0/3246219845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27080d17a0 msgr2=0x7f27080c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:54.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270d754700 1 --2- 172.21.15.7:0/3246219845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27080d17a0 0x7f27080c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:54.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270d754700 1 -- 172.21.15.7:0/3246219845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f8009a70 con 0x7f2708160b10 2024-09-13T18:53:54.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270d754700 1 --2- 172.21.15.7:0/3246219845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2708160b10 0x7f27080c8c20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f270000e820 tx=0x7f270000eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:54.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f27057fa700 1 -- 172.21.15.7:0/3246219845 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27000095c0 con 0x7f2708160b10 2024-09-13T18:53:54.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f27057fa700 1 -- 172.21.15.7:0/3246219845 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2700013680 con 0x7f2708160b10 2024-09-13T18:53:54.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270df55700 1 -- 172.21.15.7:0/3246219845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270815f550 con 0x7f2708160b10 2024-09-13T18:53:54.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f27057fa700 1 -- 172.21.15.7:0/3246219845 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2700013a10 con 0x7f2708160b10 2024-09-13T18:53:54.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270df55700 1 -- 172.21.15.7:0/3246219845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27080cafd0 con 0x7f2708160b10 2024-09-13T18:53:54.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.895+0000 7f270df55700 1 -- 172.21.15.7:0/3246219845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27080cae30 con 0x7f2708160b10 2024-09-13T18:53:54.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.899+0000 7f27057fa700 1 -- 172.21.15.7:0/3246219845 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2700013b70 con 0x7f2708160b10 2024-09-13T18:53:54.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.899+0000 7f27057fa700 1 --2- 172.21.15.7:0/3246219845 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f26f4066760 0x7f26f4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:54.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.899+0000 7f27057fa700 1 -- 172.21.15.7:0/3246219845 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2700083f40 con 0x7f2708160b10 2024-09-13T18:53:54.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.899+0000 7f270cf53700 1 --2- 172.21.15.7:0/3246219845 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f26f4066760 0x7f26f4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:54.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.899+0000 7f270cf53700 1 --2- 172.21.15.7:0/3246219845 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f26f4066760 0x7f26f4068c20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f26f8000f80 tx=0x7f26f8011000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:54.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:54.907+0000 7f27057fa700 1 -- 172.21.15.7:0/3246219845 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f27080cae30 con 0x7f2708160b10 2024-09-13T18:53:55.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.075+0000 7f270df55700 1 -- 172.21.15.7:0/3246219845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f27080cae30 con 0x7f2708160b10 2024-09-13T18:53:55.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.075+0000 7f27057fa700 1 -- 172.21.15.7:0/3246219845 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f27080cae30 con 0x7f2708160b10 2024-09-13T18:53:55.086 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:53:55.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.079+0000 7f270df55700 1 -- 172.21.15.7:0/3246219845 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f26f4066760 msgr2=0x7f26f4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.079+0000 7f270df55700 1 --2- 172.21.15.7:0/3246219845 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f26f4066760 0x7f26f4068c20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f26f8000f80 tx=0x7f26f8011000 comp rx=0 tx=0).stop 2024-09-13T18:53:55.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.079+0000 7f270df55700 1 -- 172.21.15.7:0/3246219845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2708160b10 msgr2=0x7f27080c8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.079+0000 7f270df55700 1 --2- 172.21.15.7:0/3246219845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2708160b10 0x7f27080c8c20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f270000e820 tx=0x7f270000eb30 comp rx=0 tx=0).stop 2024-09-13T18:53:55.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.079+0000 7f270df55700 1 -- 172.21.15.7:0/3246219845 shutdown_connections 2024-09-13T18:53:55.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.079+0000 7f270df55700 1 --2- 172.21.15.7:0/3246219845 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f26f4066760 0x7f26f4068c20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.079+0000 7f270df55700 1 --2- 172.21.15.7:0/3246219845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2708160b10 0x7f27080c8c20 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.079+0000 7f270df55700 1 --2- 172.21.15.7:0/3246219845 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f27081606a0 0x7f27080c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.079+0000 7f270df55700 1 --2- 172.21.15.7:0/3246219845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27080d17a0 0x7f27080c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.079+0000 7f270df55700 1 -- 172.21.15.7:0/3246219845 >> 172.21.15.7:0/3246219845 conn(0x7f27080d2d10 msgr2=0x7f27080b5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:55.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.079+0000 7f270df55700 1 -- 172.21.15.7:0/3246219845 shutdown_connections 2024-09-13T18:53:55.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.079+0000 7f270df55700 1 -- 172.21.15.7:0/3246219845 wait complete. 2024-09-13T18:53:55.101 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:53:55.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.219+0000 7fa27cbd8700 1 Processor -- start 2024-09-13T18:53:55.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.219+0000 7fa27cbd8700 1 -- start start 2024-09-13T18:53:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.219+0000 7fa27cbd8700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa27815ce80 0x7fa2780ac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.219+0000 7fa27cbd8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa27806f4b0 0x7fa27806d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.219+0000 7fa27cbd8700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa27806d500 con 0x7fa2780aed60 2024-09-13T18:53:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.219+0000 7fa27cbd8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2780d0910 con 0x7fa27806f4b0 2024-09-13T18:53:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.219+0000 7fa27cbd8700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2780d0570 con 0x7fa27815ce80 2024-09-13T18:53:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.219+0000 7fa277fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa27806f4b0 0x7fa27806d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.219+0000 7fa276ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa2780aed60 0x7fa27806cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45554/0 (socket says 172.21.15.7:45554) 2024-09-13T18:53:55.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.219+0000 7fa276ffd700 1 -- 172.21.15.7:0/3378830459 learned_addr learned my addr 172.21.15.7:0/3378830459 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:55.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.219+0000 7fa2777fe700 1 --2- 172.21.15.7:0/3378830459 >> v2:172.21.15.171:3300/0 conn(0x7fa27815ce80 0x7fa2780ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.223+0000 7fa277fff700 1 -- 172.21.15.7:0/3378830459 >> v2:172.21.15.171:3300/0 conn(0x7fa27815ce80 msgr2=0x7fa2780ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:55.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.223+0000 7fa277fff700 1 --2- 172.21.15.7:0/3378830459 >> v2:172.21.15.171:3300/0 conn(0x7fa27815ce80 0x7fa2780ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.223+0000 7fa277fff700 1 -- 172.21.15.7:0/3378830459 >> v1:172.21.15.7:6789/0 conn(0x7fa2780aed60 legacy=0x7fa27806cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.223+0000 7fa2767fc700 1 -- 172.21.15.7:0/3378830459 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576861151 0 0) 0x7fa27806d500 con 0x7fa2780aed60 2024-09-13T18:53:55.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.223+0000 7fa277fff700 1 -- 172.21.15.7:0/3378830459 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2780b48d0 con 0x7fa27806f4b0 2024-09-13T18:53:55.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.223+0000 7fa277fff700 1 --2- 172.21.15.7:0/3378830459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa27806f4b0 0x7fa27806d800 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa278068810 tx=0x7fa27000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93253dc517620c6c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:55.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.223+0000 7fa2767fc700 1 -- 172.21.15.7:0/3378830459 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27000e070 con 0x7fa27806f4b0 2024-09-13T18:53:55.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.223+0000 7fa2767fc700 1 -- 172.21.15.7:0/3378830459 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27000bb20 con 0x7fa27806f4b0 2024-09-13T18:53:55.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.223+0000 7fa2767fc700 1 -- 172.21.15.7:0/3378830459 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa270012430 con 0x7fa27806f4b0 2024-09-13T18:53:55.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.223+0000 7fa27cbd8700 1 -- 172.21.15.7:0/3378830459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa27806f4b0 msgr2=0x7fa27806d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.223+0000 7fa27cbd8700 1 --2- 172.21.15.7:0/3378830459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa27806f4b0 0x7fa27806d800 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa278068810 tx=0x7fa27000b330 comp rx=0 tx=0).stop 2024-09-13T18:53:55.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.223+0000 7fa27cbd8700 1 -- 172.21.15.7:0/3378830459 shutdown_connections 2024-09-13T18:53:55.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.223+0000 7fa27cbd8700 1 --2- 172.21.15.7:0/3378830459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa27806f4b0 0x7fa27806d800 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.223+0000 7fa27cbd8700 1 --2- 172.21.15.7:0/3378830459 >> v2:172.21.15.171:3300/0 conn(0x7fa27815ce80 0x7fa2780ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.223+0000 7fa27cbd8700 1 -- 172.21.15.7:0/3378830459 >> 172.21.15.7:0/3378830459 conn(0x7fa2780cf9b0 msgr2=0x7fa2780cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:55.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.223+0000 7fa27cbd8700 1 -- 172.21.15.7:0/3378830459 shutdown_connections 2024-09-13T18:53:55.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.223+0000 7fa27cbd8700 1 -- 172.21.15.7:0/3378830459 wait complete. 2024-09-13T18:53:55.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa27cbd8700 1 Processor -- start 2024-09-13T18:53:55.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa27cbd8700 1 -- start start 2024-09-13T18:53:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa27cbd8700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa27806f4b0 0x7fa278144520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa27cbd8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2780aed60 0x7fa27815f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa27cbd8700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa27815ce80 0x7fa278152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa27cbd8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa278144a60 con 0x7fa2780aed60 2024-09-13T18:53:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa27cbd8700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2781563b0 con 0x7fa27815ce80 2024-09-13T18:53:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa27cbd8700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa278155a80 con 0x7fa27806f4b0 2024-09-13T18:53:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa276ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2780aed60 0x7fa27815f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa276ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2780aed60 0x7fa27815f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33280/0 (socket says 172.21.15.7:33280) 2024-09-13T18:53:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa276ffd700 1 -- 172.21.15.7:0/2847240674 learned_addr learned my addr 172.21.15.7:0/2847240674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa2777fe700 1 --2- 172.21.15.7:0/2847240674 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa27806f4b0 0x7fa278144520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa277fff700 1 --2- 172.21.15.7:0/2847240674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa27815ce80 0x7fa278152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa276ffd700 1 -- 172.21.15.7:0/2847240674 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa27806f4b0 msgr2=0x7fa278144520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa276ffd700 1 --2- 172.21.15.7:0/2847240674 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa27806f4b0 0x7fa278144520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa276ffd700 1 -- 172.21.15.7:0/2847240674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa27815ce80 msgr2=0x7fa278152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa276ffd700 1 --2- 172.21.15.7:0/2847240674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa27815ce80 0x7fa278152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa276ffd700 1 -- 172.21.15.7:0/2847240674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa270009a70 con 0x7fa2780aed60 2024-09-13T18:53:55.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa277fff700 1 --2- 172.21.15.7:0/2847240674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa27815ce80 0x7fa278152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:55.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa276ffd700 1 --2- 172.21.15.7:0/2847240674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2780aed60 0x7fa27815f980 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00c550 tx=0x7fa26c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:55.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa25ffff700 1 -- 172.21.15.7:0/2847240674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c004ca0 con 0x7fa2780aed60 2024-09-13T18:53:55.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa25ffff700 1 -- 172.21.15.7:0/2847240674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26c004e00 con 0x7fa2780aed60 2024-09-13T18:53:55.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa27cbd8700 1 -- 172.21.15.7:0/2847240674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa278155d60 con 0x7fa2780aed60 2024-09-13T18:53:55.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa25ffff700 1 -- 172.21.15.7:0/2847240674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c007310 con 0x7fa2780aed60 2024-09-13T18:53:55.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.227+0000 7fa27cbd8700 1 -- 172.21.15.7:0/2847240674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27815fec0 con 0x7fa2780aed60 2024-09-13T18:53:55.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.231+0000 7fa27cbd8700 1 -- 172.21.15.7:0/2847240674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa278067830 con 0x7fa2780aed60 2024-09-13T18:53:55.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.231+0000 7fa25ffff700 1 -- 172.21.15.7:0/2847240674 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa26c007470 con 0x7fa2780aed60 2024-09-13T18:53:55.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.231+0000 7fa25ffff700 1 --2- 172.21.15.7:0/2847240674 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa264066760 0x7fa264068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.231+0000 7fa25ffff700 1 -- 172.21.15.7:0/2847240674 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa26c083e30 con 0x7fa2780aed60 2024-09-13T18:53:55.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.231+0000 7fa2777fe700 1 --2- 172.21.15.7:0/2847240674 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa264066760 0x7fa264068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.235+0000 7fa2777fe700 1 --2- 172.21.15.7:0/2847240674 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa264066760 0x7fa264068c20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa268009b50 tx=0x7fa268009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:55.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.239+0000 7fa25ffff700 1 -- 172.21.15.7:0/2847240674 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa26c0511d0 con 0x7fa2780aed60 2024-09-13T18:53:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.407+0000 7fa27cbd8700 1 -- 172.21.15.7:0/2847240674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa2780ac860 con 0x7fa2780aed60 2024-09-13T18:53:55.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.407+0000 7fa25ffff700 1 -- 172.21.15.7:0/2847240674 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa26c007720 con 0x7fa2780aed60 2024-09-13T18:53:55.416 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:53:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.411+0000 7fa27cbd8700 1 -- 172.21.15.7:0/2847240674 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa264066760 msgr2=0x7fa264068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.411+0000 7fa27cbd8700 1 --2- 172.21.15.7:0/2847240674 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa264066760 0x7fa264068c20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa268009b50 tx=0x7fa268009240 comp rx=0 tx=0).stop 2024-09-13T18:53:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.411+0000 7fa27cbd8700 1 -- 172.21.15.7:0/2847240674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2780aed60 msgr2=0x7fa27815f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.411+0000 7fa27cbd8700 1 --2- 172.21.15.7:0/2847240674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2780aed60 0x7fa27815f980 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00c550 tx=0x7fa26c00c860 comp rx=0 tx=0).stop 2024-09-13T18:53:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.411+0000 7fa27cbd8700 1 -- 172.21.15.7:0/2847240674 shutdown_connections 2024-09-13T18:53:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.411+0000 7fa27cbd8700 1 --2- 172.21.15.7:0/2847240674 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa264066760 0x7fa264068c20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.411+0000 7fa27cbd8700 1 --2- 172.21.15.7:0/2847240674 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa27815ce80 0x7fa278152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.411+0000 7fa27cbd8700 1 --2- 172.21.15.7:0/2847240674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2780aed60 0x7fa27815f980 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.411+0000 7fa27cbd8700 1 --2- 172.21.15.7:0/2847240674 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa27806f4b0 0x7fa278144520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.411+0000 7fa27cbd8700 1 -- 172.21.15.7:0/2847240674 >> 172.21.15.7:0/2847240674 conn(0x7fa2780cf9b0 msgr2=0x7fa2780ceba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.411+0000 7fa27cbd8700 1 -- 172.21.15.7:0/2847240674 shutdown_connections 2024-09-13T18:53:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.411+0000 7fa27cbd8700 1 -- 172.21.15.7:0/2847240674 wait complete. 2024-09-13T18:53:55.430 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:55.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.531+0000 7fccf359e700 1 Processor -- start 2024-09-13T18:53:55.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.531+0000 7fccf359e700 1 -- start start 2024-09-13T18:53:55.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf359e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccf40ada40 0x7fccf4070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf359e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf40b3b40 con 0x7fccf406cdc0 2024-09-13T18:53:55.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf359e700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf40b3860 con 0x7fccf4160ff0 2024-09-13T18:53:55.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf359e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf4071a20 con 0x7fccf40ada40 2024-09-13T18:53:55.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf2d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fccf406cdc0 0x7fccf4164b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45568/0 (socket says 172.21.15.7:45568) 2024-09-13T18:53:55.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf2d9d700 1 -- 172.21.15.7:0/3281051927 learned_addr learned my addr 172.21.15.7:0/3281051927 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:55.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf1d9b700 1 --2- 172.21.15.7:0/3281051927 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccf40ada40 0x7fccf4070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 -- 172.21.15.7:0/3281051927 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855931425 0 0) 0x7fccf40b3b40 con 0x7fccf406cdc0 2024-09-13T18:53:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 -- 172.21.15.7:0/3281051927 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccdc0035d0 con 0x7fccf406cdc0 2024-09-13T18:53:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 -- 172.21.15.7:0/3281051927 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929046808 0 0) 0x7fccf40b3860 con 0x7fccf4160ff0 2024-09-13T18:53:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 -- 172.21.15.7:0/3281051927 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccf40b3b40 con 0x7fccf4160ff0 2024-09-13T18:53:55.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 -- 172.21.15.7:0/3281051927 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 239050153 0 0) 0x7fccdc0035d0 con 0x7fccf406cdc0 2024-09-13T18:53:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 -- 172.21.15.7:0/3281051927 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccf40ada40 msgr2=0x7fccf4070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 --2- 172.21.15.7:0/3281051927 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccf40ada40 0x7fccf4070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 -- 172.21.15.7:0/3281051927 >> v1:172.21.15.171:6789/0 conn(0x7fccf4160ff0 legacy=0x7fccf406c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 -- 172.21.15.7:0/3281051927 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf4071d00 con 0x7fccf406cdc0 2024-09-13T18:53:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 -- 172.21.15.7:0/3281051927 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7fccec001670 con 0x7fccf406cdc0 2024-09-13T18:53:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 -- 172.21.15.7:0/3281051927 >> v1:172.21.15.7:6789/0 conn(0x7fccf406cdc0 legacy=0x7fccf4164b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 --2- 172.21.15.7:0/3281051927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccdc003be0 0x7fccdc003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 --2- 172.21.15.7:0/3281051927 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccf40ada40 0x7fccdc004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 --2- 172.21.15.7:0/3281051927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccdc004db0 0x7fccdc005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 -- 172.21.15.7:0/3281051927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf4071a20 con 0x7fccdc003be0 2024-09-13T18:53:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 -- 172.21.15.7:0/3281051927 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc00bae0 con 0x7fccdc004db0 2024-09-13T18:53:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 -- 172.21.15.7:0/3281051927 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc00bc50 con 0x7fccf40ada40 2024-09-13T18:53:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf2d9d700 1 --2- 172.21.15.7:0/3281051927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccdc004db0 0x7fccdc005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf259c700 1 --2- 172.21.15.7:0/3281051927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccdc003be0 0x7fccdc003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf1d9b700 1 --2- 172.21.15.7:0/3281051927 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccf40ada40 0x7fccdc004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf1d9b700 1 -- 172.21.15.7:0/3281051927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccdc004db0 msgr2=0x7fccdc005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf1d9b700 1 --2- 172.21.15.7:0/3281051927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccdc004db0 0x7fccdc005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf1d9b700 1 -- 172.21.15.7:0/3281051927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccdc003be0 msgr2=0x7fccdc003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf1d9b700 1 --2- 172.21.15.7:0/3281051927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccdc003be0 0x7fccdc003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf1d9b700 1 -- 172.21.15.7:0/3281051927 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccdc0035d0 con 0x7fccf40ada40 2024-09-13T18:53:55.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf2d9d700 1 --2- 172.21.15.7:0/3281051927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccdc004db0 0x7fccdc005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf1d9b700 1 --2- 172.21.15.7:0/3281051927 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccf40ada40 0x7fccdc004870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009f90 tx=0x7fccdc0058a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b58d856de647272 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 -- 172.21.15.7:0/3281051927 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce400ec30 con 0x7fccf40ada40 2024-09-13T18:53:55.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 -- 172.21.15.7:0/3281051927 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce400f040 con 0x7fccf40ada40 2024-09-13T18:53:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.535+0000 7fccf159a700 1 -- 172.21.15.7:0/3281051927 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce4014480 con 0x7fccf40ada40 2024-09-13T18:53:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.539+0000 7fccf359e700 1 -- 172.21.15.7:0/3281051927 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccf40ada40 msgr2=0x7fccdc004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.539+0000 7fccf359e700 1 --2- 172.21.15.7:0/3281051927 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccf40ada40 0x7fccdc004870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009f90 tx=0x7fccdc0058a0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.539+0000 7fccf2d9d700 1 -- 172.21.15.7:0/3281051927 reap_dead start 2024-09-13T18:53:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.539+0000 7fccf359e700 1 -- 172.21.15.7:0/3281051927 shutdown_connections 2024-09-13T18:53:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.539+0000 7fccf359e700 1 -- 172.21.15.7:0/3281051927 >> 172.21.15.7:0/3281051927 conn(0x7fccf40b0570 msgr2=0x7fccf40b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.539+0000 7fccf359e700 1 -- 172.21.15.7:0/3281051927 shutdown_connections 2024-09-13T18:53:55.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.539+0000 7fccf359e700 1 -- 172.21.15.7:0/3281051927 wait complete. 2024-09-13T18:53:55.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.539+0000 7fccf359e700 1 Processor -- start 2024-09-13T18:53:55.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf359e700 1 -- start start 2024-09-13T18:53:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf359e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccf40ada40 0x7fccf4147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccf4149af0 0x7fccf4149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf359e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccf414a490 0x7fccf4158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf359e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf415a030 con 0x7fccf4149af0 2024-09-13T18:53:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf359e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf4159a70 con 0x7fccf40ada40 2024-09-13T18:53:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf359e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf4159d50 con 0x7fccf414a490 2024-09-13T18:53:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccf4149af0 0x7fccf4149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccf4149af0 0x7fccf4149f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33302/0 (socket says 172.21.15.7:33302) 2024-09-13T18:53:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf1d9b700 1 -- 172.21.15.7:0/4140904668 learned_addr learned my addr 172.21.15.7:0/4140904668 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:55.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf259c700 1 --2- 172.21.15.7:0/4140904668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccf40ada40 0x7fccf4147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf2d9d700 1 --2- 172.21.15.7:0/4140904668 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccf414a490 0x7fccf4158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf1d9b700 1 -- 172.21.15.7:0/4140904668 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccf414a490 msgr2=0x7fccf4158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf1d9b700 1 --2- 172.21.15.7:0/4140904668 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccf414a490 0x7fccf4158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf1d9b700 1 -- 172.21.15.7:0/4140904668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccf40ada40 msgr2=0x7fccf4147150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf1d9b700 1 --2- 172.21.15.7:0/4140904668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccf40ada40 0x7fccf4147150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf1d9b700 1 -- 172.21.15.7:0/4140904668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf4147c00 con 0x7fccf4149af0 2024-09-13T18:53:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf1d9b700 1 --2- 172.21.15.7:0/4140904668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccf4149af0 0x7fccf4149f50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcce400afd0 tx=0x7fcce4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fcce2ffd700 1 -- 172.21.15.7:0/4140904668 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce4004450 con 0x7fccf4149af0 2024-09-13T18:53:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fcce2ffd700 1 -- 172.21.15.7:0/4140904668 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce4022070 con 0x7fccf4149af0 2024-09-13T18:53:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf359e700 1 -- 172.21.15.7:0/4140904668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf4147ee0 con 0x7fccf4149af0 2024-09-13T18:53:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fcce2ffd700 1 -- 172.21.15.7:0/4140904668 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce40047d0 con 0x7fccf4149af0 2024-09-13T18:53:55.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.543+0000 7fccf359e700 1 -- 172.21.15.7:0/4140904668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf414a910 con 0x7fccf4149af0 2024-09-13T18:53:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.547+0000 7fccf359e700 1 -- 172.21.15.7:0/4140904668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf4147a60 con 0x7fccf4149af0 2024-09-13T18:53:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.547+0000 7fcce2ffd700 1 -- 172.21.15.7:0/4140904668 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcce4029480 con 0x7fccf4149af0 2024-09-13T18:53:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.547+0000 7fcce2ffd700 1 --2- 172.21.15.7:0/4140904668 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fccd4066730 0x7fccd4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.547+0000 7fcce2ffd700 1 -- 172.21.15.7:0/4140904668 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcce4024070 con 0x7fccf4149af0 2024-09-13T18:53:55.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.547+0000 7fccf259c700 1 --2- 172.21.15.7:0/4140904668 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fccd4066730 0x7fccd4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.547+0000 7fccf259c700 1 --2- 172.21.15.7:0/4140904668 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fccd4066730 0x7fccd4068bf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcce8000c00 tx=0x7fcce8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:55.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.555+0000 7fcce2ffd700 1 -- 172.21.15.7:0/4140904668 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fccf4147a60 con 0x7fccf4149af0 2024-09-13T18:53:55.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.763+0000 7fccf359e700 1 -- 172.21.15.7:0/4140904668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fccf4147a60 con 0x7fccf4149af0 2024-09-13T18:53:55.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.763+0000 7fcce2ffd700 1 -- 172.21.15.7:0/4140904668 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4732 (secure 0 0 0) 0x7fccf4147a60 con 0x7fccf4149af0 2024-09-13T18:53:55.771 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:55.771 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2458672850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2458672850},{"type":"v1","addr":"172.21.15.171:6835","nonce":2458672850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3381042298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3381042298},{"type":"v1","addr":"172.21.15.171:6837","nonce":3381042298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2068823578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2068823578},{"type":"v1","addr":"172.21.15.7:6833","nonce":2068823578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:53:17.897317+0000","modified":"2024-09-13T18:53:30.253515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5463":{"gid":5463,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/127797392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":127797392},{"type":"v1","addr":"172.21.15.7:6835","nonce":127797392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T18:53:55.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.763+0000 7fccf359e700 1 -- 172.21.15.7:0/4140904668 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fccd4066730 msgr2=0x7fccd4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.763+0000 7fccf359e700 1 --2- 172.21.15.7:0/4140904668 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fccd4066730 0x7fccd4068bf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcce8000c00 tx=0x7fcce8009040 comp rx=0 tx=0).stop 2024-09-13T18:53:55.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.763+0000 7fccf359e700 1 -- 172.21.15.7:0/4140904668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccf4149af0 msgr2=0x7fccf4149f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.763+0000 7fccf359e700 1 --2- 172.21.15.7:0/4140904668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccf4149af0 0x7fccf4149f50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcce400afd0 tx=0x7fcce4009750 comp rx=0 tx=0).stop 2024-09-13T18:53:55.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.763+0000 7fccf359e700 1 -- 172.21.15.7:0/4140904668 shutdown_connections 2024-09-13T18:53:55.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.763+0000 7fccf359e700 1 --2- 172.21.15.7:0/4140904668 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fccd4066730 0x7fccd4068bf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.763+0000 7fccf359e700 1 --2- 172.21.15.7:0/4140904668 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fccf414a490 0x7fccf4158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.763+0000 7fccf359e700 1 --2- 172.21.15.7:0/4140904668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccf4149af0 0x7fccf4149f50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.763+0000 7fccf359e700 1 --2- 172.21.15.7:0/4140904668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccf40ada40 0x7fccf4147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.763+0000 7fccf359e700 1 -- 172.21.15.7:0/4140904668 >> 172.21.15.7:0/4140904668 conn(0x7fccf40b0570 msgr2=0x7fccf40b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:55.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.763+0000 7fccf359e700 1 -- 172.21.15.7:0/4140904668 shutdown_connections 2024-09-13T18:53:55.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.763+0000 7fccf359e700 1 -- 172.21.15.7:0/4140904668 wait complete. 2024-09-13T18:53:55.777 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-09-13T18:53:55.786 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:53:55.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.887+0000 7fade6366700 1 Processor -- start 2024-09-13T18:53:55.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.887+0000 7fade6366700 1 -- start start 2024-09-13T18:53:55.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade6366700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fade00ad6b0 0x7fade015bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade6366700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fade00718d0 0x7fade00ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade6366700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade006fc80 con 0x7fade015e600 2024-09-13T18:53:55.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade6366700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade00adfd0 con 0x7fade00718d0 2024-09-13T18:53:55.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade6366700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade006fed0 con 0x7fade00ad6b0 2024-09-13T18:53:55.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade4b63700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fade00718d0 0x7fade00ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade5b65700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fade015e600 0x7fade015eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45572/0 (socket says 172.21.15.7:45572) 2024-09-13T18:53:55.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade5b65700 1 -- 172.21.15.7:0/1073586147 learned_addr learned my addr 172.21.15.7:0/1073586147 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:55.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade5364700 1 --2- 172.21.15.7:0/1073586147 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fade00ad6b0 0x7fade015bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade4b63700 1 -- 172.21.15.7:0/1073586147 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fade00ad6b0 msgr2=0x7fade015bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade4b63700 1 --2- 172.21.15.7:0/1073586147 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fade00ad6b0 0x7fade015bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade4b63700 1 -- 172.21.15.7:0/1073586147 >> v1:172.21.15.7:6789/0 conn(0x7fade015e600 legacy=0x7fade015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade4b63700 1 -- 172.21.15.7:0/1073586147 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade00701b0 con 0x7fade00718d0 2024-09-13T18:53:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade4b63700 1 --2- 172.21.15.7:0/1073586147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade00718d0 0x7fade00ada90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7faddc009850 tx=0x7faddc009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a080d8245573d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fadd7fff700 1 -- 172.21.15.7:0/1073586147 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc011830 con 0x7fade00718d0 2024-09-13T18:53:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fadd7fff700 1 -- 172.21.15.7:0/1073586147 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc011990 con 0x7fade00718d0 2024-09-13T18:53:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fadd7fff700 1 -- 172.21.15.7:0/1073586147 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc011d20 con 0x7fade00718d0 2024-09-13T18:53:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade6366700 1 -- 172.21.15.7:0/1073586147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade00718d0 msgr2=0x7fade00ada90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade6366700 1 --2- 172.21.15.7:0/1073586147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade00718d0 0x7fade00ada90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7faddc009850 tx=0x7faddc009b60 comp rx=0 tx=0).stop 2024-09-13T18:53:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade6366700 1 -- 172.21.15.7:0/1073586147 shutdown_connections 2024-09-13T18:53:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade6366700 1 --2- 172.21.15.7:0/1073586147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade00718d0 0x7fade00ada90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade6366700 1 --2- 172.21.15.7:0/1073586147 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fade00ad6b0 0x7fade015bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade6366700 1 -- 172.21.15.7:0/1073586147 >> 172.21.15.7:0/1073586147 conn(0x7fade00b0570 msgr2=0x7fade00b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:55.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade6366700 1 -- 172.21.15.7:0/1073586147 shutdown_connections 2024-09-13T18:53:55.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.891+0000 7fade6366700 1 -- 172.21.15.7:0/1073586147 wait complete. 2024-09-13T18:53:55.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.895+0000 7fade6366700 1 Processor -- start 2024-09-13T18:53:55.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.895+0000 7fade6366700 1 -- start start 2024-09-13T18:53:55.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.895+0000 7fade6366700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade00718d0 0x7fade0151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.895+0000 7fade6366700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fade00ad6b0 0x7fade015a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.895+0000 7fade6366700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fade015e600 0x7fade0152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.895+0000 7fade6366700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade0152760 con 0x7fade00718d0 2024-09-13T18:53:55.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.895+0000 7fade6366700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade015aad0 con 0x7fade015e600 2024-09-13T18:53:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.895+0000 7fade6366700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade01530c0 con 0x7fade00ad6b0 2024-09-13T18:53:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.895+0000 7fade5364700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade00718d0 0x7fade0151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.899+0000 7fade5364700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade00718d0 0x7fade0151470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33312/0 (socket says 172.21.15.7:33312) 2024-09-13T18:53:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.899+0000 7fade5364700 1 -- 172.21.15.7:0/2092566849 learned_addr learned my addr 172.21.15.7:0/2092566849 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.899+0000 7fade5b65700 1 --2- 172.21.15.7:0/2092566849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fade015e600 0x7fade0152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.899+0000 7fade4b63700 1 --2- 172.21.15.7:0/2092566849 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fade00ad6b0 0x7fade015a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.899+0000 7fade5364700 1 -- 172.21.15.7:0/2092566849 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fade00ad6b0 msgr2=0x7fade015a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.899+0000 7fade5364700 1 --2- 172.21.15.7:0/2092566849 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fade00ad6b0 0x7fade015a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.899+0000 7fade5364700 1 -- 172.21.15.7:0/2092566849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fade015e600 msgr2=0x7fade0152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.899+0000 7fade5364700 1 --2- 172.21.15.7:0/2092566849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fade015e600 0x7fade0152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.899+0000 7fade5364700 1 -- 172.21.15.7:0/2092566849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faddc0094b0 con 0x7fade00718d0 2024-09-13T18:53:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.899+0000 7fade5b65700 1 --2- 172.21.15.7:0/2092566849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fade015e600 0x7fade0152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:55.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.899+0000 7fade4b63700 1 --2- 172.21.15.7:0/2092566849 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fade00ad6b0 0x7fade015a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:55.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.899+0000 7fade5364700 1 --2- 172.21.15.7:0/2092566849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade00718d0 0x7fade0151470 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fadd000e760 tx=0x7fadd000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:55.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.899+0000 7fadd5ffb700 1 -- 172.21.15.7:0/2092566849 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0013610 con 0x7fade00718d0 2024-09-13T18:53:55.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.899+0000 7fadd5ffb700 1 -- 172.21.15.7:0/2092566849 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd0013770 con 0x7fade00718d0 2024-09-13T18:53:55.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.899+0000 7fade6366700 1 -- 172.21.15.7:0/2092566849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade0151ae0 con 0x7fade00718d0 2024-09-13T18:53:55.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.899+0000 7fadd5ffb700 1 -- 172.21.15.7:0/2092566849 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0013ae0 con 0x7fade00718d0 2024-09-13T18:53:55.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.899+0000 7fade6366700 1 -- 172.21.15.7:0/2092566849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade01520a0 con 0x7fade00718d0 2024-09-13T18:53:55.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.899+0000 7fade6366700 1 -- 172.21.15.7:0/2092566849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade0071d10 con 0x7fade00718d0 2024-09-13T18:53:55.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.903+0000 7fadd5ffb700 1 -- 172.21.15.7:0/2092566849 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fadd0013c40 con 0x7fade00718d0 2024-09-13T18:53:55.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.903+0000 7fadd5ffb700 1 --2- 172.21.15.7:0/2092566849 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fadc4066730 0x7fadc4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:55.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.903+0000 7fadd5ffb700 1 -- 172.21.15.7:0/2092566849 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fadd0083df0 con 0x7fade00718d0 2024-09-13T18:53:55.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.903+0000 7fade4b63700 1 --2- 172.21.15.7:0/2092566849 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fadc4066730 0x7fadc4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:55.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.903+0000 7fade4b63700 1 --2- 172.21.15.7:0/2092566849 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fadc4066730 0x7fadc4068bf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faddc009850 tx=0x7faddc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:55.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:55.907+0000 7fadd5ffb700 1 -- 172.21.15.7:0/2092566849 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fadd0051190 con 0x7fade00718d0 2024-09-13T18:53:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.087+0000 7fade6366700 1 -- 172.21.15.7:0/2092566849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fade01529d0 con 0x7fade00718d0 2024-09-13T18:53:56.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.087+0000 7fadd5ffb700 1 -- 172.21.15.7:0/2092566849 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16648 (secure 0 0 0) 0x7fadd00547b0 con 0x7fade00718d0 2024-09-13T18:53:56.098 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:56.098 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:53:26.238445+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:53:15.862087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T18:53:16.885280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/2843879273":"2024-09-14T18:53:26.228249+0000","172.21.15.7:6834/2843879273":"2024-09-14T18:53:26.228249+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:53:56.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.091+0000 7fade6366700 1 -- 172.21.15.7:0/2092566849 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fadc4066730 msgr2=0x7fadc4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.091+0000 7fade6366700 1 --2- 172.21.15.7:0/2092566849 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fadc4066730 0x7fadc4068bf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faddc009850 tx=0x7faddc008030 comp rx=0 tx=0).stop 2024-09-13T18:53:56.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.091+0000 7fade6366700 1 -- 172.21.15.7:0/2092566849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade00718d0 msgr2=0x7fade0151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.091+0000 7fade6366700 1 --2- 172.21.15.7:0/2092566849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade00718d0 0x7fade0151470 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fadd000e760 tx=0x7fadd000eb20 comp rx=0 tx=0).stop 2024-09-13T18:53:56.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.091+0000 7fade6366700 1 -- 172.21.15.7:0/2092566849 shutdown_connections 2024-09-13T18:53:56.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.091+0000 7fade6366700 1 --2- 172.21.15.7:0/2092566849 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fadc4066730 0x7fadc4068bf0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.091+0000 7fade6366700 1 --2- 172.21.15.7:0/2092566849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fade015e600 0x7fade0152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.091+0000 7fade6366700 1 --2- 172.21.15.7:0/2092566849 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fade00ad6b0 0x7fade015a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.091+0000 7fade6366700 1 --2- 172.21.15.7:0/2092566849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fade00718d0 0x7fade0151470 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.091+0000 7fade6366700 1 -- 172.21.15.7:0/2092566849 >> 172.21.15.7:0/2092566849 conn(0x7fade00b0570 msgr2=0x7fade015ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:56.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.091+0000 7fade6366700 1 -- 172.21.15.7:0/2092566849 shutdown_connections 2024-09-13T18:53:56.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.091+0000 7fade6366700 1 -- 172.21.15.7:0/2092566849 wait complete. 2024-09-13T18:53:56.113 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T18:53:56.114 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:56.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc505ba6700 1 Processor -- start 2024-09-13T18:53:56.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc505ba6700 1 -- start start 2024-09-13T18:53:56.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc505ba6700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc50006fee0 0x7fc5000702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc505ba6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50015da20 con 0x7fc50006cef0 2024-09-13T18:53:56.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc505ba6700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5000b1870 con 0x7fc5000b3320 2024-09-13T18:53:56.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc505ba6700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50006d390 con 0x7fc50006fee0 2024-09-13T18:53:56.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc4fffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc50006cef0 0x7fc50015cc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45576/0 (socket says 172.21.15.7:45576) 2024-09-13T18:53:56.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc504ba4700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc50006fee0 0x7fc5000702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc4fffff700 1 -- 172.21.15.7:0/547665003 learned_addr learned my addr 172.21.15.7:0/547665003 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:56.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc4ff7fe700 1 -- 172.21.15.7:0/547665003 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3269234470 0 0) 0x7fc50015da20 con 0x7fc50006cef0 2024-09-13T18:53:56.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc504ba4700 1 -- 172.21.15.7:0/547665003 >> v1:172.21.15.171:6789/0 conn(0x7fc5000b3320 legacy=0x7fc50015d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc504ba4700 1 -- 172.21.15.7:0/547665003 >> v1:172.21.15.7:6789/0 conn(0x7fc50006cef0 legacy=0x7fc50015cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc504ba4700 1 -- 172.21.15.7:0/547665003 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50006d670 con 0x7fc50006fee0 2024-09-13T18:53:56.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc504ba4700 1 --2- 172.21.15.7:0/547665003 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc50006fee0 0x7fc5000702c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009dc0 tx=0x7fc4f800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7f9cd1f623029b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:56.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc4ff7fe700 1 -- 172.21.15.7:0/547665003 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f800e070 con 0x7fc50006fee0 2024-09-13T18:53:56.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc4ff7fe700 1 -- 172.21.15.7:0/547665003 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f800bbd0 con 0x7fc50006fee0 2024-09-13T18:53:56.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc4ff7fe700 1 -- 172.21.15.7:0/547665003 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8012430 con 0x7fc50006fee0 2024-09-13T18:53:56.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc505ba6700 1 -- 172.21.15.7:0/547665003 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc50006fee0 msgr2=0x7fc5000702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc505ba6700 1 --2- 172.21.15.7:0/547665003 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc50006fee0 0x7fc5000702c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009dc0 tx=0x7fc4f800b330 comp rx=0 tx=0).stop 2024-09-13T18:53:56.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc505ba6700 1 -- 172.21.15.7:0/547665003 shutdown_connections 2024-09-13T18:53:56.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc505ba6700 1 --2- 172.21.15.7:0/547665003 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc50006fee0 0x7fc5000702c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc505ba6700 1 -- 172.21.15.7:0/547665003 >> 172.21.15.7:0/547665003 conn(0x7fc5000b0570 msgr2=0x7fc5000b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.223+0000 7fc505ba6700 1 -- 172.21.15.7:0/547665003 shutdown_connections 2024-09-13T18:53:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.227+0000 7fc505ba6700 1 -- 172.21.15.7:0/547665003 wait complete. 2024-09-13T18:53:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.227+0000 7fc505ba6700 1 Processor -- start 2024-09-13T18:53:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.227+0000 7fc505ba6700 1 -- start start 2024-09-13T18:53:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.227+0000 7fc505ba6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50006cef0 0x7fc5000ac540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.227+0000 7fc505ba6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc50006fee0 0x7fc5000a8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.227+0000 7fc505ba6700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5000b3320 0x7fc5000a8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.227+0000 7fc505ba6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc500144df0 con 0x7fc50006cef0 2024-09-13T18:53:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.227+0000 7fc505ba6700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5001450d0 con 0x7fc50006fee0 2024-09-13T18:53:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.227+0000 7fc505ba6700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5001453b0 con 0x7fc5000b3320 2024-09-13T18:53:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.227+0000 7fc504ba4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50006cef0 0x7fc5000ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.227+0000 7fc504ba4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50006cef0 0x7fc5000ac540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33326/0 (socket says 172.21.15.7:33326) 2024-09-13T18:53:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.227+0000 7fc504ba4700 1 -- 172.21.15.7:0/1578615872 learned_addr learned my addr 172.21.15.7:0/1578615872 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.227+0000 7fc4fffff700 1 --2- 172.21.15.7:0/1578615872 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc50006fee0 0x7fc5000a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.227+0000 7fc5053a5700 1 --2- 172.21.15.7:0/1578615872 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5000b3320 0x7fc5000a8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.231+0000 7fc504ba4700 1 -- 172.21.15.7:0/1578615872 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5000b3320 msgr2=0x7fc5000a8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.231+0000 7fc504ba4700 1 --2- 172.21.15.7:0/1578615872 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5000b3320 0x7fc5000a8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.231+0000 7fc504ba4700 1 -- 172.21.15.7:0/1578615872 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc50006fee0 msgr2=0x7fc5000a8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.231+0000 7fc504ba4700 1 --2- 172.21.15.7:0/1578615872 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc50006fee0 0x7fc5000a8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.231+0000 7fc504ba4700 1 -- 172.21.15.7:0/1578615872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f8009a70 con 0x7fc50006cef0 2024-09-13T18:53:56.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.231+0000 7fc4fffff700 1 --2- 172.21.15.7:0/1578615872 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc50006fee0 0x7fc5000a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:56.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.231+0000 7fc504ba4700 1 --2- 172.21.15.7:0/1578615872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50006cef0 0x7fc5000ac540 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8014040 tx=0x7fc4f8004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:56.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.231+0000 7fc4fd7fa700 1 -- 172.21.15.7:0/1578615872 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8012430 con 0x7fc50006cef0 2024-09-13T18:53:56.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.231+0000 7fc4fd7fa700 1 -- 172.21.15.7:0/1578615872 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f8020070 con 0x7fc50006cef0 2024-09-13T18:53:56.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.231+0000 7fc505ba6700 1 -- 172.21.15.7:0/1578615872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc500145690 con 0x7fc50006cef0 2024-09-13T18:53:56.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.231+0000 7fc4fd7fa700 1 -- 172.21.15.7:0/1578615872 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8004280 con 0x7fc50006cef0 2024-09-13T18:53:56.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.231+0000 7fc505ba6700 1 -- 172.21.15.7:0/1578615872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc500144a30 con 0x7fc50006cef0 2024-09-13T18:53:56.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.231+0000 7fc505ba6700 1 -- 172.21.15.7:0/1578615872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5000aa4f0 con 0x7fc50006cef0 2024-09-13T18:53:56.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.231+0000 7fc4fd7fa700 1 -- 172.21.15.7:0/1578615872 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc4f80043e0 con 0x7fc50006cef0 2024-09-13T18:53:56.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.231+0000 7fc4fd7fa700 1 --2- 172.21.15.7:0/1578615872 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc4ec066760 0x7fc4ec068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.235+0000 7fc4fd7fa700 1 -- 172.21.15.7:0/1578615872 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc4f8022070 con 0x7fc50006cef0 2024-09-13T18:53:56.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.235+0000 7fc4fffff700 1 --2- 172.21.15.7:0/1578615872 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc4ec066760 0x7fc4ec068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.235+0000 7fc4fffff700 1 --2- 172.21.15.7:0/1578615872 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc4ec066760 0x7fc4ec068c20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc5001f4040 tx=0x7fc4f0007d00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:56.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.239+0000 7fc4fd7fa700 1 -- 172.21.15.7:0/1578615872 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc4f8053770 con 0x7fc50006cef0 2024-09-13T18:53:56.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.455+0000 7fc505ba6700 1 -- 172.21.15.7:0/1578615872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc50006bca0 con 0x7fc50006cef0 2024-09-13T18:53:56.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.455+0000 7fc4fd7fa700 1 -- 172.21.15.7:0/1578615872 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4732 (secure 0 0 0) 0x7fc4f8056d90 con 0x7fc50006cef0 2024-09-13T18:53:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:56.463 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2458672850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2458672850},{"type":"v1","addr":"172.21.15.171:6835","nonce":2458672850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3381042298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3381042298},{"type":"v1","addr":"172.21.15.171:6837","nonce":3381042298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2068823578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2068823578},{"type":"v1","addr":"172.21.15.7:6833","nonce":2068823578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:53:17.897317+0000","modified":"2024-09-13T18:53:30.253515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5463":{"gid":5463,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/127797392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":127797392},{"type":"v1","addr":"172.21.15.7:6835","nonce":127797392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T18:53:56.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.455+0000 7fc505ba6700 1 -- 172.21.15.7:0/1578615872 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc4ec066760 msgr2=0x7fc4ec068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.455+0000 7fc505ba6700 1 --2- 172.21.15.7:0/1578615872 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc4ec066760 0x7fc4ec068c20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc5001f4040 tx=0x7fc4f0007d00 comp rx=0 tx=0).stop 2024-09-13T18:53:56.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.455+0000 7fc505ba6700 1 -- 172.21.15.7:0/1578615872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50006cef0 msgr2=0x7fc5000ac540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.455+0000 7fc505ba6700 1 --2- 172.21.15.7:0/1578615872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50006cef0 0x7fc5000ac540 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8014040 tx=0x7fc4f8004af0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.455+0000 7fc505ba6700 1 -- 172.21.15.7:0/1578615872 shutdown_connections 2024-09-13T18:53:56.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.455+0000 7fc505ba6700 1 --2- 172.21.15.7:0/1578615872 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc4ec066760 0x7fc4ec068c20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.455+0000 7fc505ba6700 1 --2- 172.21.15.7:0/1578615872 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5000b3320 0x7fc5000a8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.455+0000 7fc505ba6700 1 --2- 172.21.15.7:0/1578615872 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc50006fee0 0x7fc5000a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.455+0000 7fc505ba6700 1 --2- 172.21.15.7:0/1578615872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50006cef0 0x7fc5000ac540 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.455+0000 7fc505ba6700 1 -- 172.21.15.7:0/1578615872 >> 172.21.15.7:0/1578615872 conn(0x7fc5000b0570 msgr2=0x7fc50015d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:56.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.455+0000 7fc505ba6700 1 -- 172.21.15.7:0/1578615872 shutdown_connections 2024-09-13T18:53:56.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.455+0000 7fc505ba6700 1 -- 172.21.15.7:0/1578615872 wait complete. 2024-09-13T18:53:56.467 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-09-13T18:53:56.478 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:53:56.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.579+0000 7fb96d919700 1 Processor -- start 2024-09-13T18:53:56.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.579+0000 7fb96d919700 1 -- start start 2024-09-13T18:53:56.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96d919700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb9681597a0 0x7fb9680ad280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96d919700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fb9680aef00 0x7fb9680b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96d919700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9680b2ec0 con 0x7fb9680b3b30 2024-09-13T18:53:56.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96d919700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9680b3f80 con 0x7fb9681597a0 2024-09-13T18:53:56.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96d919700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9680ad7c0 con 0x7fb9680aef00 2024-09-13T18:53:56.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96c917700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb9681597a0 0x7fb9680ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96c917700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb9681597a0 0x7fb9680ad280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33340/0 (socket says 172.21.15.7:33340) 2024-09-13T18:53:56.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96d118700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb9680b3b30 0x7fb968070f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45584/0 (socket says 172.21.15.7:45584) 2024-09-13T18:53:56.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96c917700 1 -- 172.21.15.7:0/2999855016 learned_addr learned my addr 172.21.15.7:0/2999855016 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:56.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb967fff700 1 --2- 172.21.15.7:0/2999855016 >> v2:172.21.15.171:3300/0 conn(0x7fb9680aef00 0x7fb9680b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96c917700 1 -- 172.21.15.7:0/2999855016 >> v2:172.21.15.171:3300/0 conn(0x7fb9680aef00 msgr2=0x7fb9680b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:56.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96c917700 1 --2- 172.21.15.7:0/2999855016 >> v2:172.21.15.171:3300/0 conn(0x7fb9680aef00 0x7fb9680b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96c917700 1 -- 172.21.15.7:0/2999855016 >> v1:172.21.15.7:6789/0 conn(0x7fb9680b3b30 legacy=0x7fb968070f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96c917700 1 -- 172.21.15.7:0/2999855016 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9680ada10 con 0x7fb9681597a0 2024-09-13T18:53:56.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96c917700 1 --2- 172.21.15.7:0/2999855016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9681597a0 0x7fb9680ad280 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb9680d0730 tx=0x7fb95c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4e4ba7295c18d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:56.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb9677fe700 1 -- 172.21.15.7:0/2999855016 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c00b8a0 con 0x7fb9681597a0 2024-09-13T18:53:56.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb9677fe700 1 -- 172.21.15.7:0/2999855016 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95c00ba00 con 0x7fb9681597a0 2024-09-13T18:53:56.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb9677fe700 1 -- 172.21.15.7:0/2999855016 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c00bd90 con 0x7fb9681597a0 2024-09-13T18:53:56.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96d919700 1 -- 172.21.15.7:0/2999855016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9681597a0 msgr2=0x7fb9680ad280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96d919700 1 --2- 172.21.15.7:0/2999855016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9681597a0 0x7fb9680ad280 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb9680d0730 tx=0x7fb95c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96d919700 1 -- 172.21.15.7:0/2999855016 shutdown_connections 2024-09-13T18:53:56.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96d919700 1 --2- 172.21.15.7:0/2999855016 >> v2:172.21.15.171:3300/0 conn(0x7fb9680aef00 0x7fb9680b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96d919700 1 --2- 172.21.15.7:0/2999855016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9681597a0 0x7fb9680ad280 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96d919700 1 -- 172.21.15.7:0/2999855016 >> 172.21.15.7:0/2999855016 conn(0x7fb96806b470 msgr2=0x7fb96806b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:56.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96d919700 1 -- 172.21.15.7:0/2999855016 shutdown_connections 2024-09-13T18:53:56.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.583+0000 7fb96d919700 1 -- 172.21.15.7:0/2999855016 wait complete. 2024-09-13T18:53:56.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96d919700 1 Processor -- start 2024-09-13T18:53:56.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96d919700 1 -- start start 2024-09-13T18:53:56.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96d919700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9680aef00 0x7fb96813a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96d919700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb9680b3b30 0x7fb96815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96d919700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9681597a0 0x7fb968156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96d919700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb968155b90 con 0x7fb9680aef00 2024-09-13T18:53:56.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96d919700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb968155e70 con 0x7fb9681597a0 2024-09-13T18:53:56.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96d919700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb968159f50 con 0x7fb9680b3b30 2024-09-13T18:53:56.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96c917700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9680aef00 0x7fb96813a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96c917700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9680aef00 0x7fb96813a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33350/0 (socket says 172.21.15.7:33350) 2024-09-13T18:53:56.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96c917700 1 -- 172.21.15.7:0/1270039046 learned_addr learned my addr 172.21.15.7:0/1270039046 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:56.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb967fff700 1 --2- 172.21.15.7:0/1270039046 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb9680b3b30 0x7fb96815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96d118700 1 --2- 172.21.15.7:0/1270039046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9681597a0 0x7fb968156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96c917700 1 -- 172.21.15.7:0/1270039046 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb9680b3b30 msgr2=0x7fb96815c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96c917700 1 --2- 172.21.15.7:0/1270039046 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb9680b3b30 0x7fb96815c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96c917700 1 -- 172.21.15.7:0/1270039046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9681597a0 msgr2=0x7fb968156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96c917700 1 --2- 172.21.15.7:0/1270039046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9681597a0 0x7fb968156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96c917700 1 -- 172.21.15.7:0/1270039046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95c009570 con 0x7fb9680aef00 2024-09-13T18:53:56.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96c917700 1 --2- 172.21.15.7:0/1270039046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9680aef00 0x7fb96813a4b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb95c000c00 tx=0x7fb95c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:56.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb9657fa700 1 -- 172.21.15.7:0/1270039046 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c0038c0 con 0x7fb9680aef00 2024-09-13T18:53:56.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb9657fa700 1 -- 172.21.15.7:0/1270039046 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95c003a20 con 0x7fb9680aef00 2024-09-13T18:53:56.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96d919700 1 -- 172.21.15.7:0/1270039046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb96815a230 con 0x7fb9680aef00 2024-09-13T18:53:56.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb9657fa700 1 -- 172.21.15.7:0/1270039046 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c003dd0 con 0x7fb9680aef00 2024-09-13T18:53:56.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.587+0000 7fb96d919700 1 -- 172.21.15.7:0/1270039046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96813a9f0 con 0x7fb9680aef00 2024-09-13T18:53:56.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.591+0000 7fb96d919700 1 -- 172.21.15.7:0/1270039046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9680af5f0 con 0x7fb9680aef00 2024-09-13T18:53:56.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.591+0000 7fb9657fa700 1 -- 172.21.15.7:0/1270039046 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb95c011420 con 0x7fb9680aef00 2024-09-13T18:53:56.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.591+0000 7fb9657fa700 1 --2- 172.21.15.7:0/1270039046 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb94c066730 0x7fb94c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.591+0000 7fb9657fa700 1 -- 172.21.15.7:0/1270039046 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb95c020030 con 0x7fb9680aef00 2024-09-13T18:53:56.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.591+0000 7fb967fff700 1 --2- 172.21.15.7:0/1270039046 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb94c066730 0x7fb94c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.591+0000 7fb967fff700 1 --2- 172.21.15.7:0/1270039046 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb94c066730 0x7fb94c068bf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb96806ed80 tx=0x7fb954008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:56.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.599+0000 7fb9657fa700 1 -- 172.21.15.7:0/1270039046 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb95c052c60 con 0x7fb9680aef00 2024-09-13T18:53:56.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.811+0000 7fb96d919700 1 -- 172.21.15.7:0/1270039046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb968159b80 con 0x7fb9680aef00 2024-09-13T18:53:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.811+0000 7fb9657fa700 1 -- 172.21.15.7:0/1270039046 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4732 (secure 0 0 0) 0x7fb95c056280 con 0x7fb9680aef00 2024-09-13T18:53:56.820 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:56.820 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2458672850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2458672850},{"type":"v1","addr":"172.21.15.171:6835","nonce":2458672850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3381042298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3381042298},{"type":"v1","addr":"172.21.15.171:6837","nonce":3381042298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2068823578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2068823578},{"type":"v1","addr":"172.21.15.7:6833","nonce":2068823578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:53:17.897317+0000","modified":"2024-09-13T18:53:30.253515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":39,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5463":{"gid":5463,"name":"c","rank":0,"incarnation":18,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/127797392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":127797392},{"type":"v1","addr":"172.21.15.7:6835","nonce":127797392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T18:53:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.815+0000 7fb96d919700 1 -- 172.21.15.7:0/1270039046 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb94c066730 msgr2=0x7fb94c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.815+0000 7fb96d919700 1 --2- 172.21.15.7:0/1270039046 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb94c066730 0x7fb94c068bf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb96806ed80 tx=0x7fb954008040 comp rx=0 tx=0).stop 2024-09-13T18:53:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.815+0000 7fb96d919700 1 -- 172.21.15.7:0/1270039046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9680aef00 msgr2=0x7fb96813a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.815+0000 7fb96d919700 1 --2- 172.21.15.7:0/1270039046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9680aef00 0x7fb96813a4b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb95c000c00 tx=0x7fb95c004770 comp rx=0 tx=0).stop 2024-09-13T18:53:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.815+0000 7fb96d919700 1 -- 172.21.15.7:0/1270039046 shutdown_connections 2024-09-13T18:53:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.815+0000 7fb96d919700 1 --2- 172.21.15.7:0/1270039046 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb94c066730 0x7fb94c068bf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.815+0000 7fb96d919700 1 --2- 172.21.15.7:0/1270039046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb9681597a0 0x7fb968156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.815+0000 7fb96d919700 1 --2- 172.21.15.7:0/1270039046 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb9680b3b30 0x7fb96815c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.815+0000 7fb96d919700 1 --2- 172.21.15.7:0/1270039046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9680aef00 0x7fb96813a4b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.815+0000 7fb96d919700 1 -- 172.21.15.7:0/1270039046 >> 172.21.15.7:0/1270039046 conn(0x7fb96806b470 msgr2=0x7fb96806ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.815+0000 7fb96d919700 1 -- 172.21.15.7:0/1270039046 shutdown_connections 2024-09-13T18:53:56.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.815+0000 7fb96d919700 1 -- 172.21.15.7:0/1270039046 wait complete. 2024-09-13T18:53:56.824 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2024-09-13T18:53:56.835 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:53:56.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.943+0000 7fe1851ed700 1 Processor -- start 2024-09-13T18:53:56.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.943+0000 7fe1851ed700 1 -- start start 2024-09-13T18:53:56.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.943+0000 7fe1851ed700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fe1800ade50 0x7fe180163ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.943+0000 7fe1851ed700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe18006cbb0 con 0x7fe180158b30 2024-09-13T18:53:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.943+0000 7fe1851ed700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe18006ce90 con 0x7fe18006c6b0 2024-09-13T18:53:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.943+0000 7fe1851ed700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe18006d170 con 0x7fe1800ade50 2024-09-13T18:53:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.943+0000 7fe17ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe180158b30 0x7fe180070160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45594/0 (socket says 172.21.15.7:45594) 2024-09-13T18:53:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.943+0000 7fe17ffff700 1 -- 172.21.15.7:0/702587632 learned_addr learned my addr 172.21.15.7:0/702587632 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:56.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe1849ec700 1 --2- 172.21.15.7:0/702587632 >> v2:172.21.15.171:3300/0 conn(0x7fe1800ade50 0x7fe180163ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 -- 172.21.15.7:0/702587632 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224326814 0 0) 0x7fe18006cbb0 con 0x7fe180158b30 2024-09-13T18:53:56.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 -- 172.21.15.7:0/702587632 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1680034f0 con 0x7fe180158b30 2024-09-13T18:53:56.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 -- 172.21.15.7:0/702587632 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107000694 0 0) 0x7fe18006ce90 con 0x7fe18006c6b0 2024-09-13T18:53:56.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 -- 172.21.15.7:0/702587632 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe18006cbb0 con 0x7fe18006c6b0 2024-09-13T18:53:56.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 -- 172.21.15.7:0/702587632 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3703991492 0 0) 0x7fe1680034f0 con 0x7fe180158b30 2024-09-13T18:53:56.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 -- 172.21.15.7:0/702587632 >> v2:172.21.15.171:3300/0 conn(0x7fe1800ade50 msgr2=0x7fe180163ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:56.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 --2- 172.21.15.7:0/702587632 >> v2:172.21.15.171:3300/0 conn(0x7fe1800ade50 0x7fe180163ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 -- 172.21.15.7:0/702587632 >> v1:172.21.15.171:6789/0 conn(0x7fe18006c6b0 legacy=0x7fe1800ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 -- 172.21.15.7:0/702587632 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18006d450 con 0x7fe180158b30 2024-09-13T18:53:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 -- 172.21.15.7:0/702587632 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7fe178005100 con 0x7fe180158b30 2024-09-13T18:53:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe1849ec700 1 --2- 172.21.15.7:0/702587632 >> v2:172.21.15.171:3300/0 conn(0x7fe1800ade50 0x7fe180163ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 -- 172.21.15.7:0/702587632 >> v1:172.21.15.7:6789/0 conn(0x7fe180158b30 legacy=0x7fe180070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 --2- 172.21.15.7:0/702587632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe168003ca0 0x7fe1680040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 --2- 172.21.15.7:0/702587632 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe168008a10 0x7fe16800aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 --2- 172.21.15.7:0/702587632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16800b510 0x7fe16800d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 -- 172.21.15.7:0/702587632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18006d170 con 0x7fe16800b510 2024-09-13T18:53:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 -- 172.21.15.7:0/702587632 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16800e2b0 con 0x7fe168003ca0 2024-09-13T18:53:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 -- 172.21.15.7:0/702587632 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16800e470 con 0x7fe168008a10 2024-09-13T18:53:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe1849ec700 1 --2- 172.21.15.7:0/702587632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16800b510 0x7fe16800d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17ffff700 1 --2- 172.21.15.7:0/702587632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe168003ca0 0x7fe1680040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17f7fe700 1 --2- 172.21.15.7:0/702587632 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe168008a10 0x7fe16800aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe1849ec700 1 -- 172.21.15.7:0/702587632 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe168008a10 msgr2=0x7fe16800aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:56.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe1849ec700 1 --2- 172.21.15.7:0/702587632 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe168008a10 0x7fe16800aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe1849ec700 1 -- 172.21.15.7:0/702587632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe168003ca0 msgr2=0x7fe1680040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:56.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe1849ec700 1 --2- 172.21.15.7:0/702587632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe168003ca0 0x7fe1680040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe1849ec700 1 -- 172.21.15.7:0/702587632 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1680034f0 con 0x7fe16800b510 2024-09-13T18:53:56.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17ffff700 1 --2- 172.21.15.7:0/702587632 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe168003ca0 0x7fe1680040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:56.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe1849ec700 1 -- 172.21.15.7:0/702587632 reap_dead start 2024-09-13T18:53:56.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe1849ec700 1 --2- 172.21.15.7:0/702587632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16800b510 0x7fe16800d9d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe18006fa60 tx=0x7fe1680037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328a7902f7f43b02 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:56.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 -- 172.21.15.7:0/702587632 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17000e7e0 con 0x7fe16800b510 2024-09-13T18:53:56.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 -- 172.21.15.7:0/702587632 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17000e940 con 0x7fe16800b510 2024-09-13T18:53:56.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe17effd700 1 -- 172.21.15.7:0/702587632 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17000ecf0 con 0x7fe16800b510 2024-09-13T18:53:56.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe1851ed700 1 -- 172.21.15.7:0/702587632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16800b510 msgr2=0x7fe16800d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe1851ed700 1 --2- 172.21.15.7:0/702587632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16800b510 0x7fe16800d9d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe18006fa60 tx=0x7fe1680037a0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe1851ed700 1 -- 172.21.15.7:0/702587632 shutdown_connections 2024-09-13T18:53:56.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe1851ed700 1 --2- 172.21.15.7:0/702587632 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16800b510 0x7fe16800d9d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe1851ed700 1 -- 172.21.15.7:0/702587632 >> 172.21.15.7:0/702587632 conn(0x7fe1800b0570 msgr2=0x7fe1800b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:56.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe1851ed700 1 -- 172.21.15.7:0/702587632 shutdown_connections 2024-09-13T18:53:56.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.947+0000 7fe1851ed700 1 -- 172.21.15.7:0/702587632 wait complete. 2024-09-13T18:53:56.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe1851ed700 1 Processor -- start 2024-09-13T18:53:56.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe1851ed700 1 -- start start 2024-09-13T18:53:56.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe1851ed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16800b510 0x7fe180153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe1851ed700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe180153a70 0x7fe1800ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe1851ed700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1801e3490 0x7fe1801e3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe1851ed700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1801e3e50 con 0x7fe16800b510 2024-09-13T18:53:56.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe1851ed700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1801542f0 con 0x7fe1801e3490 2024-09-13T18:53:56.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe1851ed700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1800ce310 con 0x7fe180153a70 2024-09-13T18:53:56.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe17ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16800b510 0x7fe180153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe17ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16800b510 0x7fe180153450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33366/0 (socket says 172.21.15.7:33366) 2024-09-13T18:53:56.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe17ffff700 1 -- 172.21.15.7:0/1151899047 learned_addr learned my addr 172.21.15.7:0/1151899047 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:56.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe17f7fe700 1 --2- 172.21.15.7:0/1151899047 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe180153a70 0x7fe1800ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe1849ec700 1 --2- 172.21.15.7:0/1151899047 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1801e3490 0x7fe1801e3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe17ffff700 1 -- 172.21.15.7:0/1151899047 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe180153a70 msgr2=0x7fe1800ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe17ffff700 1 --2- 172.21.15.7:0/1151899047 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe180153a70 0x7fe1800ad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe17ffff700 1 -- 172.21.15.7:0/1151899047 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1801e3490 msgr2=0x7fe1801e3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:56.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe17ffff700 1 --2- 172.21.15.7:0/1151899047 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1801e3490 0x7fe1801e3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:56.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe17ffff700 1 -- 172.21.15.7:0/1151899047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1800ad560 con 0x7fe16800b510 2024-09-13T18:53:56.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe17f7fe700 1 --2- 172.21.15.7:0/1151899047 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe180153a70 0x7fe1800ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:56.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe17ffff700 1 --2- 172.21.15.7:0/1151899047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16800b510 0x7fe180153450 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe178012bd0 tx=0x7fe178012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:56.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe17cff9700 1 -- 172.21.15.7:0/1151899047 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178017070 con 0x7fe16800b510 2024-09-13T18:53:56.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe17cff9700 1 -- 172.21.15.7:0/1151899047 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe178005d00 con 0x7fe16800b510 2024-09-13T18:53:56.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe1851ed700 1 -- 172.21.15.7:0/1151899047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1800cdd50 con 0x7fe16800b510 2024-09-13T18:53:56.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe17cff9700 1 -- 172.21.15.7:0/1151899047 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17801d620 con 0x7fe16800b510 2024-09-13T18:53:56.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.951+0000 7fe1851ed700 1 -- 172.21.15.7:0/1151899047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1801e1590 con 0x7fe16800b510 2024-09-13T18:53:56.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.955+0000 7fe1851ed700 1 -- 172.21.15.7:0/1151899047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1801e13f0 con 0x7fe16800b510 2024-09-13T18:53:56.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.955+0000 7fe17cff9700 1 -- 172.21.15.7:0/1151899047 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fe17801a020 con 0x7fe16800b510 2024-09-13T18:53:56.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.955+0000 7fe17cff9700 1 --2- 172.21.15.7:0/1151899047 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe16c066730 0x7fe16c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:56.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.955+0000 7fe17cff9700 1 -- 172.21.15.7:0/1151899047 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe178087cf0 con 0x7fe16800b510 2024-09-13T18:53:56.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.955+0000 7fe17f7fe700 1 --2- 172.21.15.7:0/1151899047 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe16c066730 0x7fe16c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:56.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.955+0000 7fe17f7fe700 1 --2- 172.21.15.7:0/1151899047 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe16c066730 0x7fe16c068bf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe1800cdbc0 tx=0x7fe174009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:56.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:56.963+0000 7fe17cff9700 1 -- 172.21.15.7:0/1151899047 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe1801e13f0 con 0x7fe16800b510 2024-09-13T18:53:57.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.139+0000 7fe1851ed700 1 -- 172.21.15.7:0/1151899047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1801e13f0 con 0x7fe16800b510 2024-09-13T18:53:57.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.143+0000 7fe17cff9700 1 -- 172.21.15.7:0/1151899047 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16648 (secure 0 0 0) 0x7fe1801e13f0 con 0x7fe16800b510 2024-09-13T18:53:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:53:57.152 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:53:26.238445+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:53:15.862087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T18:53:16.885280+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/2843879273":"2024-09-14T18:53:26.228249+0000","172.21.15.7:6834/2843879273":"2024-09-14T18:53:26.228249+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:53:57.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.147+0000 7fe1851ed700 1 -- 172.21.15.7:0/1151899047 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe16c066730 msgr2=0x7fe16c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:57.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.147+0000 7fe1851ed700 1 --2- 172.21.15.7:0/1151899047 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe16c066730 0x7fe16c068bf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe1800cdbc0 tx=0x7fe174009040 comp rx=0 tx=0).stop 2024-09-13T18:53:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.147+0000 7fe1851ed700 1 -- 172.21.15.7:0/1151899047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16800b510 msgr2=0x7fe180153450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.147+0000 7fe1851ed700 1 --2- 172.21.15.7:0/1151899047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16800b510 0x7fe180153450 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe178012bd0 tx=0x7fe178012f90 comp rx=0 tx=0).stop 2024-09-13T18:53:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.147+0000 7fe1851ed700 1 -- 172.21.15.7:0/1151899047 shutdown_connections 2024-09-13T18:53:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.147+0000 7fe1851ed700 1 --2- 172.21.15.7:0/1151899047 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fe16c066730 0x7fe16c068bf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.147+0000 7fe1851ed700 1 --2- 172.21.15.7:0/1151899047 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe1801e3490 0x7fe1801e3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.147+0000 7fe1851ed700 1 --2- 172.21.15.7:0/1151899047 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fe180153a70 0x7fe1800ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.147+0000 7fe1851ed700 1 --2- 172.21.15.7:0/1151899047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe16800b510 0x7fe180153450 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.147+0000 7fe1851ed700 1 -- 172.21.15.7:0/1151899047 >> 172.21.15.7:0/1151899047 conn(0x7fe1800b0570 msgr2=0x7fe1800ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.147+0000 7fe1851ed700 1 -- 172.21.15.7:0/1151899047 shutdown_connections 2024-09-13T18:53:57.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.147+0000 7fe1851ed700 1 -- 172.21.15.7:0/1151899047 wait complete. 2024-09-13T18:53:57.173 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T18:53:57.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.283+0000 7fbb3f59e700 1 Processor -- start 2024-09-13T18:53:57.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.283+0000 7fbb3f59e700 1 -- start start 2024-09-13T18:53:57.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.283+0000 7fbb3f59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb40160ff0 0x7fbb400ada40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:57.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.283+0000 7fbb3f59e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fbb40070020 0x7fbb4006c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:57.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.283+0000 7fbb3f59e700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb400b3b40 con 0x7fbb4006cbf0 2024-09-13T18:53:57.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.283+0000 7fbb3f59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb40070480 con 0x7fbb40160ff0 2024-09-13T18:53:57.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.283+0000 7fbb3f59e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb400b3860 con 0x7fbb40070020 2024-09-13T18:53:57.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.283+0000 7fbb3e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb40160ff0 0x7fbb400ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:57.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.283+0000 7fbb3e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb40160ff0 0x7fbb400ada40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33368/0 (socket says 172.21.15.7:33368) 2024-09-13T18:53:57.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.283+0000 7fbb3e59c700 1 -- 172.21.15.7:0/2095388147 learned_addr learned my addr 172.21.15.7:0/2095388147 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:57.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.283+0000 7fbb3dd9b700 1 --2- 172.21.15.7:0/2095388147 >> v2:172.21.15.171:3300/0 conn(0x7fbb40070020 0x7fbb4006c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:57.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.283+0000 7fbb3e59c700 1 -- 172.21.15.7:0/2095388147 >> v2:172.21.15.171:3300/0 conn(0x7fbb40070020 msgr2=0x7fbb4006c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:57.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.283+0000 7fbb3e59c700 1 --2- 172.21.15.7:0/2095388147 >> v2:172.21.15.171:3300/0 conn(0x7fbb40070020 0x7fbb4006c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.283+0000 7fbb3e59c700 1 -- 172.21.15.7:0/2095388147 >> v1:172.21.15.171:6789/0 conn(0x7fbb4006cbf0 legacy=0x7fbb400b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.283+0000 7fbb3e59c700 1 -- 172.21.15.7:0/2095388147 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb40071a20 con 0x7fbb40160ff0 2024-09-13T18:53:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.287+0000 7fbb3e59c700 1 --2- 172.21.15.7:0/2095388147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb40160ff0 0x7fbb400ada40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbb40164a80 tx=0x7fbb34009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb7c2c25fcbaa1c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.287+0000 7fbb3d59a700 1 -- 172.21.15.7:0/2095388147 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb34011830 con 0x7fbb40160ff0 2024-09-13T18:53:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.287+0000 7fbb3d59a700 1 -- 172.21.15.7:0/2095388147 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb34011990 con 0x7fbb40160ff0 2024-09-13T18:53:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.287+0000 7fbb3d59a700 1 -- 172.21.15.7:0/2095388147 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb34011d20 con 0x7fbb40160ff0 2024-09-13T18:53:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.287+0000 7fbb3f59e700 1 -- 172.21.15.7:0/2095388147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb40160ff0 msgr2=0x7fbb400ada40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:57.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.287+0000 7fbb3f59e700 1 --2- 172.21.15.7:0/2095388147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb40160ff0 0x7fbb400ada40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbb40164a80 tx=0x7fbb34009bf0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.287+0000 7fbb3f59e700 1 -- 172.21.15.7:0/2095388147 shutdown_connections 2024-09-13T18:53:57.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.287+0000 7fbb3f59e700 1 --2- 172.21.15.7:0/2095388147 >> v2:172.21.15.171:3300/0 conn(0x7fbb40070020 0x7fbb4006c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.287+0000 7fbb3f59e700 1 --2- 172.21.15.7:0/2095388147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb40160ff0 0x7fbb400ada40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.287+0000 7fbb3f59e700 1 -- 172.21.15.7:0/2095388147 >> 172.21.15.7:0/2095388147 conn(0x7fbb400b0570 msgr2=0x7fbb400b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:57.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.287+0000 7fbb3f59e700 1 -- 172.21.15.7:0/2095388147 shutdown_connections 2024-09-13T18:53:57.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.287+0000 7fbb3f59e700 1 -- 172.21.15.7:0/2095388147 wait complete. 2024-09-13T18:53:57.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.291+0000 7fbb3f59e700 1 Processor -- start 2024-09-13T18:53:57.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.291+0000 7fbb3f59e700 1 -- start start 2024-09-13T18:53:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.291+0000 7fbb3f59e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb4006cbf0 0x7fbb400ab100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.291+0000 7fbb3f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb40070020 0x7fbb401f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.291+0000 7fbb3f59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb40160ff0 0x7fbb4015bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.291+0000 7fbb3f59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4015cc60 con 0x7fbb40070020 2024-09-13T18:53:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.291+0000 7fbb3f59e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb400ab790 con 0x7fbb4006cbf0 2024-09-13T18:53:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.291+0000 7fbb3dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb40070020 0x7fbb401f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.291+0000 7fbb3f59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb400aba70 con 0x7fbb40160ff0 2024-09-13T18:53:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.291+0000 7fbb3dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb40070020 0x7fbb401f6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33380/0 (socket says 172.21.15.7:33380) 2024-09-13T18:53:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.291+0000 7fbb3dd9b700 1 -- 172.21.15.7:0/1344174670 learned_addr learned my addr 172.21.15.7:0/1344174670 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.291+0000 7fbb3e59c700 1 --2- 172.21.15.7:0/1344174670 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb4006cbf0 0x7fbb400ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.291+0000 7fbb3ed9d700 1 --2- 172.21.15.7:0/1344174670 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb40160ff0 0x7fbb4015bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.291+0000 7fbb3dd9b700 1 -- 172.21.15.7:0/1344174670 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb40160ff0 msgr2=0x7fbb4015bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.291+0000 7fbb3dd9b700 1 --2- 172.21.15.7:0/1344174670 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb40160ff0 0x7fbb4015bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.291+0000 7fbb3dd9b700 1 -- 172.21.15.7:0/1344174670 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb4006cbf0 msgr2=0x7fbb400ab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.291+0000 7fbb3dd9b700 1 --2- 172.21.15.7:0/1344174670 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb4006cbf0 0x7fbb400ab100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.291+0000 7fbb3dd9b700 1 -- 172.21.15.7:0/1344174670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb34009570 con 0x7fbb40070020 2024-09-13T18:53:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.295+0000 7fbb3dd9b700 1 --2- 172.21.15.7:0/1344174670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb40070020 0x7fbb401f6e70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000efc0 tx=0x7fbb3000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.295+0000 7fbb2effd700 1 -- 172.21.15.7:0/1344174670 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3000cdc0 con 0x7fbb40070020 2024-09-13T18:53:57.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.295+0000 7fbb2effd700 1 -- 172.21.15.7:0/1344174670 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3000f040 con 0x7fbb40070020 2024-09-13T18:53:57.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.295+0000 7fbb3f59e700 1 -- 172.21.15.7:0/1344174670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb400abd50 con 0x7fbb40070020 2024-09-13T18:53:57.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.295+0000 7fbb2effd700 1 -- 172.21.15.7:0/1344174670 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb300146a0 con 0x7fbb40070020 2024-09-13T18:53:57.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.295+0000 7fbb3f59e700 1 -- 172.21.15.7:0/1344174670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb400ac310 con 0x7fbb40070020 2024-09-13T18:53:57.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.295+0000 7fbb3f59e700 1 -- 172.21.15.7:0/1344174670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb4006d9e0 con 0x7fbb40070020 2024-09-13T18:53:57.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.295+0000 7fbb2effd700 1 -- 172.21.15.7:0/1344174670 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbb30014800 con 0x7fbb40070020 2024-09-13T18:53:57.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.295+0000 7fbb2effd700 1 --2- 172.21.15.7:0/1344174670 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbb20066760 0x7fbb20068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:57.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.295+0000 7fbb2effd700 1 -- 172.21.15.7:0/1344174670 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbb30084b30 con 0x7fbb40070020 2024-09-13T18:53:57.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.295+0000 7fbb3e59c700 1 --2- 172.21.15.7:0/1344174670 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbb20066760 0x7fbb20068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:57.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.299+0000 7fbb3e59c700 1 --2- 172.21.15.7:0/1344174670 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbb20066760 0x7fbb20068c20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbb34014040 tx=0x7fbb34008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.299+0000 7fbb2effd700 1 -- 172.21.15.7:0/1344174670 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbb30051d30 con 0x7fbb40070020 2024-09-13T18:53:57.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.511+0000 7fbb3f59e700 1 -- 172.21.15.7:0/1344174670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbb400adae0 con 0x7fbb40070020 2024-09-13T18:53:57.521 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-13T18:53:57.511+0000 7f28f0ff8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T18:53:57.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.527+0000 7fbb2effd700 1 -- 172.21.15.7:0/1344174670 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbb30055350 con 0x7fbb40070020 2024-09-13T18:53:57.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.531+0000 7fbb3f59e700 1 -- 172.21.15.7:0/1344174670 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbb20066760 msgr2=0x7fbb20068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:57.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.531+0000 7fbb3f59e700 1 --2- 172.21.15.7:0/1344174670 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbb20066760 0x7fbb20068c20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbb34014040 tx=0x7fbb34008030 comp rx=0 tx=0).stop 2024-09-13T18:53:57.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.531+0000 7fbb3f59e700 1 -- 172.21.15.7:0/1344174670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb40070020 msgr2=0x7fbb401f6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:57.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.531+0000 7fbb3f59e700 1 --2- 172.21.15.7:0/1344174670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb40070020 0x7fbb401f6e70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbb3000efc0 tx=0x7fbb3000c3a0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.531+0000 7fbb3f59e700 1 -- 172.21.15.7:0/1344174670 shutdown_connections 2024-09-13T18:53:57.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.531+0000 7fbb3f59e700 1 --2- 172.21.15.7:0/1344174670 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbb20066760 0x7fbb20068c20 secure :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbb34014040 tx=0x7fbb34008030 comp rx=0 tx=0).stop 2024-09-13T18:53:57.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.531+0000 7fbb3f59e700 1 --2- 172.21.15.7:0/1344174670 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb40160ff0 0x7fbb4015bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.531+0000 7fbb3f59e700 1 --2- 172.21.15.7:0/1344174670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb40070020 0x7fbb401f6e70 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.531+0000 7fbb3f59e700 1 --2- 172.21.15.7:0/1344174670 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb4006cbf0 0x7fbb400ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.531+0000 7fbb3f59e700 1 -- 172.21.15.7:0/1344174670 >> 172.21.15.7:0/1344174670 conn(0x7fbb400b0570 msgr2=0x7fbb400af5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:57.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.531+0000 7fbb3f59e700 1 -- 172.21.15.7:0/1344174670 shutdown_connections 2024-09-13T18:53:57.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.531+0000 7fbb3f59e700 1 -- 172.21.15.7:0/1344174670 wait complete. 2024-09-13T18:53:57.540 INFO:tasks.ceph.mds.c.smithi007.stderr: -9895> 2024-09-13T18:53:31.244+0000 7fa8b0cc3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T18:53:57.540 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T18:53:57.552 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T18:53:57.575 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-13T18:53:57.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.659+0000 7f883e0ec700 1 Processor -- start 2024-09-13T18:53:57.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.659+0000 7f883e0ec700 1 -- start start 2024-09-13T18:53:57.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.659+0000 7f883e0ec700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f883815dc80 0x7f883806d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:57.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.659+0000 7f883e0ec700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f883806dd40 0x7f88380b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:57.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883e0ec700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f883815e3b0 con 0x7f8838157530 2024-09-13T18:53:57.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883e0ec700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88380b2b00 con 0x7f883806dd40 2024-09-13T18:53:57.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883e0ec700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883806e1f0 con 0x7f883815dc80 2024-09-13T18:53:57.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883c8e9700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f883815dc80 0x7f883806d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:57.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883d8eb700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f883806dd40 0x7f88380b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:57.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883c8e9700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f883815dc80 0x7f883806d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:44750/0 (socket says 172.21.15.7:44750) 2024-09-13T18:53:57.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883d0ea700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f8838157530 0x7f883815d570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:57322/0 (socket says 172.21.15.7:57322) 2024-09-13T18:53:57.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883c8e9700 1 -- 172.21.15.7:0/3036159188 learned_addr learned my addr 172.21.15.7:0/3036159188 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:57.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f8837fff700 1 -- 172.21.15.7:0/3036159188 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295184630 0 0) 0x7f883815e3b0 con 0x7f8838157530 2024-09-13T18:53:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f8837fff700 1 -- 172.21.15.7:0/3036159188 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88200035d0 con 0x7f8838157530 2024-09-13T18:53:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883c8e9700 1 -- 172.21.15.7:0/3036159188 >> v2:172.21.15.171:3300/0 conn(0x7f883806dd40 msgr2=0x7f88380b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883c8e9700 1 --2- 172.21.15.7:0/3036159188 >> v2:172.21.15.171:3300/0 conn(0x7f883806dd40 0x7f88380b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883c8e9700 1 -- 172.21.15.7:0/3036159188 >> v1:172.21.15.171:6789/0 conn(0x7f8838157530 legacy=0x7f883815d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883c8e9700 1 -- 172.21.15.7:0/3036159188 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883806ec80 con 0x7f883815dc80 2024-09-13T18:53:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883c8e9700 1 --2- 172.21.15.7:0/3036159188 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f883815dc80 0x7f883806d800 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f882c009e80 tx=0x7f882c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daefc0d8f9003aca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f8837fff700 1 -- 172.21.15.7:0/3036159188 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c00e070 con 0x7f883815dc80 2024-09-13T18:53:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f8837fff700 1 -- 172.21.15.7:0/3036159188 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882c00b8d0 con 0x7f883815dc80 2024-09-13T18:53:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f8837fff700 1 -- 172.21.15.7:0/3036159188 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c00bc80 con 0x7f883815dc80 2024-09-13T18:53:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883e0ec700 1 -- 172.21.15.7:0/3036159188 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f883815dc80 msgr2=0x7f883806d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883e0ec700 1 --2- 172.21.15.7:0/3036159188 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f883815dc80 0x7f883806d800 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f882c009e80 tx=0x7f882c00b330 comp rx=0 tx=0).stop 2024-09-13T18:53:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883e0ec700 1 -- 172.21.15.7:0/3036159188 shutdown_connections 2024-09-13T18:53:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883e0ec700 1 --2- 172.21.15.7:0/3036159188 >> v2:172.21.15.171:3300/0 conn(0x7f883806dd40 0x7f88380b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883e0ec700 1 --2- 172.21.15.7:0/3036159188 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f883815dc80 0x7f883806d800 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883e0ec700 1 -- 172.21.15.7:0/3036159188 >> 172.21.15.7:0/3036159188 conn(0x7f88380cf9b0 msgr2=0x7f88380cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883e0ec700 1 -- 172.21.15.7:0/3036159188 shutdown_connections 2024-09-13T18:53:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.663+0000 7f883e0ec700 1 -- 172.21.15.7:0/3036159188 wait complete. 2024-09-13T18:53:57.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883e0ec700 1 Processor -- start 2024-09-13T18:53:57.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883e0ec700 1 -- start start 2024-09-13T18:53:57.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883e0ec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f883806dd40 0x7f883814dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:57.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883e0ec700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8838157530 0x7f88380b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:57.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883e0ec700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f883815dc80 0x7f88380b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:57.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883e0ec700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88380cdd80 con 0x7f883806dd40 2024-09-13T18:53:57.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883e0ec700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88380cd100 con 0x7f883815dc80 2024-09-13T18:53:57.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883e0ec700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88380cd3e0 con 0x7f8838157530 2024-09-13T18:53:57.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883d0ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f883806dd40 0x7f883814dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883d0ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f883806dd40 0x7f883814dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33400/0 (socket says 172.21.15.7:33400) 2024-09-13T18:53:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883d0ea700 1 -- 172.21.15.7:0/1865734760 learned_addr learned my addr 172.21.15.7:0/1865734760 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883d8eb700 1 --2- 172.21.15.7:0/1865734760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f883815dc80 0x7f88380b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883c8e9700 1 --2- 172.21.15.7:0/1865734760 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8838157530 0x7f88380b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883d0ea700 1 -- 172.21.15.7:0/1865734760 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8838157530 msgr2=0x7f88380b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883d0ea700 1 --2- 172.21.15.7:0/1865734760 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8838157530 0x7f88380b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883d0ea700 1 -- 172.21.15.7:0/1865734760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f883815dc80 msgr2=0x7f88380b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883d0ea700 1 --2- 172.21.15.7:0/1865734760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f883815dc80 0x7f88380b44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883d0ea700 1 -- 172.21.15.7:0/1865734760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c009b30 con 0x7f883806dd40 2024-09-13T18:53:57.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883d0ea700 1 --2- 172.21.15.7:0/1865734760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f883806dd40 0x7f883814dfc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f882800c4d0 tx=0x7f882800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:57.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f8835ffb700 1 -- 172.21.15.7:0/1865734760 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828005cd0 con 0x7f883806dd40 2024-09-13T18:53:57.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f8835ffb700 1 -- 172.21.15.7:0/1865734760 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8828005e30 con 0x7f883806dd40 2024-09-13T18:53:57.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f883e0ec700 1 -- 172.21.15.7:0/1865734760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88380cd6c0 con 0x7f883806dd40 2024-09-13T18:53:57.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.667+0000 7f8835ffb700 1 -- 172.21.15.7:0/1865734760 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88280072f0 con 0x7f883806dd40 2024-09-13T18:53:57.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.671+0000 7f883e0ec700 1 -- 172.21.15.7:0/1865734760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883814e500 con 0x7f883806dd40 2024-09-13T18:53:57.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.671+0000 7f883e0ec700 1 -- 172.21.15.7:0/1865734760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8838067830 con 0x7f883806dd40 2024-09-13T18:53:57.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.671+0000 7f8835ffb700 1 -- 172.21.15.7:0/1865734760 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8828007450 con 0x7f883806dd40 2024-09-13T18:53:57.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.671+0000 7f8835ffb700 1 --2- 172.21.15.7:0/1865734760 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8824066730 0x7f8824068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:57.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.671+0000 7f8835ffb700 1 -- 172.21.15.7:0/1865734760 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f88280836c0 con 0x7f883806dd40 2024-09-13T18:53:57.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.671+0000 7f883c8e9700 1 --2- 172.21.15.7:0/1865734760 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8824066730 0x7f8824068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:57.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.671+0000 7f883c8e9700 1 --2- 172.21.15.7:0/1865734760 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8824066730 0x7f8824068bf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8838070760 tx=0x7f882c009000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:57.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.679+0000 7f8835ffb700 1 -- 172.21.15.7:0/1865734760 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8828050900 con 0x7f883806dd40 2024-09-13T18:53:57.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:57.887+0000 7f883e0ec700 1 -- 172.21.15.7:0/1865734760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f883806d800 con 0x7f883806dd40 2024-09-13T18:53:58.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.531+0000 7f8835ffb700 1 -- 172.21.15.7:0/1865734760 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8828053f20 con 0x7f883806dd40 2024-09-13T18:53:58.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.535+0000 7f883e0ec700 1 -- 172.21.15.7:0/1865734760 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8824066730 msgr2=0x7f8824068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:58.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.535+0000 7f883e0ec700 1 --2- 172.21.15.7:0/1865734760 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8824066730 0x7f8824068bf0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8838070760 tx=0x7f882c009000 comp rx=0 tx=0).stop 2024-09-13T18:53:58.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.535+0000 7f883e0ec700 1 -- 172.21.15.7:0/1865734760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f883806dd40 msgr2=0x7f883814dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.535+0000 7f883e0ec700 1 --2- 172.21.15.7:0/1865734760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f883806dd40 0x7f883814dfc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f882800c4d0 tx=0x7f882800c7e0 comp rx=0 tx=0).stop 2024-09-13T18:53:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.535+0000 7f883e0ec700 1 -- 172.21.15.7:0/1865734760 shutdown_connections 2024-09-13T18:53:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.535+0000 7f883e0ec700 1 --2- 172.21.15.7:0/1865734760 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8824066730 0x7f8824068bf0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.535+0000 7f883e0ec700 1 --2- 172.21.15.7:0/1865734760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f883815dc80 0x7f88380b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.535+0000 7f883e0ec700 1 --2- 172.21.15.7:0/1865734760 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8838157530 0x7f88380b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.535+0000 7f883e0ec700 1 --2- 172.21.15.7:0/1865734760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f883806dd40 0x7f883814dfc0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.535+0000 7f883e0ec700 1 -- 172.21.15.7:0/1865734760 >> 172.21.15.7:0/1865734760 conn(0x7f88380cf9b0 msgr2=0x7f8838157d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.535+0000 7f883e0ec700 1 -- 172.21.15.7:0/1865734760 shutdown_connections 2024-09-13T18:53:58.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.535+0000 7f883e0ec700 1 -- 172.21.15.7:0/1865734760 wait complete. 2024-09-13T18:53:58.561 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T18:53:58.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.675+0000 7fc85a338700 1 Processor -- start 2024-09-13T18:53:58.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.675+0000 7fc85a338700 1 -- start start 2024-09-13T18:53:58.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc85a338700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc84c005e10 0x7fc84c0575b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:58.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc85a338700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc84c053d60 con 0x7fc84c007200 2024-09-13T18:53:58.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc85a338700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc84c03bb50 con 0x7fc84c057af0 2024-09-13T18:53:58.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc85a338700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c03c0f0 con 0x7fc84c005e10 2024-09-13T18:53:58.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc859b37700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc84c007200 0x7fc84c03cc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45598/0 (socket says 172.21.15.7:45598) 2024-09-13T18:53:58.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc859b37700 1 -- 172.21.15.7:0/1502129647 learned_addr learned my addr 172.21.15.7:0/1502129647 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:58.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc859336700 1 --2- 172.21.15.7:0/1502129647 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc84c005e10 0x7fc84c0575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc847fff700 1 -- 172.21.15.7:0/1502129647 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216362112 0 0) 0x7fc84c053d60 con 0x7fc84c007200 2024-09-13T18:53:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc847fff700 1 -- 172.21.15.7:0/1502129647 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8400035d0 con 0x7fc84c007200 2024-09-13T18:53:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc847fff700 1 -- 172.21.15.7:0/1502129647 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791726548 0 0) 0x7fc84c03bb50 con 0x7fc84c057af0 2024-09-13T18:53:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc847fff700 1 -- 172.21.15.7:0/1502129647 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc84c053d60 con 0x7fc84c057af0 2024-09-13T18:53:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc847fff700 1 -- 172.21.15.7:0/1502129647 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3606797445 0 0) 0x7fc8400035d0 con 0x7fc84c007200 2024-09-13T18:53:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc859336700 1 -- 172.21.15.7:0/1502129647 >> v1:172.21.15.171:6789/0 conn(0x7fc84c057af0 legacy=0x7fc84c03c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc859336700 1 -- 172.21.15.7:0/1502129647 >> v1:172.21.15.7:6789/0 conn(0x7fc84c007200 legacy=0x7fc84c03cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc859336700 1 -- 172.21.15.7:0/1502129647 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c057f30 con 0x7fc84c005e10 2024-09-13T18:53:58.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc859336700 1 --2- 172.21.15.7:0/1502129647 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc84c005e10 0x7fc84c0575b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc850009880 tx=0x7fc850009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c9c69104f352a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:58.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc847fff700 1 -- 172.21.15.7:0/1502129647 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8540681e0 con 0x7fc84c005e10 2024-09-13T18:53:58.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc847fff700 1 -- 172.21.15.7:0/1502129647 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc850004030 con 0x7fc84c005e10 2024-09-13T18:53:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.679+0000 7fc847fff700 1 -- 172.21.15.7:0/1502129647 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8500043a0 con 0x7fc84c005e10 2024-09-13T18:53:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.683+0000 7fc85a338700 1 -- 172.21.15.7:0/1502129647 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc84c005e10 msgr2=0x7fc84c0575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.683+0000 7fc85a338700 1 --2- 172.21.15.7:0/1502129647 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc84c005e10 0x7fc84c0575b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc850009880 tx=0x7fc850009b90 comp rx=0 tx=0).stop 2024-09-13T18:53:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.683+0000 7fc85a338700 1 -- 172.21.15.7:0/1502129647 shutdown_connections 2024-09-13T18:53:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.683+0000 7fc85a338700 1 --2- 172.21.15.7:0/1502129647 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc84c005e10 0x7fc84c0575b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.683+0000 7fc85a338700 1 -- 172.21.15.7:0/1502129647 >> 172.21.15.7:0/1502129647 conn(0x7fc84c03a550 msgr2=0x7fc84c03a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.683+0000 7fc85a338700 1 -- 172.21.15.7:0/1502129647 shutdown_connections 2024-09-13T18:53:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.683+0000 7fc85a338700 1 -- 172.21.15.7:0/1502129647 wait complete. 2024-09-13T18:53:58.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.683+0000 7fc85a338700 1 Processor -- start 2024-09-13T18:53:58.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.683+0000 7fc85a338700 1 -- start start 2024-09-13T18:53:58.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc85a338700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc84c005e10 0x7fc84c0d4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:58.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc85a338700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc84c007200 0x7fc84c050df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:58.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc85a338700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc84c057af0 0x7fc84c02f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:58.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc85a338700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c0523e0 con 0x7fc84c057af0 2024-09-13T18:53:58.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc85a338700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c0d5490 con 0x7fc84c005e10 2024-09-13T18:53:58.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc85a338700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c0517d0 con 0x7fc84c007200 2024-09-13T18:53:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc859b37700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc84c057af0 0x7fc84c02f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc858b35700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc84c007200 0x7fc84c050df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc859b37700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc84c057af0 0x7fc84c02f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33406/0 (socket says 172.21.15.7:33406) 2024-09-13T18:53:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc859336700 1 --2- 172.21.15.7:0/835233449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc84c005e10 0x7fc84c0d4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:58.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc859b37700 1 -- 172.21.15.7:0/835233449 learned_addr learned my addr 172.21.15.7:0/835233449 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc858b35700 1 -- 172.21.15.7:0/835233449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc84c005e10 msgr2=0x7fc84c0d4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc858b35700 1 --2- 172.21.15.7:0/835233449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc84c005e10 0x7fc84c0d4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc858b35700 1 -- 172.21.15.7:0/835233449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc84c057af0 msgr2=0x7fc84c02f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc858b35700 1 --2- 172.21.15.7:0/835233449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc84c057af0 0x7fc84c02f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc858b35700 1 -- 172.21.15.7:0/835233449 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8500094b0 con 0x7fc84c007200 2024-09-13T18:53:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc858b35700 1 --2- 172.21.15.7:0/835233449 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc84c007200 0x7fc84c050df0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc84800c400 tx=0x7fc84800c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:58.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc845ffb700 1 -- 172.21.15.7:0/835233449 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8480052d0 con 0x7fc84c007200 2024-09-13T18:53:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc845ffb700 1 -- 172.21.15.7:0/835233449 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc848005450 con 0x7fc84c007200 2024-09-13T18:53:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc85a338700 1 -- 172.21.15.7:0/835233449 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84c051ab0 con 0x7fc84c007200 2024-09-13T18:53:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc845ffb700 1 -- 172.21.15.7:0/835233449 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8480038e0 con 0x7fc84c007200 2024-09-13T18:53:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.687+0000 7fc85a338700 1 -- 172.21.15.7:0/835233449 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84c052070 con 0x7fc84c007200 2024-09-13T18:53:58.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.691+0000 7fc85a338700 1 -- 172.21.15.7:0/835233449 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84c0076b0 con 0x7fc84c007200 2024-09-13T18:53:58.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.691+0000 7fc845ffb700 1 -- 172.21.15.7:0/835233449 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc848003a40 con 0x7fc84c007200 2024-09-13T18:53:58.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.691+0000 7fc845ffb700 1 --2- 172.21.15.7:0/835233449 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc838066730 0x7fc838068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:58.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.691+0000 7fc845ffb700 1 -- 172.21.15.7:0/835233449 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fc8480834f0 con 0x7fc84c007200 2024-09-13T18:53:58.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.691+0000 7fc859336700 1 --2- 172.21.15.7:0/835233449 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc838066730 0x7fc838068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:58.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.691+0000 7fc859336700 1 --2- 172.21.15.7:0/835233449 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc838066730 0x7fc838068bf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc8500097d0 tx=0x7fc850010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:58.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.703+0000 7fc845ffb700 1 -- 172.21.15.7:0/835233449 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc8480507b0 con 0x7fc84c007200 2024-09-13T18:53:58.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:58.879+0000 7fc85a338700 1 -- 172.21.15.7:0/835233449 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc84c0381a0 con 0x7fc84c007200 2024-09-13T18:53:59.271 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:53:59.274 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:53:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.543+0000 7fc845ffb700 1 -- 172.21.15.7:0/835233449 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc848053dd0 con 0x7fc84c007200 2024-09-13T18:53:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.547+0000 7fc85a338700 1 -- 172.21.15.7:0/835233449 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc838066730 msgr2=0x7fc838068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.547+0000 7fc85a338700 1 --2- 172.21.15.7:0/835233449 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc838066730 0x7fc838068bf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc8500097d0 tx=0x7fc850010000 comp rx=0 tx=0).stop 2024-09-13T18:53:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.547+0000 7fc85a338700 1 -- 172.21.15.7:0/835233449 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc84c007200 msgr2=0x7fc84c050df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.547+0000 7fc85a338700 1 --2- 172.21.15.7:0/835233449 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc84c007200 0x7fc84c050df0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc84800c400 tx=0x7fc84800c710 comp rx=0 tx=0).stop 2024-09-13T18:53:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.547+0000 7fc85a338700 1 -- 172.21.15.7:0/835233449 shutdown_connections 2024-09-13T18:53:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.547+0000 7fc85a338700 1 --2- 172.21.15.7:0/835233449 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc838066730 0x7fc838068bf0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.547+0000 7fc85a338700 1 --2- 172.21.15.7:0/835233449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc84c057af0 0x7fc84c02f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.547+0000 7fc85a338700 1 --2- 172.21.15.7:0/835233449 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc84c007200 0x7fc84c050df0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.547+0000 7fc85a338700 1 --2- 172.21.15.7:0/835233449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc84c005e10 0x7fc84c0d4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.547+0000 7fc85a338700 1 -- 172.21.15.7:0/835233449 >> 172.21.15.7:0/835233449 conn(0x7fc84c03a550 msgr2=0x7fc84c0383c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.547+0000 7fc85a338700 1 -- 172.21.15.7:0/835233449 shutdown_connections 2024-09-13T18:53:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.547+0000 7fc85a338700 1 -- 172.21.15.7:0/835233449 wait complete. 2024-09-13T18:53:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-09-13T18:53:59.570 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T18:53:59.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0d50b4700 1 Processor -- start 2024-09-13T18:53:59.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0d50b4700 1 -- start start 2024-09-13T18:53:59.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0d50b4700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa0d015cdf0 0x7fa0d00d3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:59.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0d50b4700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d00d1cf0 con 0x7fa0d00cfce0 2024-09-13T18:53:59.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0d50b4700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d015d1d0 con 0x7fa0d00d11c0 2024-09-13T18:53:59.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0d50b4700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0d00d1730 con 0x7fa0d015cdf0 2024-09-13T18:53:59.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0d48b3700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa0d00cfce0 0x7fa0d00b63d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45614/0 (socket says 172.21.15.7:45614) 2024-09-13T18:53:59.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0d48b3700 1 -- 172.21.15.7:0/693087911 learned_addr learned my addr 172.21.15.7:0/693087911 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:53:59.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0cffff700 1 --2- 172.21.15.7:0/693087911 >> v2:172.21.15.171:3300/0 conn(0x7fa0d015cdf0 0x7fa0d00d3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:59.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0ceffd700 1 -- 172.21.15.7:0/693087911 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2976348241 0 0) 0x7fa0d00d1cf0 con 0x7fa0d00cfce0 2024-09-13T18:53:59.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0ceffd700 1 -- 172.21.15.7:0/693087911 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0b80035d0 con 0x7fa0d00cfce0 2024-09-13T18:53:59.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0ceffd700 1 -- 172.21.15.7:0/693087911 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3439690553 0 0) 0x7fa0b80035d0 con 0x7fa0d00cfce0 2024-09-13T18:53:59.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0ceffd700 1 -- 172.21.15.7:0/693087911 >> v2:172.21.15.171:3300/0 conn(0x7fa0d015cdf0 msgr2=0x7fa0d00d3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0ceffd700 1 --2- 172.21.15.7:0/693087911 >> v2:172.21.15.171:3300/0 conn(0x7fa0d015cdf0 0x7fa0d00d3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0ceffd700 1 -- 172.21.15.7:0/693087911 >> v1:172.21.15.171:6789/0 conn(0x7fa0d00d11c0 legacy=0x7fa0d00cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0ceffd700 1 -- 172.21.15.7:0/693087911 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d00d1a10 con 0x7fa0d00cfce0 2024-09-13T18:53:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0ceffd700 1 -- 172.21.15.7:0/693087911 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7fa0c80019f0 con 0x7fa0d00cfce0 2024-09-13T18:53:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0ceffd700 1 -- 172.21.15.7:0/693087911 >> v1:172.21.15.7:6789/0 conn(0x7fa0d00cfce0 legacy=0x7fa0d00b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0ceffd700 1 --2- 172.21.15.7:0/693087911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b8003a20 0x7fa0b8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0ceffd700 1 --2- 172.21.15.7:0/693087911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0b8004870 0x7fa0b8004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0ceffd700 1 --2- 172.21.15.7:0/693087911 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa0b8005350 0x7fa0b800ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0ceffd700 1 -- 172.21.15.7:0/693087911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d00d1730 con 0x7fa0b8003a20 2024-09-13T18:53:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0ceffd700 1 -- 172.21.15.7:0/693087911 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b800c220 con 0x7fa0b8004870 2024-09-13T18:53:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0ceffd700 1 -- 172.21.15.7:0/693087911 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b800c3e0 con 0x7fa0b8005350 2024-09-13T18:53:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0cffff700 1 --2- 172.21.15.7:0/693087911 >> v2:172.21.15.171:3300/0 conn(0x7fa0d015cdf0 0x7fa0d00d3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:53:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0cffff700 1 --2- 172.21.15.7:0/693087911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b8003a20 0x7fa0b8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0cf7fe700 1 --2- 172.21.15.7:0/693087911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0b8004870 0x7fa0b8004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0d48b3700 1 --2- 172.21.15.7:0/693087911 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa0b8005350 0x7fa0b800ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0cffff700 1 -- 172.21.15.7:0/693087911 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa0b8005350 msgr2=0x7fa0b800ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0cffff700 1 --2- 172.21.15.7:0/693087911 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa0b8005350 0x7fa0b800ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0cffff700 1 -- 172.21.15.7:0/693087911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0b8004870 msgr2=0x7fa0b8004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:53:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0cffff700 1 --2- 172.21.15.7:0/693087911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0b8004870 0x7fa0b8004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0cffff700 1 -- 172.21.15.7:0/693087911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b80035d0 con 0x7fa0b8003a20 2024-09-13T18:53:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0d48b3700 1 -- 172.21.15.7:0/693087911 reap_dead start 2024-09-13T18:53:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0cf7fe700 1 --2- 172.21.15.7:0/693087911 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0b8004870 0x7fa0b8004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0cffff700 1 --2- 172.21.15.7:0/693087911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b8003a20 0x7fa0b8003e40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009cf0 tx=0x7fa0b80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf9adac440be926 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:59.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0ceffd700 1 -- 172.21.15.7:0/693087911 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0004030 con 0x7fa0b8003a20 2024-09-13T18:53:59.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0ceffd700 1 -- 172.21.15.7:0/693087911 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c0004190 con 0x7fa0b8003a20 2024-09-13T18:53:59.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.675+0000 7fa0ceffd700 1 -- 172.21.15.7:0/693087911 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0004500 con 0x7fa0b8003a20 2024-09-13T18:53:59.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0d50b4700 1 -- 172.21.15.7:0/693087911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b8003a20 msgr2=0x7fa0b8003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:59.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0d50b4700 1 --2- 172.21.15.7:0/693087911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b8003a20 0x7fa0b8003e40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009cf0 tx=0x7fa0b80058a0 comp rx=0 tx=0).stop 2024-09-13T18:53:59.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0d50b4700 1 -- 172.21.15.7:0/693087911 shutdown_connections 2024-09-13T18:53:59.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0d50b4700 1 --2- 172.21.15.7:0/693087911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b8003a20 0x7fa0b8003e40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:59.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0d50b4700 1 -- 172.21.15.7:0/693087911 >> 172.21.15.7:0/693087911 conn(0x7fa0d00d2be0 msgr2=0x7fa0d00d2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:53:59.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0d50b4700 1 -- 172.21.15.7:0/693087911 shutdown_connections 2024-09-13T18:53:59.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0d50b4700 1 -- 172.21.15.7:0/693087911 wait complete. 2024-09-13T18:53:59.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0d50b4700 1 Processor -- start 2024-09-13T18:53:59.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0d50b4700 1 -- start start 2024-09-13T18:53:59.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0d50b4700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa0b8003a20 0x7fa0d015fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:59.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0d50b4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d0155fa0 0x7fa0d01556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:59.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0d50b4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0d0158b30 0x7fa0d0158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:59.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0d50b4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d01594f0 con 0x7fa0d0158b30 2024-09-13T18:53:59.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0d50b4700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d00ca680 con 0x7fa0d0155fa0 2024-09-13T18:53:59.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0d50b4700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d01600b0 con 0x7fa0b8003a20 2024-09-13T18:53:59.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0d48b3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0d0158b30 0x7fa0d0158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:59.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0d48b3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0d0158b30 0x7fa0d0158fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33426/0 (socket says 172.21.15.7:33426) 2024-09-13T18:53:59.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0d48b3700 1 -- 172.21.15.7:0/1122298370 learned_addr learned my addr 172.21.15.7:0/1122298370 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:53:59.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0cffff700 1 --2- 172.21.15.7:0/1122298370 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa0b8003a20 0x7fa0d015fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:59.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.679+0000 7fa0cf7fe700 1 --2- 172.21.15.7:0/1122298370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d0155fa0 0x7fa0d01556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:59.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.683+0000 7fa0d48b3700 1 -- 172.21.15.7:0/1122298370 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa0b8003a20 msgr2=0x7fa0d015fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:59.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.683+0000 7fa0d48b3700 1 --2- 172.21.15.7:0/1122298370 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa0b8003a20 0x7fa0d015fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:59.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.683+0000 7fa0d48b3700 1 -- 172.21.15.7:0/1122298370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d0155fa0 msgr2=0x7fa0d01556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:53:59.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.683+0000 7fa0d48b3700 1 --2- 172.21.15.7:0/1122298370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d0155fa0 0x7fa0d01556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:53:59.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.683+0000 7fa0d48b3700 1 -- 172.21.15.7:0/1122298370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d00c9d50 con 0x7fa0d0158b30 2024-09-13T18:53:59.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.683+0000 7fa0cf7fe700 1 --2- 172.21.15.7:0/1122298370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d0155fa0 0x7fa0d01556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:53:59.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.683+0000 7fa0d48b3700 1 --2- 172.21.15.7:0/1122298370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0d0158b30 0x7fa0d0158fb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa0d00c9c40 tx=0x7fa0c8007cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:59.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.683+0000 7fa0ccff9700 1 -- 172.21.15.7:0/1122298370 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c800e8d0 con 0x7fa0d0158b30 2024-09-13T18:53:59.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.683+0000 7fa0ccff9700 1 -- 172.21.15.7:0/1122298370 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c800ea30 con 0x7fa0d0158b30 2024-09-13T18:53:59.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.683+0000 7fa0d50b4700 1 -- 172.21.15.7:0/1122298370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d00ca030 con 0x7fa0d0158b30 2024-09-13T18:53:59.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.683+0000 7fa0ccff9700 1 -- 172.21.15.7:0/1122298370 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c800ede0 con 0x7fa0d0158b30 2024-09-13T18:53:59.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.683+0000 7fa0d50b4700 1 -- 172.21.15.7:0/1122298370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d0160330 con 0x7fa0d0158b30 2024-09-13T18:53:59.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.683+0000 7fa0ccff9700 1 -- 172.21.15.7:0/1122298370 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa0c801d460 con 0x7fa0d0158b30 2024-09-13T18:53:59.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.683+0000 7fa0d50b4700 1 -- 172.21.15.7:0/1122298370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d00cf5d0 con 0x7fa0d0158b30 2024-09-13T18:53:59.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.683+0000 7fa0ccff9700 1 --2- 172.21.15.7:0/1122298370 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa0bc066730 0x7fa0bc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:53:59.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.683+0000 7fa0ccff9700 1 -- 172.21.15.7:0/1122298370 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5443+0+0 (secure 0 0 0) 0x7fa0c80842b0 con 0x7fa0d0158b30 2024-09-13T18:53:59.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.683+0000 7fa0cffff700 1 --2- 172.21.15.7:0/1122298370 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa0bc066730 0x7fa0bc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:53:59.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.687+0000 7fa0cffff700 1 --2- 172.21.15.7:0/1122298370 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa0bc066730 0x7fa0bc068bf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00090b0 tx=0x7fa0c001e040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:53:59.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.691+0000 7fa0ccff9700 1 -- 172.21.15.7:0/1122298370 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa0c8051720 con 0x7fa0d0158b30 2024-09-13T18:53:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:53:59.863+0000 7fa0d50b4700 1 -- 172.21.15.7:0/1122298370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa0d00cf5d0 con 0x7fa0d0158b30 2024-09-13T18:54:00.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.551+0000 7fa0ccff9700 1 -- 172.21.15.7:0/1122298370 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa0c8054d40 con 0x7fa0d0158b30 2024-09-13T18:54:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.555+0000 7fa0d50b4700 1 -- 172.21.15.7:0/1122298370 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa0bc066730 msgr2=0x7fa0bc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.555+0000 7fa0d50b4700 1 --2- 172.21.15.7:0/1122298370 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa0bc066730 0x7fa0bc068bf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00090b0 tx=0x7fa0c001e040 comp rx=0 tx=0).stop 2024-09-13T18:54:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.555+0000 7fa0d50b4700 1 -- 172.21.15.7:0/1122298370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0d0158b30 msgr2=0x7fa0d0158fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:00.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.555+0000 7fa0d50b4700 1 --2- 172.21.15.7:0/1122298370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0d0158b30 0x7fa0d0158fb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa0d00c9c40 tx=0x7fa0c8007cb0 comp rx=0 tx=0).stop 2024-09-13T18:54:00.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.555+0000 7fa0d50b4700 1 -- 172.21.15.7:0/1122298370 shutdown_connections 2024-09-13T18:54:00.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.555+0000 7fa0d50b4700 1 --2- 172.21.15.7:0/1122298370 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa0bc066730 0x7fa0bc068bf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:00.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.555+0000 7fa0d50b4700 1 --2- 172.21.15.7:0/1122298370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0d0158b30 0x7fa0d0158fb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:00.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.555+0000 7fa0d50b4700 1 --2- 172.21.15.7:0/1122298370 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d0155fa0 0x7fa0d01556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:00.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.555+0000 7fa0d50b4700 1 --2- 172.21.15.7:0/1122298370 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa0b8003a20 0x7fa0d015fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:00.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.555+0000 7fa0d50b4700 1 -- 172.21.15.7:0/1122298370 >> 172.21.15.7:0/1122298370 conn(0x7fa0d00d2be0 msgr2=0x7fa0d00b7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:00.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.559+0000 7fa0d50b4700 1 -- 172.21.15.7:0/1122298370 shutdown_connections 2024-09-13T18:54:00.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.559+0000 7fa0d50b4700 1 -- 172.21.15.7:0/1122298370 wait complete. 2024-09-13T18:54:00.569 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-09-13T18:54:00.582 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T18:54:00.584 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T18:54:00.605 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-09-13T18:54:00.605 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-09-13T18:54:00.605 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T18:54:00.605 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-09-13T18:54:00.605 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.605 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-09-13T18:54:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-09-13T18:54:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-09-13T18:54:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-09-13T18:54:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-09-13T18:54:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-09-13T18:54:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T18:54:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-09-13T18:54:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.606 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T18:54:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T18:54:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-09-13T18:54:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-09-13T18:54:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-09-13T18:54:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-09-13T18:54:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-09-13T18:54:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-09-13T18:54:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-09-13T18:54:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-09-13T18:54:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T18:54:00.607 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T18:54:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-09-13T18:54:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T18:54:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-09-13T18:54:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-09-13T18:54:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-09-13T18:54:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-09-13T18:54:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-09-13T18:54:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.171,[v2:172.21.15.171:3301,v1:172.21.15.171:6790]" 2024-09-13T18:54:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 33406d92-f401-438a-876e-a56de3962d8e 2024-09-13T18:54:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.609 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-09-13T18:54:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-09-13T18:54:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-09-13T18:54:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-09-13T18:54:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-09-13T18:54:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-09-13T18:54:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-09-13T18:54:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-09-13T18:54:00.609 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-09-13T18:54:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-09-13T18:54:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-09-13T18:54:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-09-13T18:54:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-09-13T18:54:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-09-13T18:54:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.610 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-09-13T18:54:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-09-13T18:54:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-09-13T18:54:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-09-13T18:54:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-09-13T18:54:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: bdev async discard = True 2024-09-13T18:54:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: bdev enable discard = True 2024-09-13T18:54:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore allocator = bitmap 2024-09-13T18:54:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-09-13T18:54:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-09-13T18:54:00.611 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 1/20 2024-09-13T18:54:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 1/20 2024-09-13T18:54:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:54:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-09-13T18:54:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 4/10 2024-09-13T18:54:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T18:54:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-09-13T18:54:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-09-13T18:54:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2024-09-13T18:54:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2024-09-13T18:54:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-09-13T18:54:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.612 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-09-13T18:54:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:54:00.612 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-09-13T18:54:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-13T18:54:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-13T18:54:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:54:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:54:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-09-13T18:54:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-09-13T18:54:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.613 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-09-13T18:54:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:54:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-13T18:54:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-09-13T18:54:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-13T18:54:00.613 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-09-13T18:54:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-09-13T18:54:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:54:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-09-13T18:54:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-09-13T18:54:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:54:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T18:54:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T18:54:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T18:54:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T18:54:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T18:54:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T18:54:00.614 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-09-13T18:54:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-09-13T18:54:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-09-13T18:54:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: mon op complaint time = 120 2024-09-13T18:54:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on pool no app = False 2024-09-13T18:54:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:00.615 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-09-13T18:54:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-09-13T18:54:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-09-13T18:54:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-09-13T18:54:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T18:54:00.615 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T18:54:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 600 2024-09-13T18:54:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-09-13T18:54:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:54:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = "" 2024-09-13T18:54:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-09-13T18:54:00.616 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-09-13T18:54:00.616 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-09-13T18:54:00.616 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-09-13T18:54:00.616 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-09-13T18:54:00.616 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-09-13T18:54:00.616 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-09-13T18:54:00.616 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-09-13T18:54:00.616 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-09-13T18:54:00.616 INFO:teuthology.orchestra.run.smithi007.stdout:[mds] 2024-09-13T18:54:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds = 20 2024-09-13T18:54:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds balancer = 20 2024-09-13T18:54:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:54:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug frag = True 2024-09-13T18:54:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug scatterstat = True 2024-09-13T18:54:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: mds op complaint time = 180 2024-09-13T18:54:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: mds verify scatter = True 2024-09-13T18:54:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-09-13T18:54:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-09-13T18:54:00.617 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-09-13T18:54:00.618 INFO:teuthology.orchestra.run.smithi171.stdout:[global] 2024-09-13T18:54:00.618 INFO:teuthology.orchestra.run.smithi171.stdout: chdir = "" 2024-09-13T18:54:00.618 INFO:teuthology.orchestra.run.smithi171.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T18:54:00.618 INFO:teuthology.orchestra.run.smithi171.stdout: auth supported = cephx 2024-09-13T18:54:00.618 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.618 INFO:teuthology.orchestra.run.smithi171.stdout: filestore xattr use omap = true 2024-09-13T18:54:00.618 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.619 INFO:teuthology.orchestra.run.smithi171.stdout: mon clock drift allowed = 1.000 2024-09-13T18:54:00.619 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.619 INFO:teuthology.orchestra.run.smithi171.stdout: osd crush chooseleaf type = 0 2024-09-13T18:54:00.619 INFO:teuthology.orchestra.run.smithi171.stdout: auth debug = true 2024-09-13T18:54:00.619 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.619 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on old message = true 2024-09-13T18:54:00.619 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on bug = true 2024-09-13T18:54:00.619 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.619 INFO:teuthology.orchestra.run.smithi171.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T18:54:00.619 INFO:teuthology.orchestra.run.smithi171.stdout: mon pg warn max object skew = 0 2024-09-13T18:54:00.619 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.619 INFO:teuthology.orchestra.run.smithi171.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T18:54:00.619 INFO:teuthology.orchestra.run.smithi171.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T18:54:00.619 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.620 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default size = 2 2024-09-13T18:54:00.620 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.620 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow primary affinity = true 2024-09-13T18:54:00.620 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow pg remap = true 2024-09-13T18:54:00.620 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on legacy crush tunables = false 2024-09-13T18:54:00.620 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on crush straw calc version zero = false 2024-09-13T18:54:00.620 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on no sortbitwise = false 2024-09-13T18:54:00.620 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on osd down out interval zero = false 2024-09-13T18:54:00.620 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on too few osds = false 2024-09-13T18:54:00.620 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T18:54:00.620 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T18:54:00.620 INFO:teuthology.orchestra.run.smithi171.stdout: mon_allow_pool_size_one = true 2024-09-13T18:54:00.620 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.620 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T18:54:00.621 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.621 INFO:teuthology.orchestra.run.smithi171.stdout: osd default data pool replay window = 5 2024-09-13T18:54:00.621 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.621 INFO:teuthology.orchestra.run.smithi171.stdout: mon allow pool delete = true 2024-09-13T18:54:00.621 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.621 INFO:teuthology.orchestra.run.smithi171.stdout: mon cluster log file level = debug 2024-09-13T18:54:00.621 INFO:teuthology.orchestra.run.smithi171.stdout: debug asserts on shutdown = true 2024-09-13T18:54:00.621 INFO:teuthology.orchestra.run.smithi171.stdout: mon health detail to clog = false 2024-09-13T18:54:00.621 INFO:teuthology.orchestra.run.smithi171.stdout: mon host = "172.21.15.7,172.21.15.171,[v2:172.21.15.171:3301,v1:172.21.15.171:6790]" 2024-09-13T18:54:00.621 INFO:teuthology.orchestra.run.smithi171.stdout: fsid = 33406d92-f401-438a-876e-a56de3962d8e 2024-09-13T18:54:00.621 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.621 INFO:teuthology.orchestra.run.smithi171.stdout:[osd] 2024-09-13T18:54:00.621 INFO:teuthology.orchestra.run.smithi171.stdout: osd journal size = 100 2024-09-13T18:54:00.622 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.622 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub load threshold = 5.0 2024-09-13T18:54:00.622 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub max interval = 600 2024-09-13T18:54:00.622 INFO:teuthology.orchestra.run.smithi171.stdout: osd mclock profile = high_recovery_ops 2024-09-13T18:54:00.622 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.622 INFO:teuthology.orchestra.run.smithi171.stdout: osd recover clone overlap = true 2024-09-13T18:54:00.622 INFO:teuthology.orchestra.run.smithi171.stdout: osd recovery max chunk = 1048576 2024-09-13T18:54:00.622 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.622 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug shutdown = true 2024-09-13T18:54:00.622 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug op order = true 2024-09-13T18:54:00.622 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug verify stray on activate = true 2024-09-13T18:54:00.622 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.622 INFO:teuthology.orchestra.run.smithi171.stdout: osd open classes on start = true 2024-09-13T18:54:00.623 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug pg log writeout = true 2024-09-13T18:54:00.623 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.623 INFO:teuthology.orchestra.run.smithi171.stdout: osd deep scrub update digest min age = 30 2024-09-13T18:54:00.623 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.623 INFO:teuthology.orchestra.run.smithi171.stdout: osd map max advance = 10 2024-09-13T18:54:00.623 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.623 INFO:teuthology.orchestra.run.smithi171.stdout: journal zero on create = true 2024-09-13T18:54:00.623 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.623 INFO:teuthology.orchestra.run.smithi171.stdout: filestore ondisk finisher threads = 3 2024-09-13T18:54:00.623 INFO:teuthology.orchestra.run.smithi171.stdout: filestore apply finisher threads = 3 2024-09-13T18:54:00.623 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.623 INFO:teuthology.orchestra.run.smithi171.stdout: bdev debug aio = true 2024-09-13T18:54:00.623 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug misdirected ops = true 2024-09-13T18:54:00.623 INFO:teuthology.orchestra.run.smithi171.stdout: bdev async discard = True 2024-09-13T18:54:00.624 INFO:teuthology.orchestra.run.smithi171.stdout: bdev enable discard = True 2024-09-13T18:54:00.624 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore allocator = bitmap 2024-09-13T18:54:00.624 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore block size = 96636764160 2024-09-13T18:54:00.624 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore fsck on mount = True 2024-09-13T18:54:00.624 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluefs = 1/20 2024-09-13T18:54:00.624 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluestore = 1/20 2024-09-13T18:54:00.624 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:54:00.624 INFO:teuthology.orchestra.run.smithi171.stdout: debug osd = 20 2024-09-13T18:54:00.624 INFO:teuthology.orchestra.run.smithi171.stdout: debug rocksdb = 4/10 2024-09-13T18:54:00.624 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T18:54:00.624 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd full ratio = 0.9 2024-09-13T18:54:00.624 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd nearfull ratio = 0.8 2024-09-13T18:54:00.624 INFO:teuthology.orchestra.run.smithi171.stdout: osd failsafe full ratio = 0.95 2024-09-13T18:54:00.624 INFO:teuthology.orchestra.run.smithi171.stdout: osd objectstore = bluestore 2024-09-13T18:54:00.624 INFO:teuthology.orchestra.run.smithi171.stdout: osd op complaint time = 180 2024-09-13T18:54:00.625 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.625 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr] 2024-09-13T18:54:00.625 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:54:00.625 INFO:teuthology.orchestra.run.smithi171.stdout: debug mgr = 20 2024-09-13T18:54:00.625 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-09-13T18:54:00.625 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-09-13T18:54:00.625 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:54:00.625 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:54:00.625 INFO:teuthology.orchestra.run.smithi171.stdout: mgr/telemetry/nag = false 2024-09-13T18:54:00.625 INFO:teuthology.orchestra.run.smithi171.stdout: debug client = 20 2024-09-13T18:54:00.625 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.625 INFO:teuthology.orchestra.run.smithi171.stdout:[mon] 2024-09-13T18:54:00.625 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:54:00.626 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-09-13T18:54:00.626 INFO:teuthology.orchestra.run.smithi171.stdout: debug paxos = 20 2024-09-13T18:54:00.626 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-09-13T18:54:00.626 INFO:teuthology.orchestra.run.smithi171.stdout: mon data avail warn = 5 2024-09-13T18:54:00.626 INFO:teuthology.orchestra.run.smithi171.stdout: mon mgr mkfs grace = 240 2024-09-13T18:54:00.626 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:54:00.626 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd reporter subtree level = osd 2024-09-13T18:54:00.626 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd prime pg temp = true 2024-09-13T18:54:00.626 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:54:00.626 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.626 INFO:teuthology.orchestra.run.smithi171.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T18:54:00.626 INFO:teuthology.orchestra.run.smithi171.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T18:54:00.626 INFO:teuthology.orchestra.run.smithi171.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T18:54:00.626 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.627 INFO:teuthology.orchestra.run.smithi171.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T18:54:00.627 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T18:54:00.627 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T18:54:00.627 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.627 INFO:teuthology.orchestra.run.smithi171.stdout: # 1m isn't quite enough 2024-09-13T18:54:00.627 INFO:teuthology.orchestra.run.smithi171.stdout: mon_down_mkfs_grace = 2m 2024-09-13T18:54:00.627 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.627 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_filestore_osds = false 2024-09-13T18:54:00.627 INFO:teuthology.orchestra.run.smithi171.stdout: mon op complaint time = 120 2024-09-13T18:54:00.627 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on pool no app = False 2024-09-13T18:54:00.627 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:00.627 INFO:teuthology.orchestra.run.smithi171.stdout:[client] 2024-09-13T18:54:00.627 INFO:teuthology.orchestra.run.smithi171.stdout: rgw cache enabled = true 2024-09-13T18:54:00.627 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable ops log = true 2024-09-13T18:54:00.628 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable usage log = true 2024-09-13T18:54:00.628 INFO:teuthology.orchestra.run.smithi171.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T18:54:00.628 INFO:teuthology.orchestra.run.smithi171.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T18:54:00.628 INFO:teuthology.orchestra.run.smithi171.stdout: client mount timeout = 600 2024-09-13T18:54:00.628 INFO:teuthology.orchestra.run.smithi171.stdout: debug client = 20 2024-09-13T18:54:00.628 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:54:00.628 INFO:teuthology.orchestra.run.smithi171.stdout: pid file = "" 2024-09-13T18:54:00.628 INFO:teuthology.orchestra.run.smithi171.stdout: rados mon op timeout = 900 2024-09-13T18:54:00.628 INFO:teuthology.orchestra.run.smithi171.stdout: rados osd op timeout = 900 2024-09-13T18:54:00.628 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.a] 2024-09-13T18:54:00.628 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.b] 2024-09-13T18:54:00.628 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.c] 2024-09-13T18:54:00.628 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.a] 2024-09-13T18:54:00.629 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.c] 2024-09-13T18:54:00.629 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.b] 2024-09-13T18:54:00.629 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.d] 2024-09-13T18:54:00.629 INFO:teuthology.orchestra.run.smithi171.stdout:[mds] 2024-09-13T18:54:00.629 INFO:teuthology.orchestra.run.smithi171.stdout: debug mds = 20 2024-09-13T18:54:00.629 INFO:teuthology.orchestra.run.smithi171.stdout: debug mds balancer = 20 2024-09-13T18:54:00.629 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:54:00.629 INFO:teuthology.orchestra.run.smithi171.stdout: mds debug frag = True 2024-09-13T18:54:00.629 INFO:teuthology.orchestra.run.smithi171.stdout: mds debug scatterstat = True 2024-09-13T18:54:00.629 INFO:teuthology.orchestra.run.smithi171.stdout: mds op complaint time = 180 2024-09-13T18:54:00.629 INFO:teuthology.orchestra.run.smithi171.stdout: mds verify scatter = True 2024-09-13T18:54:00.629 INFO:teuthology.orchestra.run.smithi171.stdout: osd op complaint time = 180 2024-09-13T18:54:00.629 INFO:teuthology.orchestra.run.smithi171.stdout: rados mon op timeout = 900 2024-09-13T18:54:00.630 INFO:teuthology.orchestra.run.smithi171.stdout: rados osd op timeout = 900 2024-09-13T18:54:00.631 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-13T18:54:00.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.799+0000 7f4426277700 1 Processor -- start 2024-09-13T18:54:00.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.799+0000 7f4426277700 1 -- start start 2024-09-13T18:54:00.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.799+0000 7f4426277700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44201461e0 0x7f44200b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:00.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.799+0000 7f4426277700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4420144330 con 0x7f4420146cd0 2024-09-13T18:54:00.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.799+0000 7f4426277700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44200b3680 con 0x7f44200b30c0 2024-09-13T18:54:00.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.799+0000 7f4426277700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44200b4720 con 0x7f44201461e0 2024-09-13T18:54:00.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.799+0000 7f4424814700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4420146cd0 0x7f4420144600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45620/0 (socket says 172.21.15.7:45620) 2024-09-13T18:54:00.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.799+0000 7f4424814700 1 -- 172.21.15.7:0/4290527768 learned_addr learned my addr 172.21.15.7:0/4290527768 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:00.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.799+0000 7f441ffff700 1 --2- 172.21.15.7:0/4290527768 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44201461e0 0x7f44200b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:00.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.799+0000 7f441effd700 1 -- 172.21.15.7:0/4290527768 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2718410670 0 0) 0x7f4420144330 con 0x7f4420146cd0 2024-09-13T18:54:00.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.799+0000 7f441effd700 1 -- 172.21.15.7:0/4290527768 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44080035d0 con 0x7f4420146cd0 2024-09-13T18:54:00.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.803+0000 7f441effd700 1 -- 172.21.15.7:0/4290527768 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610493300 0 0) 0x7f44200b3680 con 0x7f44200b30c0 2024-09-13T18:54:00.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.803+0000 7f441effd700 1 -- 172.21.15.7:0/4290527768 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4420144330 con 0x7f44200b30c0 2024-09-13T18:54:00.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.803+0000 7f441ffff700 1 -- 172.21.15.7:0/4290527768 >> v1:172.21.15.171:6789/0 conn(0x7f44200b30c0 legacy=0x7f44201465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:00.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.803+0000 7f441ffff700 1 -- 172.21.15.7:0/4290527768 >> v1:172.21.15.7:6789/0 conn(0x7f4420146cd0 legacy=0x7f4420144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:00.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.803+0000 7f441ffff700 1 -- 172.21.15.7:0/4290527768 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44200b3960 con 0x7f44201461e0 2024-09-13T18:54:00.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.803+0000 7f441ffff700 1 --2- 172.21.15.7:0/4290527768 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44201461e0 0x7f44200b2b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4414009880 tx=0x7f4414009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f65d1cd8dcd2c89 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:00.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.803+0000 7f441effd700 1 -- 172.21.15.7:0/4290527768 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414011830 con 0x7f44201461e0 2024-09-13T18:54:00.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.803+0000 7f441effd700 1 -- 172.21.15.7:0/4290527768 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4414011990 con 0x7f44201461e0 2024-09-13T18:54:00.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.803+0000 7f441effd700 1 -- 172.21.15.7:0/4290527768 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414011d00 con 0x7f44201461e0 2024-09-13T18:54:00.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.803+0000 7f4426277700 1 -- 172.21.15.7:0/4290527768 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44201461e0 msgr2=0x7f44200b2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:00.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.803+0000 7f4426277700 1 --2- 172.21.15.7:0/4290527768 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44201461e0 0x7f44200b2b80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4414009880 tx=0x7f4414009b90 comp rx=0 tx=0).stop 2024-09-13T18:54:00.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.803+0000 7f4426277700 1 -- 172.21.15.7:0/4290527768 shutdown_connections 2024-09-13T18:54:00.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.803+0000 7f4426277700 1 --2- 172.21.15.7:0/4290527768 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f44201461e0 0x7f44200b2b80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:00.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.803+0000 7f4426277700 1 -- 172.21.15.7:0/4290527768 >> 172.21.15.7:0/4290527768 conn(0x7f44200af620 msgr2=0x7f44200afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:00.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.803+0000 7f4426277700 1 -- 172.21.15.7:0/4290527768 shutdown_connections 2024-09-13T18:54:00.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.803+0000 7f4426277700 1 -- 172.21.15.7:0/4290527768 wait complete. 2024-09-13T18:54:00.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.803+0000 7f4426277700 1 Processor -- start 2024-09-13T18:54:00.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.803+0000 7f4426277700 1 -- start start 2024-09-13T18:54:00.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f4426277700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44200b30c0 0x7f4420143400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:00.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f4426277700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44201461e0 0x7f44201ed1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:00.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f4426277700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4420146cd0 0x7f44200ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:00.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f4426277700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44201ecdd0 con 0x7f44200b30c0 2024-09-13T18:54:00.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f4426277700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44201ec150 con 0x7f44201461e0 2024-09-13T18:54:00.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f4426277700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44201ec430 con 0x7f4420146cd0 2024-09-13T18:54:00.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f441ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44200b30c0 0x7f4420143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:00.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f441ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44200b30c0 0x7f4420143400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33428/0 (socket says 172.21.15.7:33428) 2024-09-13T18:54:00.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f441ffff700 1 -- 172.21.15.7:0/937788113 learned_addr learned my addr 172.21.15.7:0/937788113 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:00.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f441f7fe700 1 --2- 172.21.15.7:0/937788113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44201461e0 0x7f44201ed1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:00.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f4424814700 1 --2- 172.21.15.7:0/937788113 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4420146cd0 0x7f44200ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:00.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f441f7fe700 1 -- 172.21.15.7:0/937788113 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4420146cd0 msgr2=0x7f44200ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:00.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f441f7fe700 1 --2- 172.21.15.7:0/937788113 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4420146cd0 0x7f44200ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:00.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f441f7fe700 1 -- 172.21.15.7:0/937788113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44200b30c0 msgr2=0x7f4420143400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f441f7fe700 1 --2- 172.21.15.7:0/937788113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44200b30c0 0x7f4420143400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f441f7fe700 1 -- 172.21.15.7:0/937788113 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44140094b0 con 0x7f44201461e0 2024-09-13T18:54:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f441f7fe700 1 --2- 172.21.15.7:0/937788113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44201461e0 0x7f44201ed1f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f441000c500 tx=0x7f441000c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:00.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f441cff9700 1 -- 172.21.15.7:0/937788113 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44100052d0 con 0x7f44201461e0 2024-09-13T18:54:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f441cff9700 1 -- 172.21.15.7:0/937788113 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4410005430 con 0x7f44201461e0 2024-09-13T18:54:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f4426277700 1 -- 172.21.15.7:0/937788113 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44201ec710 con 0x7f44201461e0 2024-09-13T18:54:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f441cff9700 1 -- 172.21.15.7:0/937788113 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44100055a0 con 0x7f44201461e0 2024-09-13T18:54:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.807+0000 7f4426277700 1 -- 172.21.15.7:0/937788113 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44201ed730 con 0x7f44201461e0 2024-09-13T18:54:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.811+0000 7f4426277700 1 -- 172.21.15.7:0/937788113 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44200cb2c0 con 0x7f44201461e0 2024-09-13T18:54:00.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.811+0000 7f441cff9700 1 -- 172.21.15.7:0/937788113 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f44100038e0 con 0x7f44201461e0 2024-09-13T18:54:00.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.811+0000 7f441cff9700 1 --2- 172.21.15.7:0/937788113 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f440c066730 0x7f440c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:00.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.811+0000 7f441cff9700 1 -- 172.21.15.7:0/937788113 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f44100832c0 con 0x7f44201461e0 2024-09-13T18:54:00.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.811+0000 7f441ffff700 1 --2- 172.21.15.7:0/937788113 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f440c066730 0x7f440c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:00.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.811+0000 7f441ffff700 1 --2- 172.21.15.7:0/937788113 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f440c066730 0x7f440c068bf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4414000c00 tx=0x7f4414010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:00.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:00.823+0000 7f441cff9700 1 -- 172.21.15.7:0/937788113 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4410050850 con 0x7f44201461e0 2024-09-13T18:54:01.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.003+0000 7f4426277700 1 -- 172.21.15.7:0/937788113 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f44201ed8f0 con 0x7f44201461e0 2024-09-13T18:54:01.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.015+0000 7f441cff9700 1 -- 172.21.15.7:0/937788113 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4410053e70 con 0x7f44201461e0 2024-09-13T18:54:01.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.015+0000 7f4426277700 1 -- 172.21.15.7:0/937788113 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f440c066730 msgr2=0x7f440c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:01.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.015+0000 7f4426277700 1 --2- 172.21.15.7:0/937788113 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f440c066730 0x7f440c068bf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4414000c00 tx=0x7f4414010000 comp rx=0 tx=0).stop 2024-09-13T18:54:01.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.015+0000 7f4426277700 1 -- 172.21.15.7:0/937788113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44201461e0 msgr2=0x7f44201ed1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:01.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.015+0000 7f4426277700 1 --2- 172.21.15.7:0/937788113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44201461e0 0x7f44201ed1f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f441000c500 tx=0x7f441000c810 comp rx=0 tx=0).stop 2024-09-13T18:54:01.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.015+0000 7f4426277700 1 -- 172.21.15.7:0/937788113 shutdown_connections 2024-09-13T18:54:01.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.015+0000 7f4426277700 1 --2- 172.21.15.7:0/937788113 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f440c066730 0x7f440c068bf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.015+0000 7f4426277700 1 --2- 172.21.15.7:0/937788113 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4420146cd0 0x7f44200ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.015+0000 7f4426277700 1 --2- 172.21.15.7:0/937788113 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f44201461e0 0x7f44201ed1f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.015+0000 7f4426277700 1 --2- 172.21.15.7:0/937788113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44200b30c0 0x7f4420143400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.015+0000 7f4426277700 1 -- 172.21.15.7:0/937788113 >> 172.21.15.7:0/937788113 conn(0x7f44200af620 msgr2=0x7f44201632c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:01.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.015+0000 7f4426277700 1 -- 172.21.15.7:0/937788113 shutdown_connections 2024-09-13T18:54:01.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.015+0000 7f4426277700 1 -- 172.21.15.7:0/937788113 wait complete. 2024-09-13T18:54:01.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj' 2024-09-13T18:54:01.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.143+0000 7fea3eeec700 1 Processor -- start 2024-09-13T18:54:01.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.143+0000 7fea3eeec700 1 -- start start 2024-09-13T18:54:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3eeec700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fea380b38d0 0x7fea3815d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3eeec700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fea3806b5b0 0x7fea3815b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3eeec700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea3815a630 con 0x7fea38159060 2024-09-13T18:54:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3eeec700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3815a450 con 0x7fea380b38d0 2024-09-13T18:54:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3eeec700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3806bd30 con 0x7fea3806b5b0 2024-09-13T18:54:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3d6e9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fea380b38d0 0x7fea3815d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3d6e9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fea380b38d0 0x7fea3815d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39136/0 (socket says 172.21.15.7:39136) 2024-09-13T18:54:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3d6e9700 1 -- 172.21.15.7:0/1778487001 learned_addr learned my addr 172.21.15.7:0/1778487001 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3e6eb700 1 --2- 172.21.15.7:0/1778487001 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fea3806b5b0 0x7fea3815b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:01.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3d6e9700 1 -- 172.21.15.7:0/1778487001 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fea3806b5b0 msgr2=0x7fea3815b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:01.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3d6e9700 1 --2- 172.21.15.7:0/1778487001 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fea3806b5b0 0x7fea3815b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3d6e9700 1 -- 172.21.15.7:0/1778487001 >> v1:172.21.15.171:6789/0 conn(0x7fea38159060 legacy=0x7fea380b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:01.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3d6e9700 1 -- 172.21.15.7:0/1778487001 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3806c010 con 0x7fea380b38d0 2024-09-13T18:54:01.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3cee8700 1 -- 172.21.15.7:0/1778487001 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579521693 0 0) 0x7fea3815a630 con 0x7fea38159060 2024-09-13T18:54:01.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3d6e9700 1 --2- 172.21.15.7:0/1778487001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea380b38d0 0x7fea3815d5b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fea28006fd0 tx=0x7fea2800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8baecfc0d51461d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:01.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3cee8700 1 -- 172.21.15.7:0/1778487001 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2800dc20 con 0x7fea380b38d0 2024-09-13T18:54:01.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3cee8700 1 -- 172.21.15.7:0/1778487001 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2800dd80 con 0x7fea380b38d0 2024-09-13T18:54:01.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3cee8700 1 -- 172.21.15.7:0/1778487001 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28004250 con 0x7fea380b38d0 2024-09-13T18:54:01.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3eeec700 1 -- 172.21.15.7:0/1778487001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea380b38d0 msgr2=0x7fea3815d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:01.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3eeec700 1 --2- 172.21.15.7:0/1778487001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea380b38d0 0x7fea3815d5b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fea28006fd0 tx=0x7fea2800bea0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3eeec700 1 -- 172.21.15.7:0/1778487001 shutdown_connections 2024-09-13T18:54:01.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3eeec700 1 --2- 172.21.15.7:0/1778487001 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fea3806b5b0 0x7fea3815b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3eeec700 1 --2- 172.21.15.7:0/1778487001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea380b38d0 0x7fea3815d5b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3eeec700 1 -- 172.21.15.7:0/1778487001 >> 172.21.15.7:0/1778487001 conn(0x7fea3806d900 msgr2=0x7fea3806dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:01.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3eeec700 1 -- 172.21.15.7:0/1778487001 shutdown_connections 2024-09-13T18:54:01.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.147+0000 7fea3eeec700 1 -- 172.21.15.7:0/1778487001 wait complete. 2024-09-13T18:54:01.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.151+0000 7fea3eeec700 1 Processor -- start 2024-09-13T18:54:01.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.151+0000 7fea3eeec700 1 -- start start 2024-09-13T18:54:01.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.151+0000 7fea3eeec700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fea3806b5b0 0x7fea381519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:01.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.151+0000 7fea3eeec700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea380b38d0 0x7fea38149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:01.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.151+0000 7fea3eeec700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea38159060 0x7fea38156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:01.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.151+0000 7fea3eeec700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea38149f60 con 0x7fea38159060 2024-09-13T18:54:01.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.151+0000 7fea3eeec700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea380ad2d0 con 0x7fea380b38d0 2024-09-13T18:54:01.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.151+0000 7fea3eeec700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea380ad5b0 con 0x7fea3806b5b0 2024-09-13T18:54:01.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.151+0000 7fea3e6eb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea38159060 0x7fea38156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:01.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.151+0000 7fea3deea700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fea3806b5b0 0x7fea381519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:01.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.151+0000 7fea3d6e9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea380b38d0 0x7fea38149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:01.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.151+0000 7fea3deea700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fea3806b5b0 0x7fea381519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:54104/0 (socket says 172.21.15.7:54104) 2024-09-13T18:54:01.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.151+0000 7fea3deea700 1 -- 172.21.15.7:0/2361416870 learned_addr learned my addr 172.21.15.7:0/2361416870 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:01.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.151+0000 7fea3deea700 1 -- 172.21.15.7:0/2361416870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea380b38d0 msgr2=0x7fea38149a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:01.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.151+0000 7fea3deea700 1 --2- 172.21.15.7:0/2361416870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea380b38d0 0x7fea38149a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.151+0000 7fea3deea700 1 -- 172.21.15.7:0/2361416870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea38159060 msgr2=0x7fea38156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:01.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.151+0000 7fea3deea700 1 --2- 172.21.15.7:0/2361416870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea38159060 0x7fea38156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.151+0000 7fea3deea700 1 -- 172.21.15.7:0/2361416870 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2800b820 con 0x7fea3806b5b0 2024-09-13T18:54:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.155+0000 7fea3deea700 1 --2- 172.21.15.7:0/2361416870 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fea3806b5b0 0x7fea381519a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fea3400c4a0 tx=0x7fea3400c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.155+0000 7fea2e7fc700 1 -- 172.21.15.7:0/2361416870 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea340052d0 con 0x7fea3806b5b0 2024-09-13T18:54:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.155+0000 7fea2e7fc700 1 -- 172.21.15.7:0/2361416870 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea34005430 con 0x7fea3806b5b0 2024-09-13T18:54:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.155+0000 7fea2e7fc700 1 -- 172.21.15.7:0/2361416870 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea340055a0 con 0x7fea3806b5b0 2024-09-13T18:54:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.155+0000 7fea3eeec700 1 -- 172.21.15.7:0/2361416870 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea380ad890 con 0x7fea3806b5b0 2024-09-13T18:54:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.155+0000 7fea3eeec700 1 -- 172.21.15.7:0/2361416870 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea380ade50 con 0x7fea3806b5b0 2024-09-13T18:54:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.155+0000 7fea3eeec700 1 -- 172.21.15.7:0/2361416870 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3806a740 con 0x7fea3806b5b0 2024-09-13T18:54:01.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.155+0000 7fea2e7fc700 1 -- 172.21.15.7:0/2361416870 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fea34003900 con 0x7fea3806b5b0 2024-09-13T18:54:01.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.155+0000 7fea2e7fc700 1 --2- 172.21.15.7:0/2361416870 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fea1c066730 0x7fea1c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:01.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.159+0000 7fea2e7fc700 1 -- 172.21.15.7:0/2361416870 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fea34083680 con 0x7fea3806b5b0 2024-09-13T18:54:01.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.159+0000 7fea3d6e9700 1 --2- 172.21.15.7:0/2361416870 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fea1c066730 0x7fea1c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:01.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.159+0000 7fea3d6e9700 1 --2- 172.21.15.7:0/2361416870 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fea1c066730 0x7fea1c068bf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fea28019040 tx=0x7fea28014000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:01.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.163+0000 7fea2e7fc700 1 -- 172.21.15.7:0/2361416870 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fea34050d00 con 0x7fea3806b5b0 2024-09-13T18:54:01.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.395+0000 7fea3eeec700 1 -- 172.21.15.7:0/2361416870 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj"]} v 0) v1 -- 0x7fea380ae010 con 0x7fea3806b5b0 2024-09-13T18:54:01.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.547+0000 7fea2e7fc700 1 -- 172.21.15.7:0/2361416870 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_max_items_per_obj"]}]=0 v108) v1 ==== 147+0+0 (secure 0 0 0) 0x7fea34054320 con 0x7fea3806b5b0 2024-09-13T18:54:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.551+0000 7fea3eeec700 1 -- 172.21.15.7:0/2361416870 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fea1c066730 msgr2=0x7fea1c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.551+0000 7fea3eeec700 1 --2- 172.21.15.7:0/2361416870 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fea1c066730 0x7fea1c068bf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fea28019040 tx=0x7fea28014000 comp rx=0 tx=0).stop 2024-09-13T18:54:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.551+0000 7fea3eeec700 1 -- 172.21.15.7:0/2361416870 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fea3806b5b0 msgr2=0x7fea381519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.551+0000 7fea3eeec700 1 --2- 172.21.15.7:0/2361416870 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fea3806b5b0 0x7fea381519a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fea3400c4a0 tx=0x7fea3400c7b0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.551+0000 7fea3eeec700 1 -- 172.21.15.7:0/2361416870 shutdown_connections 2024-09-13T18:54:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.551+0000 7fea3eeec700 1 --2- 172.21.15.7:0/2361416870 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fea1c066730 0x7fea1c068bf0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.551+0000 7fea3eeec700 1 --2- 172.21.15.7:0/2361416870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea38159060 0x7fea38156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.551+0000 7fea3eeec700 1 --2- 172.21.15.7:0/2361416870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fea380b38d0 0x7fea38149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.551+0000 7fea3eeec700 1 --2- 172.21.15.7:0/2361416870 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fea3806b5b0 0x7fea381519a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.551+0000 7fea3eeec700 1 -- 172.21.15.7:0/2361416870 >> 172.21.15.7:0/2361416870 conn(0x7fea3806d900 msgr2=0x7fea380b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.551+0000 7fea3eeec700 1 -- 172.21.15.7:0/2361416870 shutdown_connections 2024-09-13T18:54:01.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.551+0000 7fea3eeec700 1 -- 172.21.15.7:0/2361416870 wait complete. 2024-09-13T18:54:01.577 INFO:tasks.cephfs_test_runner:The maximum number of openfiles omap objects keys are now equal to ... ok 2024-09-13T18:54:01.577 INFO:tasks.cephfs_test_runner:Starting test: test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable) Opening a file should increment omap_total_updates by 1. 2024-09-13T18:54:01.577 INFO:tasks.cephfs_test_runner:test_perf_counters (tasks.cephfs.test_openfiletable.OpenFileTable) 2024-09-13T18:54:01.578 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:54:01.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.687+0000 7f40da39d700 1 Processor -- start 2024-09-13T18:54:01.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.687+0000 7f40da39d700 1 -- start start 2024-09-13T18:54:01.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40da39d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40d40712b0 0x7f40d406d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40da39d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d406c3c0 con 0x7f40d4159870 2024-09-13T18:54:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40da39d700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d406c600 con 0x7f40d40b3f90 2024-09-13T18:54:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40da39d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d406dbe0 con 0x7f40d40712b0 2024-09-13T18:54:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40d9b9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40d40712b0 0x7f40d406d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40d939b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f40d4159870 0x7f40d406bcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57806/0 (socket says 172.21.15.7:57806) 2024-09-13T18:54:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40d939b700 1 -- 172.21.15.7:0/636371369 learned_addr learned my addr 172.21.15.7:0/636371369 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40d9b9c700 1 -- 172.21.15.7:0/636371369 >> v1:172.21.15.171:6789/0 conn(0x7f40d40b3f90 legacy=0x7f40d4070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40d9b9c700 1 -- 172.21.15.7:0/636371369 >> v1:172.21.15.7:6789/0 conn(0x7f40d4159870 legacy=0x7f40d406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40d9b9c700 1 -- 172.21.15.7:0/636371369 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d406c8e0 con 0x7f40d40712b0 2024-09-13T18:54:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40d9b9c700 1 --2- 172.21.15.7:0/636371369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d40712b0 0x7f40d406d6a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009dc0 tx=0x7f40cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6bc9f47380a17be server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40cbfff700 1 -- 172.21.15.7:0/636371369 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc00e070 con 0x7f40d40712b0 2024-09-13T18:54:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40cbfff700 1 -- 172.21.15.7:0/636371369 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc00bb90 con 0x7f40d40712b0 2024-09-13T18:54:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40cbfff700 1 -- 172.21.15.7:0/636371369 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc012430 con 0x7f40d40712b0 2024-09-13T18:54:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40da39d700 1 -- 172.21.15.7:0/636371369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d40712b0 msgr2=0x7f40d406d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40da39d700 1 --2- 172.21.15.7:0/636371369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d40712b0 0x7f40d406d6a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f40cc009dc0 tx=0x7f40cc00b330 comp rx=0 tx=0).stop 2024-09-13T18:54:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40da39d700 1 -- 172.21.15.7:0/636371369 shutdown_connections 2024-09-13T18:54:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40da39d700 1 --2- 172.21.15.7:0/636371369 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d40712b0 0x7f40d406d6a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40da39d700 1 -- 172.21.15.7:0/636371369 >> 172.21.15.7:0/636371369 conn(0x7f40d40b1630 msgr2=0x7f40d40b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40da39d700 1 -- 172.21.15.7:0/636371369 shutdown_connections 2024-09-13T18:54:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.691+0000 7f40da39d700 1 -- 172.21.15.7:0/636371369 wait complete. 2024-09-13T18:54:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.695+0000 7f40da39d700 1 Processor -- start 2024-09-13T18:54:01.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.695+0000 7f40da39d700 1 -- start start 2024-09-13T18:54:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.695+0000 7f40da39d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f40d40712b0 0x7f40d414c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.695+0000 7f40da39d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40d40b3f90 0x7f40d414f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.695+0000 7f40da39d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d4159870 0x7f40d4153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.695+0000 7f40da39d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d414fef0 con 0x7f40d4159870 2024-09-13T18:54:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.695+0000 7f40da39d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d41f05c0 con 0x7f40d40b3f90 2024-09-13T18:54:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.695+0000 7f40da39d700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d41f08a0 con 0x7f40d40712b0 2024-09-13T18:54:01.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.695+0000 7f40d9b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d4159870 0x7f40d4153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:01.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.695+0000 7f40d9b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d4159870 0x7f40d4153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39166/0 (socket says 172.21.15.7:39166) 2024-09-13T18:54:01.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.695+0000 7f40d9b9c700 1 -- 172.21.15.7:0/1075802303 learned_addr learned my addr 172.21.15.7:0/1075802303 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:01.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.695+0000 7f40d8b9a700 1 --2- 172.21.15.7:0/1075802303 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40d40b3f90 0x7f40d414f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:01.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.695+0000 7f40d939b700 1 --2- 172.21.15.7:0/1075802303 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f40d40712b0 0x7f40d414c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:01.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.695+0000 7f40d9b9c700 1 -- 172.21.15.7:0/1075802303 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f40d40712b0 msgr2=0x7f40d414c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:01.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.699+0000 7f40d9b9c700 1 --2- 172.21.15.7:0/1075802303 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f40d40712b0 0x7f40d414c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.699+0000 7f40d9b9c700 1 -- 172.21.15.7:0/1075802303 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40d40b3f90 msgr2=0x7f40d414f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:01.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.699+0000 7f40d9b9c700 1 --2- 172.21.15.7:0/1075802303 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40d40b3f90 0x7f40d414f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.699+0000 7f40d9b9c700 1 -- 172.21.15.7:0/1075802303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40cc009a70 con 0x7f40d4159870 2024-09-13T18:54:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.699+0000 7f40d9b9c700 1 --2- 172.21.15.7:0/1075802303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d4159870 0x7f40d4153e40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00b370 tx=0x7f40cc004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.699+0000 7f40c9ffb700 1 -- 172.21.15.7:0/1075802303 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc004570 con 0x7f40d4159870 2024-09-13T18:54:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.699+0000 7f40c9ffb700 1 -- 172.21.15.7:0/1075802303 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc020070 con 0x7f40d4159870 2024-09-13T18:54:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.699+0000 7f40da39d700 1 -- 172.21.15.7:0/1075802303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d41f0b80 con 0x7f40d4159870 2024-09-13T18:54:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.699+0000 7f40c9ffb700 1 -- 172.21.15.7:0/1075802303 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc004930 con 0x7f40d4159870 2024-09-13T18:54:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.699+0000 7f40da39d700 1 -- 172.21.15.7:0/1075802303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d414c8f0 con 0x7f40d4159870 2024-09-13T18:54:01.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.699+0000 7f40da39d700 1 -- 172.21.15.7:0/1075802303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d40b4ea0 con 0x7f40d4159870 2024-09-13T18:54:01.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.699+0000 7f40c9ffb700 1 -- 172.21.15.7:0/1075802303 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f40cc022030 con 0x7f40d4159870 2024-09-13T18:54:01.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.703+0000 7f40c9ffb700 1 --2- 172.21.15.7:0/1075802303 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f40b8066760 0x7f40b8068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:01.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.703+0000 7f40d939b700 1 --2- 172.21.15.7:0/1075802303 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f40b8066760 0x7f40b8068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:01.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.703+0000 7f40c9ffb700 1 -- 172.21.15.7:0/1075802303 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f40cc085d40 con 0x7f40d4159870 2024-09-13T18:54:01.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.703+0000 7f40d939b700 1 --2- 172.21.15.7:0/1075802303 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f40b8066760 0x7f40b8068c20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f40c4005930 tx=0x7f40c4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:01.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.707+0000 7f40c9ffb700 1 -- 172.21.15.7:0/1075802303 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f40cc056e70 con 0x7f40d4159870 2024-09-13T18:54:01.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.887+0000 7f40da39d700 1 -- 172.21.15.7:0/1075802303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40d406d6a0 con 0x7f40d4159870 2024-09-13T18:54:01.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.891+0000 7f40c9ffb700 1 -- 172.21.15.7:0/1075802303 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+13062 (secure 0 0 0) 0x7f40cc056a00 con 0x7f40d4159870 2024-09-13T18:54:01.898 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:01.898 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":43,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:54:00.538273+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/127797392":"2024-09-14T18:53:57.517430+0000","172.21.15.7:6834/127797392":"2024-09-14T18:53:57.517430+0000","172.21.15.7:6835/2843879273":"2024-09-14T18:53:26.228249+0000","172.21.15.7:6834/2843879273":"2024-09-14T18:53:26.228249+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:54:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.891+0000 7f40da39d700 1 -- 172.21.15.7:0/1075802303 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f40b8066760 msgr2=0x7f40b8068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.891+0000 7f40da39d700 1 --2- 172.21.15.7:0/1075802303 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f40b8066760 0x7f40b8068c20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f40c4005930 tx=0x7f40c4009040 comp rx=0 tx=0).stop 2024-09-13T18:54:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.891+0000 7f40da39d700 1 -- 172.21.15.7:0/1075802303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d4159870 msgr2=0x7f40d4153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.891+0000 7f40da39d700 1 --2- 172.21.15.7:0/1075802303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d4159870 0x7f40d4153e40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00b370 tx=0x7f40cc004af0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.891+0000 7f40da39d700 1 -- 172.21.15.7:0/1075802303 shutdown_connections 2024-09-13T18:54:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.891+0000 7f40da39d700 1 --2- 172.21.15.7:0/1075802303 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f40b8066760 0x7f40b8068c20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.891+0000 7f40da39d700 1 --2- 172.21.15.7:0/1075802303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40d4159870 0x7f40d4153e40 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.891+0000 7f40da39d700 1 --2- 172.21.15.7:0/1075802303 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f40d40b3f90 0x7f40d414f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.891+0000 7f40da39d700 1 --2- 172.21.15.7:0/1075802303 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f40d40712b0 0x7f40d414c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.891+0000 7f40da39d700 1 -- 172.21.15.7:0/1075802303 >> 172.21.15.7:0/1075802303 conn(0x7f40d40b1630 msgr2=0x7f40d40b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.891+0000 7f40da39d700 1 -- 172.21.15.7:0/1075802303 shutdown_connections 2024-09-13T18:54:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:01.891+0000 7f40da39d700 1 -- 172.21.15.7:0/1075802303 wait complete. 2024-09-13T18:54:01.914 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:54:01.914 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:54:02.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.027+0000 7f785759e700 1 Processor -- start 2024-09-13T18:54:02.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.027+0000 7f785759e700 1 -- start start 2024-09-13T18:54:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.027+0000 7f785759e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f78580712b0 0x7f7858159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.027+0000 7f785759e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f785815a460 0x7f785806c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.027+0000 7f785759e700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f785815ac50 con 0x7f78581573e0 2024-09-13T18:54:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.027+0000 7f785759e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78580cc1b0 con 0x7f78580712b0 2024-09-13T18:54:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.027+0000 7f785759e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78580cb3f0 con 0x7f785815a460 2024-09-13T18:54:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.027+0000 7f7855d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f78580712b0 0x7f7858159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.027+0000 7f7855d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f78580712b0 0x7f7858159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39168/0 (socket says 172.21.15.7:39168) 2024-09-13T18:54:02.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.027+0000 7f7855d9b700 1 -- 172.21.15.7:0/2049939165 learned_addr learned my addr 172.21.15.7:0/2049939165 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.027+0000 7f7856d9d700 1 --2- 172.21.15.7:0/2049939165 >> v2:172.21.15.171:3300/0 conn(0x7f785815a460 0x7f785806c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.027+0000 7f7855d9b700 1 -- 172.21.15.7:0/2049939165 >> v2:172.21.15.171:3300/0 conn(0x7f785815a460 msgr2=0x7f785806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.027+0000 7f7855d9b700 1 --2- 172.21.15.7:0/2049939165 >> v2:172.21.15.171:3300/0 conn(0x7f785815a460 0x7f785806c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.027+0000 7f7855d9b700 1 -- 172.21.15.7:0/2049939165 >> v1:172.21.15.171:6789/0 conn(0x7f78581573e0 legacy=0x7f7858070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.027+0000 7f7855d9b700 1 -- 172.21.15.7:0/2049939165 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78580cb6d0 con 0x7f78580712b0 2024-09-13T18:54:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.027+0000 7f7855d9b700 1 --2- 172.21.15.7:0/2049939165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78580712b0 0x7f7858159f20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f785806b990 tx=0x7f7848009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7669ce5d5f31413c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.031+0000 7f785559a700 1 -- 172.21.15.7:0/2049939165 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784800b8a0 con 0x7f78580712b0 2024-09-13T18:54:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.031+0000 7f785559a700 1 -- 172.21.15.7:0/2049939165 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784800ba00 con 0x7f78580712b0 2024-09-13T18:54:02.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.031+0000 7f785559a700 1 -- 172.21.15.7:0/2049939165 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784800bd90 con 0x7f78580712b0 2024-09-13T18:54:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.031+0000 7f785759e700 1 -- 172.21.15.7:0/2049939165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78580712b0 msgr2=0x7f7858159f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.031+0000 7f785759e700 1 --2- 172.21.15.7:0/2049939165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78580712b0 0x7f7858159f20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f785806b990 tx=0x7f7848009bf0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.031+0000 7f785759e700 1 -- 172.21.15.7:0/2049939165 shutdown_connections 2024-09-13T18:54:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.031+0000 7f785759e700 1 --2- 172.21.15.7:0/2049939165 >> v2:172.21.15.171:3300/0 conn(0x7f785815a460 0x7f785806c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.031+0000 7f785759e700 1 --2- 172.21.15.7:0/2049939165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78580712b0 0x7f7858159f20 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.031+0000 7f785759e700 1 -- 172.21.15.7:0/2049939165 >> 172.21.15.7:0/2049939165 conn(0x7f78580b0e10 msgr2=0x7f78580b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.031+0000 7f785759e700 1 -- 172.21.15.7:0/2049939165 shutdown_connections 2024-09-13T18:54:02.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.031+0000 7f785759e700 1 -- 172.21.15.7:0/2049939165 wait complete. 2024-09-13T18:54:02.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.031+0000 7f785759e700 1 Processor -- start 2024-09-13T18:54:02.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.031+0000 7f785759e700 1 -- start start 2024-09-13T18:54:02.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f785759e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78580712b0 0x7f785814f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f785759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78581573e0 0x7f78580af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f785759e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f785815a460 0x7f78580af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f785759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78580afd40 con 0x7f78581573e0 2024-09-13T18:54:02.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f785759e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78580ae590 con 0x7f78580712b0 2024-09-13T18:54:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f785759e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78580aee30 con 0x7f785815a460 2024-09-13T18:54:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f7855d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78581573e0 0x7f78580af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f7855d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78581573e0 0x7f78580af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39178/0 (socket says 172.21.15.7:39178) 2024-09-13T18:54:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f7855d9b700 1 -- 172.21.15.7:0/3734112727 learned_addr learned my addr 172.21.15.7:0/3734112727 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f785659c700 1 --2- 172.21.15.7:0/3734112727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78580712b0 0x7f785814f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f7856d9d700 1 --2- 172.21.15.7:0/3734112727 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f785815a460 0x7f78580af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f7855d9b700 1 -- 172.21.15.7:0/3734112727 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f785815a460 msgr2=0x7f78580af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f7855d9b700 1 --2- 172.21.15.7:0/3734112727 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f785815a460 0x7f78580af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f7855d9b700 1 -- 172.21.15.7:0/3734112727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78580712b0 msgr2=0x7f785814f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:02.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f7855d9b700 1 --2- 172.21.15.7:0/3734112727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78580712b0 0x7f785814f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f7855d9b700 1 -- 172.21.15.7:0/3734112727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7848009570 con 0x7f78581573e0 2024-09-13T18:54:02.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f785659c700 1 --2- 172.21.15.7:0/3734112727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78580712b0 0x7f785814f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:02.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f7855d9b700 1 --2- 172.21.15.7:0/3734112727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78581573e0 0x7f78580af2c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f78580b3c80 tx=0x7f7848004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:02.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f7846ffd700 1 -- 172.21.15.7:0/3734112727 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78480038c0 con 0x7f78581573e0 2024-09-13T18:54:02.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f7846ffd700 1 -- 172.21.15.7:0/3734112727 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784801f070 con 0x7f78581573e0 2024-09-13T18:54:02.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f785759e700 1 -- 172.21.15.7:0/3734112727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785814a9b0 con 0x7f78581573e0 2024-09-13T18:54:02.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f7846ffd700 1 -- 172.21.15.7:0/3734112727 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7848003c60 con 0x7f78581573e0 2024-09-13T18:54:02.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f785759e700 1 -- 172.21.15.7:0/3734112727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78580aeac0 con 0x7f78581573e0 2024-09-13T18:54:02.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f785759e700 1 -- 172.21.15.7:0/3734112727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7858159f20 con 0x7f78581573e0 2024-09-13T18:54:02.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.035+0000 7f7846ffd700 1 -- 172.21.15.7:0/3734112727 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7848026480 con 0x7f78581573e0 2024-09-13T18:54:02.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.039+0000 7f7846ffd700 1 --2- 172.21.15.7:0/3734112727 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7838066730 0x7f7838068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.039+0000 7f7846ffd700 1 -- 172.21.15.7:0/3734112727 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f7848021070 con 0x7f78581573e0 2024-09-13T18:54:02.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.039+0000 7f785659c700 1 --2- 172.21.15.7:0/3734112727 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7838066730 0x7f7838068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.039+0000 7f785659c700 1 --2- 172.21.15.7:0/3734112727 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7838066730 0x7f7838068bf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f784c0017a0 tx=0x7f784c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:02.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.043+0000 7f7846ffd700 1 -- 172.21.15.7:0/3734112727 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7848053050 con 0x7f78581573e0 2024-09-13T18:54:02.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.215+0000 7f785759e700 1 -- 172.21.15.7:0/3734112727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7858151570 con 0x7f78581573e0 2024-09-13T18:54:02.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.215+0000 7f7846ffd700 1 -- 172.21.15.7:0/3734112727 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f784802b020 con 0x7f78581573e0 2024-09-13T18:54:02.223 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:54:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.215+0000 7f785759e700 1 -- 172.21.15.7:0/3734112727 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7838066730 msgr2=0x7f7838068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.215+0000 7f785759e700 1 --2- 172.21.15.7:0/3734112727 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7838066730 0x7f7838068bf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f784c0017a0 tx=0x7f784c009040 comp rx=0 tx=0).stop 2024-09-13T18:54:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.215+0000 7f785759e700 1 -- 172.21.15.7:0/3734112727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78581573e0 msgr2=0x7f78580af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.215+0000 7f785759e700 1 --2- 172.21.15.7:0/3734112727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78581573e0 0x7f78580af2c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f78580b3c80 tx=0x7f7848004770 comp rx=0 tx=0).stop 2024-09-13T18:54:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.215+0000 7f785759e700 1 -- 172.21.15.7:0/3734112727 shutdown_connections 2024-09-13T18:54:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.215+0000 7f785759e700 1 --2- 172.21.15.7:0/3734112727 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7838066730 0x7f7838068bf0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.215+0000 7f785759e700 1 --2- 172.21.15.7:0/3734112727 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f785815a460 0x7f78580af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.215+0000 7f785759e700 1 --2- 172.21.15.7:0/3734112727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78581573e0 0x7f78580af2c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.215+0000 7f785759e700 1 --2- 172.21.15.7:0/3734112727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78580712b0 0x7f785814f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.215+0000 7f785759e700 1 -- 172.21.15.7:0/3734112727 >> 172.21.15.7:0/3734112727 conn(0x7f78580b0e10 msgr2=0x7f785806aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:02.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.215+0000 7f785759e700 1 -- 172.21.15.7:0/3734112727 shutdown_connections 2024-09-13T18:54:02.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.215+0000 7f785759e700 1 -- 172.21.15.7:0/3734112727 wait complete. 2024-09-13T18:54:02.237 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters' 2024-09-13T18:54:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.343+0000 7f0f859ea700 1 Processor -- start 2024-09-13T18:54:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f859ea700 1 -- start start 2024-09-13T18:54:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f859ea700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f800b3ae0 0x7f0f80165590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f859ea700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f0f80161290 0x7f0f801616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f859ea700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f80071f30 con 0x7f0f80160eb0 2024-09-13T18:54:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f859ea700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f801622e0 con 0x7f0f800b3ae0 2024-09-13T18:54:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f859ea700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f80162000 con 0x7f0f80161290 2024-09-13T18:54:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f849e8700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0f80160eb0 0x7f0f800b33d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57822/0 (socket says 172.21.15.7:57822) 2024-09-13T18:54:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f849e8700 1 -- 172.21.15.7:0/2745088268 learned_addr learned my addr 172.21.15.7:0/2745088268 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7ffff700 1 --2- 172.21.15.7:0/2745088268 >> v2:172.21.15.7:3300/0 conn(0x7f0f800b3ae0 0x7f0f80165590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f851e9700 1 --2- 172.21.15.7:0/2745088268 >> v2:172.21.15.171:3300/0 conn(0x7f0f80161290 0x7f0f801616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 -- 172.21.15.7:0/2745088268 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643352806 0 0) 0x7f0f80071f30 con 0x7f0f80160eb0 2024-09-13T18:54:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 -- 172.21.15.7:0/2745088268 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f6c0035d0 con 0x7f0f80160eb0 2024-09-13T18:54:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 -- 172.21.15.7:0/2745088268 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 862698041 0 0) 0x7f0f6c0035d0 con 0x7f0f80160eb0 2024-09-13T18:54:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 -- 172.21.15.7:0/2745088268 >> v2:172.21.15.171:3300/0 conn(0x7f0f80161290 msgr2=0x7f0f801616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 --2- 172.21.15.7:0/2745088268 >> v2:172.21.15.171:3300/0 conn(0x7f0f80161290 0x7f0f801616c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 -- 172.21.15.7:0/2745088268 >> v2:172.21.15.7:3300/0 conn(0x7f0f800b3ae0 msgr2=0x7f0f80165590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 --2- 172.21.15.7:0/2745088268 >> v2:172.21.15.7:3300/0 conn(0x7f0f800b3ae0 0x7f0f80165590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 -- 172.21.15.7:0/2745088268 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f80164ab0 con 0x7f0f80160eb0 2024-09-13T18:54:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 -- 172.21.15.7:0/2745088268 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7f0f780019f0 con 0x7f0f80160eb0 2024-09-13T18:54:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 -- 172.21.15.7:0/2745088268 >> v1:172.21.15.7:6789/0 conn(0x7f0f80160eb0 legacy=0x7f0f800b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 --2- 172.21.15.7:0/2745088268 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f6c003a20 0x7f0f6c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 --2- 172.21.15.7:0/2745088268 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f6c004870 0x7f0f6c004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 --2- 172.21.15.7:0/2745088268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f6c005230 0x7f0f6c00ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 -- 172.21.15.7:0/2745088268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f801622e0 con 0x7f0f6c005230 2024-09-13T18:54:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 -- 172.21.15.7:0/2745088268 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80162000 con 0x7f0f6c004870 2024-09-13T18:54:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 -- 172.21.15.7:0/2745088268 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c00bfc0 con 0x7f0f6c003a20 2024-09-13T18:54:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7ffff700 1 --2- 172.21.15.7:0/2745088268 >> v2:172.21.15.7:3300/0 conn(0x7f0f800b3ae0 0x7f0f80165590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f851e9700 1 --2- 172.21.15.7:0/2745088268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f6c005230 0x7f0f6c00ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f849e8700 1 --2- 172.21.15.7:0/2745088268 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f6c003a20 0x7f0f6c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f851e9700 1 -- 172.21.15.7:0/2745088268 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f6c003a20 msgr2=0x7f0f6c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7ffff700 1 --2- 172.21.15.7:0/2745088268 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f6c004870 0x7f0f6c004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f851e9700 1 --2- 172.21.15.7:0/2745088268 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f6c003a20 0x7f0f6c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f851e9700 1 -- 172.21.15.7:0/2745088268 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f6c004870 msgr2=0x7f0f6c004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f851e9700 1 --2- 172.21.15.7:0/2745088268 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f6c004870 0x7f0f6c004cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f851e9700 1 -- 172.21.15.7:0/2745088268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6c0035d0 con 0x7f0f6c005230 2024-09-13T18:54:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f849e8700 1 --2- 172.21.15.7:0/2745088268 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f6c003a20 0x7f0f6c003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:54:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f851e9700 1 -- 172.21.15.7:0/2745088268 reap_dead start 2024-09-13T18:54:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f851e9700 1 --2- 172.21.15.7:0/2745088268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f6c005230 0x7f0f6c00ba30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009b70 tx=0x7f0f6c0058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199e698af7ccff1c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 -- 172.21.15.7:0/2745088268 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f70004020 con 0x7f0f6c005230 2024-09-13T18:54:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 -- 172.21.15.7:0/2745088268 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f70004180 con 0x7f0f6c005230 2024-09-13T18:54:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f7f7fe700 1 -- 172.21.15.7:0/2745088268 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f700044f0 con 0x7f0f6c005230 2024-09-13T18:54:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f859ea700 1 -- 172.21.15.7:0/2745088268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f6c005230 msgr2=0x7f0f6c00ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f859ea700 1 --2- 172.21.15.7:0/2745088268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f6c005230 0x7f0f6c00ba30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009b70 tx=0x7f0f6c0058a0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f859ea700 1 -- 172.21.15.7:0/2745088268 shutdown_connections 2024-09-13T18:54:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f859ea700 1 --2- 172.21.15.7:0/2745088268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f6c005230 0x7f0f6c00ba30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f859ea700 1 -- 172.21.15.7:0/2745088268 >> 172.21.15.7:0/2745088268 conn(0x7f0f800b0f00 msgr2=0x7f0f800b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f859ea700 1 -- 172.21.15.7:0/2745088268 shutdown_connections 2024-09-13T18:54:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.347+0000 7f0f859ea700 1 -- 172.21.15.7:0/2745088268 wait complete. 2024-09-13T18:54:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f859ea700 1 Processor -- start 2024-09-13T18:54:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f859ea700 1 -- start start 2024-09-13T18:54:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f859ea700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f6c005230 0x7f0f80158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f859ea700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f8006b310 0x7f0f8006c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f859ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f800b1920 0x7f0f800b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f859ea700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f800b22e0 con 0x7f0f800b1920 2024-09-13T18:54:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f859ea700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80158fa0 con 0x7f0f6c005230 2024-09-13T18:54:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f859ea700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8006ca40 con 0x7f0f8006b310 2024-09-13T18:54:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f851e9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f800b1920 0x7f0f800b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f851e9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f800b1920 0x7f0f800b1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39206/0 (socket says 172.21.15.7:39206) 2024-09-13T18:54:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f851e9700 1 -- 172.21.15.7:0/3998326556 learned_addr learned my addr 172.21.15.7:0/3998326556 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f7ffff700 1 --2- 172.21.15.7:0/3998326556 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f8006b310 0x7f0f8006c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f849e8700 1 --2- 172.21.15.7:0/3998326556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f6c005230 0x7f0f80158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f851e9700 1 -- 172.21.15.7:0/3998326556 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f8006b310 msgr2=0x7f0f8006c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f851e9700 1 --2- 172.21.15.7:0/3998326556 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f8006b310 0x7f0f8006c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f851e9700 1 -- 172.21.15.7:0/3998326556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f6c005230 msgr2=0x7f0f80158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f851e9700 1 --2- 172.21.15.7:0/3998326556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f6c005230 0x7f0f80158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f851e9700 1 -- 172.21.15.7:0/3998326556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8006cd20 con 0x7f0f800b1920 2024-09-13T18:54:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f851e9700 1 --2- 172.21.15.7:0/3998326556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f800b1920 0x7f0f800b1da0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0f80071f00 tx=0x7f0f700047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f7d7fa700 1 -- 172.21.15.7:0/3998326556 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f70003ad0 con 0x7f0f800b1920 2024-09-13T18:54:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f7d7fa700 1 -- 172.21.15.7:0/3998326556 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f70003c50 con 0x7f0f800b1920 2024-09-13T18:54:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f7d7fa700 1 -- 172.21.15.7:0/3998326556 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f70004000 con 0x7f0f800b1920 2024-09-13T18:54:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f859ea700 1 -- 172.21.15.7:0/3998326556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8006d000 con 0x7f0f800b1920 2024-09-13T18:54:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.351+0000 7f0f859ea700 1 -- 172.21.15.7:0/3998326556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f800a9800 con 0x7f0f800b1920 2024-09-13T18:54:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.355+0000 7f0f859ea700 1 -- 172.21.15.7:0/3998326556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f800b33d0 con 0x7f0f800b1920 2024-09-13T18:54:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.355+0000 7f0f7d7fa700 1 -- 172.21.15.7:0/3998326556 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0f70013470 con 0x7f0f800b1920 2024-09-13T18:54:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.355+0000 7f0f7d7fa700 1 --2- 172.21.15.7:0/3998326556 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0f60066730 0x7f0f60068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.355+0000 7f0f7d7fa700 1 -- 172.21.15.7:0/3998326556 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f0f70084dd0 con 0x7f0f800b1920 2024-09-13T18:54:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.355+0000 7f0f849e8700 1 --2- 172.21.15.7:0/3998326556 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0f60066730 0x7f0f60068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.355+0000 7f0f849e8700 1 --2- 172.21.15.7:0/3998326556 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0f60066730 0x7f0f60068bf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0f78001330 tx=0x7f0f78009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:02.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.363+0000 7f0f7d7fa700 1 -- 172.21.15.7:0/3998326556 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0f70052590 con 0x7f0f800b1920 2024-09-13T18:54:02.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:02.599+0000 7f0f859ea700 1 -- 172.21.15.7:0/3998326556 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters"]} v 0) v1 -- 0x7f0f8015c070 con 0x7f0f800b1920 2024-09-13T18:54:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.559+0000 7f0f7d7fa700 1 -- 172.21.15.7:0/3998326556 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters"]}]=0 v110) v1 ==== 146+0+0 (secure 0 0 0) 0x7f0f70055bb0 con 0x7f0f800b1920 2024-09-13T18:54:03.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.563+0000 7f0f859ea700 1 -- 172.21.15.7:0/3998326556 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0f60066730 msgr2=0x7f0f60068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:03.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.563+0000 7f0f859ea700 1 --2- 172.21.15.7:0/3998326556 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0f60066730 0x7f0f60068bf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0f78001330 tx=0x7f0f78009040 comp rx=0 tx=0).stop 2024-09-13T18:54:03.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.563+0000 7f0f859ea700 1 -- 172.21.15.7:0/3998326556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f800b1920 msgr2=0x7f0f800b1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:03.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.563+0000 7f0f859ea700 1 --2- 172.21.15.7:0/3998326556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f800b1920 0x7f0f800b1da0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0f80071f00 tx=0x7f0f700047e0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.563+0000 7f0f859ea700 1 -- 172.21.15.7:0/3998326556 shutdown_connections 2024-09-13T18:54:03.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.563+0000 7f0f859ea700 1 --2- 172.21.15.7:0/3998326556 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0f60066730 0x7f0f60068bf0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.563+0000 7f0f859ea700 1 --2- 172.21.15.7:0/3998326556 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f800b1920 0x7f0f800b1da0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.563+0000 7f0f859ea700 1 --2- 172.21.15.7:0/3998326556 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0f8006b310 0x7f0f8006c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.563+0000 7f0f859ea700 1 --2- 172.21.15.7:0/3998326556 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0f6c005230 0x7f0f80158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.563+0000 7f0f859ea700 1 -- 172.21.15.7:0/3998326556 >> 172.21.15.7:0/3998326556 conn(0x7f0f800b0f00 msgr2=0x7f0f800cdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:03.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.563+0000 7f0f859ea700 1 -- 172.21.15.7:0/3998326556 shutdown_connections 2024-09-13T18:54:03.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.563+0000 7f0f859ea700 1 -- 172.21.15.7:0/3998326556 wait complete. 2024-09-13T18:54:03.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T18:54:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac3fff700 1 Processor -- start 2024-09-13T18:54:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac3fff700 1 -- start start 2024-09-13T18:54:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faac415cda0 0x7faac406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac3fff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7faac40ceb70 0x7faac40b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac3fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faac406a520 0x7faac406a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac3fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac406be40 con 0x7faac415cda0 2024-09-13T18:54:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac3fff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac40b8430 con 0x7faac40ceb70 2024-09-13T18:54:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac3fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac406ae90 con 0x7faac406a520 2024-09-13T18:54:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faac415cda0 0x7faac406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac2ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faac415cda0 0x7faac406b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39210/0 (socket says 172.21.15.7:39210) 2024-09-13T18:54:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac2ffd700 1 -- 172.21.15.7:0/2718858684 learned_addr learned my addr 172.21.15.7:0/2718858684 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac27fc700 1 --2- 172.21.15.7:0/2718858684 >> v2:172.21.15.171:3300/0 conn(0x7faac40ceb70 0x7faac40b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:03.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac37fe700 1 --2- 172.21.15.7:0/2718858684 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faac406a520 0x7faac406a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:03.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac2ffd700 1 -- 172.21.15.7:0/2718858684 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faac406a520 msgr2=0x7faac406a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:03.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac2ffd700 1 --2- 172.21.15.7:0/2718858684 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faac406a520 0x7faac406a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac2ffd700 1 -- 172.21.15.7:0/2718858684 >> v2:172.21.15.171:3300/0 conn(0x7faac40ceb70 msgr2=0x7faac40b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:03.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac2ffd700 1 --2- 172.21.15.7:0/2718858684 >> v2:172.21.15.171:3300/0 conn(0x7faac40ceb70 0x7faac40b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac2ffd700 1 -- 172.21.15.7:0/2718858684 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac40cf350 con 0x7faac415cda0 2024-09-13T18:54:03.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.711+0000 7faac27fc700 1 --2- 172.21.15.7:0/2718858684 >> v2:172.21.15.171:3300/0 conn(0x7faac40ceb70 0x7faac40b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:03.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac2ffd700 1 --2- 172.21.15.7:0/2718858684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac415cda0 0x7faac406b900 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7faac406cca0 tx=0x7faab8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6b761fbe8554bd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:03.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac1ffb700 1 -- 172.21.15.7:0/2718858684 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab800b8a0 con 0x7faac415cda0 2024-09-13T18:54:03.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac1ffb700 1 -- 172.21.15.7:0/2718858684 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab800ba00 con 0x7faac415cda0 2024-09-13T18:54:03.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac1ffb700 1 -- 172.21.15.7:0/2718858684 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab800bd90 con 0x7faac415cda0 2024-09-13T18:54:03.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac3fff700 1 -- 172.21.15.7:0/2718858684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac415cda0 msgr2=0x7faac406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:03.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac3fff700 1 --2- 172.21.15.7:0/2718858684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac415cda0 0x7faac406b900 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7faac406cca0 tx=0x7faab8009bf0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac3fff700 1 -- 172.21.15.7:0/2718858684 shutdown_connections 2024-09-13T18:54:03.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac3fff700 1 --2- 172.21.15.7:0/2718858684 >> v2:172.21.15.171:3300/0 conn(0x7faac40ceb70 0x7faac40b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac3fff700 1 --2- 172.21.15.7:0/2718858684 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faac406a520 0x7faac406a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac3fff700 1 --2- 172.21.15.7:0/2718858684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac415cda0 0x7faac406b900 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac3fff700 1 -- 172.21.15.7:0/2718858684 >> 172.21.15.7:0/2718858684 conn(0x7faac40d2cb0 msgr2=0x7faac40d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:03.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac3fff700 1 -- 172.21.15.7:0/2718858684 shutdown_connections 2024-09-13T18:54:03.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac3fff700 1 -- 172.21.15.7:0/2718858684 wait complete. 2024-09-13T18:54:03.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac3fff700 1 Processor -- start 2024-09-13T18:54:03.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac3fff700 1 -- start start 2024-09-13T18:54:03.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac3fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faac406a520 0x7faac41572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:03.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac40ceb70 0x7faac40c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:03.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac3fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faac415cda0 0x7faac40cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac3fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac4157810 con 0x7faac40ceb70 2024-09-13T18:54:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac3fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac4156230 con 0x7faac415cda0 2024-09-13T18:54:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac3fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac4156510 con 0x7faac406a520 2024-09-13T18:54:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.715+0000 7faac27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac40ceb70 0x7faac40c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.719+0000 7faac27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac40ceb70 0x7faac40c9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39220/0 (socket says 172.21.15.7:39220) 2024-09-13T18:54:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.719+0000 7faac27fc700 1 -- 172.21.15.7:0/2423062423 learned_addr learned my addr 172.21.15.7:0/2423062423 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.719+0000 7faac37fe700 1 --2- 172.21.15.7:0/2423062423 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faac415cda0 0x7faac40cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.719+0000 7faac2ffd700 1 --2- 172.21.15.7:0/2423062423 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faac406a520 0x7faac41572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.719+0000 7faac27fc700 1 -- 172.21.15.7:0/2423062423 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faac406a520 msgr2=0x7faac41572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.719+0000 7faac27fc700 1 --2- 172.21.15.7:0/2423062423 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faac406a520 0x7faac41572d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.719+0000 7faac27fc700 1 -- 172.21.15.7:0/2423062423 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faac415cda0 msgr2=0x7faac40cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.719+0000 7faac27fc700 1 --2- 172.21.15.7:0/2423062423 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faac415cda0 0x7faac40cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.719+0000 7faac27fc700 1 -- 172.21.15.7:0/2423062423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab8009570 con 0x7faac40ceb70 2024-09-13T18:54:03.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.719+0000 7faac27fc700 1 --2- 172.21.15.7:0/2423062423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac40ceb70 0x7faac40c9170 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faab400e880 tx=0x7faab400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.719+0000 7faaab7fe700 1 -- 172.21.15.7:0/2423062423 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab400c780 con 0x7faac40ceb70 2024-09-13T18:54:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.719+0000 7faaab7fe700 1 -- 172.21.15.7:0/2423062423 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab400c8e0 con 0x7faac40ceb70 2024-09-13T18:54:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.719+0000 7faac3fff700 1 -- 172.21.15.7:0/2423062423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac41567f0 con 0x7faac40ceb70 2024-09-13T18:54:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.719+0000 7faaab7fe700 1 -- 172.21.15.7:0/2423062423 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab400cc70 con 0x7faac40ceb70 2024-09-13T18:54:03.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.719+0000 7faac3fff700 1 -- 172.21.15.7:0/2423062423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac4156db0 con 0x7faac40ceb70 2024-09-13T18:54:03.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.719+0000 7faaab7fe700 1 -- 172.21.15.7:0/2423062423 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7faab4013410 con 0x7faac40ceb70 2024-09-13T18:54:03.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.719+0000 7faac3fff700 1 -- 172.21.15.7:0/2423062423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac40cf000 con 0x7faac40ceb70 2024-09-13T18:54:03.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.723+0000 7faaab7fe700 1 --2- 172.21.15.7:0/2423062423 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7faab0066760 0x7faab0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.723+0000 7faaab7fe700 1 -- 172.21.15.7:0/2423062423 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5029+0+0 (secure 0 0 0) 0x7faab4083a80 con 0x7faac40ceb70 2024-09-13T18:54:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.723+0000 7faac2ffd700 1 --2- 172.21.15.7:0/2423062423 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7faab0066760 0x7faab0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:03.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.723+0000 7faac2ffd700 1 --2- 172.21.15.7:0/2423062423 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7faab0066760 0x7faab0068c20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faab8000c00 tx=0x7faab8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:03.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.731+0000 7faaab7fe700 1 -- 172.21.15.7:0/2423062423 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faab4051090 con 0x7faac40ceb70 2024-09-13T18:54:03.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.911+0000 7faac3fff700 1 -- 172.21.15.7:0/2423062423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7faac40d3370 con 0x7faac40ceb70 2024-09-13T18:54:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.915+0000 7faaab7fe700 1 -- 172.21.15.7:0/2423062423 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7faab40546b0 con 0x7faac40ceb70 2024-09-13T18:54:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.919+0000 7faac3fff700 1 -- 172.21.15.7:0/2423062423 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7faab0066760 msgr2=0x7faab0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.919+0000 7faac3fff700 1 --2- 172.21.15.7:0/2423062423 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7faab0066760 0x7faab0068c20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faab8000c00 tx=0x7faab8008030 comp rx=0 tx=0).stop 2024-09-13T18:54:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.919+0000 7faac3fff700 1 -- 172.21.15.7:0/2423062423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac40ceb70 msgr2=0x7faac40c9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.919+0000 7faac3fff700 1 --2- 172.21.15.7:0/2423062423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac40ceb70 0x7faac40c9170 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faab400e880 tx=0x7faab400ec40 comp rx=0 tx=0).stop 2024-09-13T18:54:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.919+0000 7faac3fff700 1 -- 172.21.15.7:0/2423062423 shutdown_connections 2024-09-13T18:54:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.919+0000 7faac3fff700 1 --2- 172.21.15.7:0/2423062423 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7faab0066760 0x7faab0068c20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.919+0000 7faac3fff700 1 --2- 172.21.15.7:0/2423062423 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7faac415cda0 0x7faac40cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.919+0000 7faac3fff700 1 --2- 172.21.15.7:0/2423062423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faac40ceb70 0x7faac40c9170 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.919+0000 7faac3fff700 1 --2- 172.21.15.7:0/2423062423 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7faac406a520 0x7faac41572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.919+0000 7faac3fff700 1 -- 172.21.15.7:0/2423062423 >> 172.21.15.7:0/2423062423 conn(0x7faac40d2cb0 msgr2=0x7faac415b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.919+0000 7faac3fff700 1 -- 172.21.15.7:0/2423062423 shutdown_connections 2024-09-13T18:54:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:03.919+0000 7faac3fff700 1 -- 172.21.15.7:0/2423062423 wait complete. 2024-09-13T18:54:03.943 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T18:54:03.943 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T18:54:03.957 DEBUG:teuthology.orchestra.run.smithi171:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T18:54:03.977 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T18:54:03.977 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:54:03.978 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:54:03.985 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16349432k,nr_inodes=4087358,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276116k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276112k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276116k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T18:54:03.985 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:54:03.985 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:54:04.028 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346336k,nr_inodes=4086584,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18533 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T18:54:04.028 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:54:04.028 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:54:04.035 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16349432k,nr_inodes=4087358,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276116k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276112k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276116k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T18:54:04.036 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:54:04.036 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:54:04.043 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346336k,nr_inodes=4086584,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18533 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T18:54:04.043 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-13T18:54:04.043 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa3323700 1 Processor -- start 2024-09-13T18:54:04.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa3323700 1 -- start start 2024-09-13T18:54:04.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa3323700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f0a9c15b890 0x7f0a9c162220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:04.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa3323700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9c162760 con 0x7f0a9c0b2b00 2024-09-13T18:54:04.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa3323700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9c15bc70 con 0x7f0a9c0cdcc0 2024-09-13T18:54:04.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa3323700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9c0ccc20 con 0x7f0a9c15b890 2024-09-13T18:54:04.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa18c0700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0a9c0b2b00 0x7f0a9c06ffe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57834/0 (socket says 172.21.15.7:57834) 2024-09-13T18:54:04.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa18c0700 1 -- 172.21.15.7:0/3354280938 learned_addr learned my addr 172.21.15.7:0/3354280938 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:04.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa10bf700 1 --2- 172.21.15.7:0/3354280938 >> v2:172.21.15.171:3300/0 conn(0x7f0a9c15b890 0x7f0a9c162220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0a93fff700 1 -- 172.21.15.7:0/3354280938 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189017767 0 0) 0x7f0a9c162760 con 0x7f0a9c0b2b00 2024-09-13T18:54:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0a93fff700 1 -- 172.21.15.7:0/3354280938 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a880035d0 con 0x7f0a9c0b2b00 2024-09-13T18:54:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa10bf700 1 -- 172.21.15.7:0/3354280938 >> v1:172.21.15.171:6789/0 conn(0x7f0a9c0cdcc0 legacy=0x7f0a9c0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa10bf700 1 -- 172.21.15.7:0/3354280938 >> v1:172.21.15.7:6789/0 conn(0x7f0a9c0b2b00 legacy=0x7f0a9c06ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:04.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa10bf700 1 -- 172.21.15.7:0/3354280938 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c0ccf00 con 0x7f0a9c15b890 2024-09-13T18:54:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa10bf700 1 --2- 172.21.15.7:0/3354280938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a9c15b890 0x7f0a9c162220 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009850 tx=0x7f0a8c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d48bcf82fb9d6f60 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0a93fff700 1 -- 172.21.15.7:0/3354280938 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c011830 con 0x7f0a9c15b890 2024-09-13T18:54:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0a93fff700 1 -- 172.21.15.7:0/3354280938 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c011990 con 0x7f0a9c15b890 2024-09-13T18:54:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0a93fff700 1 -- 172.21.15.7:0/3354280938 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c011d00 con 0x7f0a9c15b890 2024-09-13T18:54:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa3323700 1 -- 172.21.15.7:0/3354280938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a9c15b890 msgr2=0x7f0a9c162220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa3323700 1 --2- 172.21.15.7:0/3354280938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a9c15b890 0x7f0a9c162220 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009850 tx=0x7f0a8c009b60 comp rx=0 tx=0).stop 2024-09-13T18:54:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa3323700 1 -- 172.21.15.7:0/3354280938 shutdown_connections 2024-09-13T18:54:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa3323700 1 --2- 172.21.15.7:0/3354280938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a9c15b890 0x7f0a9c162220 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:04.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa3323700 1 -- 172.21.15.7:0/3354280938 >> 172.21.15.7:0/3354280938 conn(0x7f0a9c0ce980 msgr2=0x7f0a9c0ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa3323700 1 -- 172.21.15.7:0/3354280938 shutdown_connections 2024-09-13T18:54:04.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.199+0000 7f0aa3323700 1 -- 172.21.15.7:0/3354280938 wait complete. 2024-09-13T18:54:04.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa3323700 1 Processor -- start 2024-09-13T18:54:04.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa3323700 1 -- start start 2024-09-13T18:54:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa3323700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a9c0b2b00 0x7f0a9c151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa3323700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a9c0cdcc0 0x7f0a9c1f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa3323700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c15b890 0x7f0a9c1f94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa3323700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c153d40 con 0x7f0a9c15b890 2024-09-13T18:54:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa3323700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c1f7c10 con 0x7f0a9c0b2b00 2024-09-13T18:54:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa3323700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c1f7ef0 con 0x7f0a9c0cdcc0 2024-09-13T18:54:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa18c0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c15b890 0x7f0a9c1f94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa10bf700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a9c0b2b00 0x7f0a9c151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa10bf700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a9c0b2b00 0x7f0a9c151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:38810/0 (socket says 172.21.15.7:38810) 2024-09-13T18:54:04.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa10bf700 1 -- 172.21.15.7:0/1194147895 learned_addr learned my addr 172.21.15.7:0/1194147895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:04.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa08be700 1 --2- 172.21.15.7:0/1194147895 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a9c0cdcc0 0x7f0a9c1f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:04.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa10bf700 1 -- 172.21.15.7:0/1194147895 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a9c0cdcc0 msgr2=0x7f0a9c1f8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:04.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa10bf700 1 --2- 172.21.15.7:0/1194147895 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a9c0cdcc0 0x7f0a9c1f8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:04.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa10bf700 1 -- 172.21.15.7:0/1194147895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c15b890 msgr2=0x7f0a9c1f94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:04.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa10bf700 1 --2- 172.21.15.7:0/1194147895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c15b890 0x7f0a9c1f94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:04.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa10bf700 1 -- 172.21.15.7:0/1194147895 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8c0094b0 con 0x7f0a9c0b2b00 2024-09-13T18:54:04.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.203+0000 7f0aa18c0700 1 --2- 172.21.15.7:0/1194147895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c15b890 0x7f0a9c1f94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.207+0000 7f0aa10bf700 1 --2- 172.21.15.7:0/1194147895 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a9c0b2b00 0x7f0a9c151c60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009ba0 tx=0x7f0a8c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.207+0000 7f0a91ffb700 1 -- 172.21.15.7:0/1194147895 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c004550 con 0x7f0a9c0b2b00 2024-09-13T18:54:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.207+0000 7f0aa3323700 1 -- 172.21.15.7:0/1194147895 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9c1f81d0 con 0x7f0a9c0b2b00 2024-09-13T18:54:04.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.207+0000 7f0a91ffb700 1 -- 172.21.15.7:0/1194147895 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c01e070 con 0x7f0a9c0b2b00 2024-09-13T18:54:04.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.207+0000 7f0aa3323700 1 -- 172.21.15.7:0/1194147895 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9c1f8a70 con 0x7f0a9c0b2b00 2024-09-13T18:54:04.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.207+0000 7f0a91ffb700 1 -- 172.21.15.7:0/1194147895 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c003900 con 0x7f0a9c0b2b00 2024-09-13T18:54:04.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.207+0000 7f0aa3323700 1 -- 172.21.15.7:0/1194147895 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a9c1f83e0 con 0x7f0a9c0b2b00 2024-09-13T18:54:04.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.211+0000 7f0a91ffb700 1 -- 172.21.15.7:0/1194147895 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0a8c003a60 con 0x7f0a9c0b2b00 2024-09-13T18:54:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.211+0000 7f0a91ffb700 1 --2- 172.21.15.7:0/1194147895 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0a80066730 0x7f0a80068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.211+0000 7f0a91ffb700 1 -- 172.21.15.7:0/1194147895 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f0a8c0852d0 con 0x7f0a9c0b2b00 2024-09-13T18:54:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.211+0000 7f0aa08be700 1 --2- 172.21.15.7:0/1194147895 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0a80066730 0x7f0a80068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:04.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.211+0000 7f0aa08be700 1 --2- 172.21.15.7:0/1194147895 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0a80066730 0x7f0a80068bf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c155680 tx=0x7f0a98009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:04.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.215+0000 7f0a91ffb700 1 -- 172.21.15.7:0/1194147895 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0a8c052970 con 0x7f0a9c0b2b00 2024-09-13T18:54:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.423+0000 7f0aa3323700 1 -- 172.21.15.7:0/1194147895 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a9c0ce120 con 0x7f0a9c0b2b00 2024-09-13T18:54:04.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.423+0000 7f0a91ffb700 1 -- 172.21.15.7:0/1194147895 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f0a8c015de0 con 0x7f0a9c0b2b00 2024-09-13T18:54:04.433 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:04.433 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2458672850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2458672850},{"type":"v1","addr":"172.21.15.171:6835","nonce":2458672850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5389,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/3381042298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":3381042298},{"type":"v1","addr":"172.21.15.171:6837","nonce":3381042298}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5451,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/2068823578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2068823578},{"type":"v1","addr":"172.21.15.7:6833","nonce":2068823578}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17},{"gid":5634,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/619579729","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":619579729},{"type":"v1","addr":"172.21.15.7:6835","nonce":619579729}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[]} 2024-09-13T18:54:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.427+0000 7f0aa3323700 1 -- 172.21.15.7:0/1194147895 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0a80066730 msgr2=0x7f0a80068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.427+0000 7f0aa3323700 1 --2- 172.21.15.7:0/1194147895 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0a80066730 0x7f0a80068bf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c155680 tx=0x7f0a98009040 comp rx=0 tx=0).stop 2024-09-13T18:54:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.427+0000 7f0aa3323700 1 -- 172.21.15.7:0/1194147895 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a9c0b2b00 msgr2=0x7f0a9c151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.427+0000 7f0aa3323700 1 --2- 172.21.15.7:0/1194147895 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a9c0b2b00 0x7f0a9c151c60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009ba0 tx=0x7f0a8c004770 comp rx=0 tx=0).stop 2024-09-13T18:54:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.427+0000 7f0aa3323700 1 -- 172.21.15.7:0/1194147895 shutdown_connections 2024-09-13T18:54:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.427+0000 7f0aa3323700 1 --2- 172.21.15.7:0/1194147895 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0a80066730 0x7f0a80068bf0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.427+0000 7f0aa3323700 1 --2- 172.21.15.7:0/1194147895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c15b890 0x7f0a9c1f94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.427+0000 7f0aa3323700 1 --2- 172.21.15.7:0/1194147895 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0a9c0cdcc0 0x7f0a9c1f8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.427+0000 7f0aa3323700 1 --2- 172.21.15.7:0/1194147895 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0a9c0b2b00 0x7f0a9c151c60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.427+0000 7f0aa3323700 1 -- 172.21.15.7:0/1194147895 >> 172.21.15.7:0/1194147895 conn(0x7f0a9c0ce980 msgr2=0x7f0a9c0ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.427+0000 7f0aa3323700 1 -- 172.21.15.7:0/1194147895 shutdown_connections 2024-09-13T18:54:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.427+0000 7f0aa3323700 1 -- 172.21.15.7:0/1194147895 wait complete. 2024-09-13T18:54:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2024-09-13T18:54:04.448 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T18:54:04.448 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T18:54:04.448 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T18:54:04.449 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:54:04.449 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T18:54:04.449 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-13T18:54:04.449 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T18:54:04.449 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:54:04.449 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T18:54:04.449 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T18:54:04.450 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T18:54:04.450 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:54:04.450 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T18:54:04.450 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-13T18:54:04.450 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T18:54:04.450 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:54:04.462 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T18:54:04.463 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T18:54:04.464 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T18:54:04.464 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T18:54:04.465 INFO:tasks.ceph.mds.a:Started 2024-09-13T18:54:04.465 DEBUG:teuthology.parallel:result is None 2024-09-13T18:54:04.466 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T18:54:04.467 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T18:54:04.467 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T18:54:04.467 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T18:54:04.469 INFO:tasks.ceph.mds.c:Started 2024-09-13T18:54:04.469 DEBUG:teuthology.parallel:result is None 2024-09-13T18:54:04.469 INFO:tasks.ceph.mds.d:Started 2024-09-13T18:54:04.470 INFO:tasks.ceph.mds.b:Started 2024-09-13T18:54:04.470 DEBUG:teuthology.parallel:result is None 2024-09-13T18:54:04.470 DEBUG:teuthology.parallel:result is None 2024-09-13T18:54:04.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T18:54:04.538 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-09-13T18:54:04.543 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-13T18:54:04.546 INFO:tasks.ceph.mds.b.smithi171.stdout:starting mds.b at 2024-09-13T18:54:04.547 INFO:tasks.ceph.mds.d.smithi171.stdout:starting mds.d at 2024-09-13T18:54:04.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.587+0000 7f916bfff700 1 Processor -- start 2024-09-13T18:54:04.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.587+0000 7f916bfff700 1 -- start start 2024-09-13T18:54:04.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.587+0000 7f916bfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f916c06b900 0x7f916c0b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:04.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.587+0000 7f916bfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916c0b8430 con 0x7f916c15cda0 2024-09-13T18:54:04.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.587+0000 7f916bfff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916c06bd60 con 0x7f916c06a520 2024-09-13T18:54:04.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.587+0000 7f916bfff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0cf350 con 0x7f916c06b900 2024-09-13T18:54:04.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.587+0000 7f916affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f916c15cda0 0x7f916c0ceb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57846/0 (socket says 172.21.15.7:57846) 2024-09-13T18:54:04.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.587+0000 7f916affd700 1 -- 172.21.15.7:0/2825173453 learned_addr learned my addr 172.21.15.7:0/2825173453 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:04.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.587+0000 7f916a7fc700 1 --2- 172.21.15.7:0/2825173453 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f916c06b900 0x7f916c0b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:04.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.587+0000 7f9169ffb700 1 -- 172.21.15.7:0/2825173453 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3938948106 0 0) 0x7f916c0b8430 con 0x7f916c15cda0 2024-09-13T18:54:04.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.587+0000 7f9169ffb700 1 -- 172.21.15.7:0/2825173453 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f914c0035d0 con 0x7f916c15cda0 2024-09-13T18:54:04.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.587+0000 7f9169ffb700 1 -- 172.21.15.7:0/2825173453 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874646230 0 0) 0x7f916c06bd60 con 0x7f916c06a520 2024-09-13T18:54:04.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.587+0000 7f9169ffb700 1 -- 172.21.15.7:0/2825173453 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f916c0b8430 con 0x7f916c06a520 2024-09-13T18:54:04.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.587+0000 7f916a7fc700 1 -- 172.21.15.7:0/2825173453 >> v1:172.21.15.171:6789/0 conn(0x7f916c06a520 legacy=0x7f916c0d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:04.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.587+0000 7f916a7fc700 1 -- 172.21.15.7:0/2825173453 >> v1:172.21.15.7:6789/0 conn(0x7f916c15cda0 legacy=0x7f916c0ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:04.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.587+0000 7f916a7fc700 1 -- 172.21.15.7:0/2825173453 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c0d0880 con 0x7f916c06b900 2024-09-13T18:54:04.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.591+0000 7f916a7fc700 1 --2- 172.21.15.7:0/2825173453 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f916c06b900 0x7f916c0b5de0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9154009880 tx=0x7f9154009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f67895edf397efc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:04.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.591+0000 7f9169ffb700 1 -- 172.21.15.7:0/2825173453 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9154011830 con 0x7f916c06b900 2024-09-13T18:54:04.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.591+0000 7f9169ffb700 1 -- 172.21.15.7:0/2825173453 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9154011990 con 0x7f916c06b900 2024-09-13T18:54:04.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.591+0000 7f9169ffb700 1 -- 172.21.15.7:0/2825173453 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9154011d20 con 0x7f916c06b900 2024-09-13T18:54:04.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.591+0000 7f916bfff700 1 -- 172.21.15.7:0/2825173453 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f916c06b900 msgr2=0x7f916c0b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:04.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.591+0000 7f916bfff700 1 --2- 172.21.15.7:0/2825173453 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f916c06b900 0x7f916c0b5de0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9154009880 tx=0x7f9154009b90 comp rx=0 tx=0).stop 2024-09-13T18:54:04.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.591+0000 7f916bfff700 1 -- 172.21.15.7:0/2825173453 shutdown_connections 2024-09-13T18:54:04.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.591+0000 7f916bfff700 1 --2- 172.21.15.7:0/2825173453 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f916c06b900 0x7f916c0b5de0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:04.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.591+0000 7f916bfff700 1 -- 172.21.15.7:0/2825173453 >> 172.21.15.7:0/2825173453 conn(0x7f916c0d2cb0 msgr2=0x7f916c0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:04.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.591+0000 7f916bfff700 1 -- 172.21.15.7:0/2825173453 shutdown_connections 2024-09-13T18:54:04.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.591+0000 7f916bfff700 1 -- 172.21.15.7:0/2825173453 wait complete. 2024-09-13T18:54:04.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916bfff700 1 Processor -- start 2024-09-13T18:54:04.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916bfff700 1 -- start start 2024-09-13T18:54:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f916c06a520 0x7f916c1f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916bfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f916c06b900 0x7f916c1f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c15cda0 0x7f916c0c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916bfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0c84b0 con 0x7f916c15cda0 2024-09-13T18:54:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916bfff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0c7830 con 0x7f916c06a520 2024-09-13T18:54:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916bfff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0c7b10 con 0x7f916c06b900 2024-09-13T18:54:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c15cda0 0x7f916c0c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916a7fc700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f916c06b900 0x7f916c1f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916affd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f916c06a520 0x7f916c1f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916a7fc700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f916c06b900 0x7f916c1f3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:54224/0 (socket says 172.21.15.7:54224) 2024-09-13T18:54:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c15cda0 0x7f916c0c88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39256/0 (socket says 172.21.15.7:39256) 2024-09-13T18:54:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916a7fc700 1 -- 172.21.15.7:0/3601947757 learned_addr learned my addr 172.21.15.7:0/3601947757 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:04.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916affd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f916c06a520 0x7f916c1f3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:38862/0 (socket says 172.21.15.7:38862) 2024-09-13T18:54:04.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916b7fe700 1 -- 172.21.15.7:0/3601947757 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f916c06b900 msgr2=0x7f916c1f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:04.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916b7fe700 1 --2- 172.21.15.7:0/3601947757 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f916c06b900 0x7f916c1f3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:04.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916b7fe700 1 -- 172.21.15.7:0/3601947757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f916c06a520 msgr2=0x7f916c1f3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:04.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916b7fe700 1 --2- 172.21.15.7:0/3601947757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f916c06a520 0x7f916c1f3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:04.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916b7fe700 1 -- 172.21.15.7:0/3601947757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91540094b0 con 0x7f916c15cda0 2024-09-13T18:54:04.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916a7fc700 1 --2- 172.21.15.7:0/3601947757 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f916c06b900 0x7f916c1f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.595+0000 7f916affd700 1 --2- 172.21.15.7:0/3601947757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f916c06a520 0x7f916c1f3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.599+0000 7f916b7fe700 1 --2- 172.21.15.7:0/3601947757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c15cda0 0x7f916c0c88d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f916400c430 tx=0x7f916400c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.599+0000 7f91637fe700 1 -- 172.21.15.7:0/3601947757 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91640050d0 con 0x7f916c15cda0 2024-09-13T18:54:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.599+0000 7f91637fe700 1 -- 172.21.15.7:0/3601947757 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9164005230 con 0x7f916c15cda0 2024-09-13T18:54:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.599+0000 7f916bfff700 1 -- 172.21.15.7:0/3601947757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916c0c7df0 con 0x7f916c15cda0 2024-09-13T18:54:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.599+0000 7f91637fe700 1 -- 172.21.15.7:0/3601947757 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91640036a0 con 0x7f916c15cda0 2024-09-13T18:54:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.599+0000 7f916bfff700 1 -- 172.21.15.7:0/3601947757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c1efba0 con 0x7f916c15cda0 2024-09-13T18:54:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.599+0000 7f916bfff700 1 -- 172.21.15.7:0/3601947757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916c0c85f0 con 0x7f916c15cda0 2024-09-13T18:54:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.599+0000 7f91637fe700 1 -- 172.21.15.7:0/3601947757 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9164003800 con 0x7f916c15cda0 2024-09-13T18:54:04.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.599+0000 7f91637fe700 1 --2- 172.21.15.7:0/3601947757 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9150066730 0x7f9150068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:04.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.599+0000 7f91637fe700 1 -- 172.21.15.7:0/3601947757 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f91640831b0 con 0x7f916c15cda0 2024-09-13T18:54:04.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.599+0000 7f916affd700 1 --2- 172.21.15.7:0/3601947757 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9150066730 0x7f9150068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:04.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.603+0000 7f916affd700 1 --2- 172.21.15.7:0/3601947757 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9150066730 0x7f9150068bf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f915c0012e0 tx=0x7f915c001a30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:04.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.611+0000 7f91637fe700 1 -- 172.21.15.7:0/3601947757 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f916c0c85f0 con 0x7f916c15cda0 2024-09-13T18:54:04.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:04.795+0000 7f916bfff700 1 -- 172.21.15.7:0/3601947757 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f916c0b5de0 con 0x7f916c15cda0 2024-09-13T18:54:05.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.579+0000 7f91637fe700 1 -- 172.21.15.7:0/3601947757 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v48) v1 ==== 97+0+0 (secure 0 0 0) 0x7f91640507d0 con 0x7f916c15cda0 2024-09-13T18:54:05.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.583+0000 7f916bfff700 1 -- 172.21.15.7:0/3601947757 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9150066730 msgr2=0x7f9150068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:05.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.583+0000 7f916bfff700 1 --2- 172.21.15.7:0/3601947757 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9150066730 0x7f9150068bf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f915c0012e0 tx=0x7f915c001a30 comp rx=0 tx=0).stop 2024-09-13T18:54:05.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.583+0000 7f916bfff700 1 -- 172.21.15.7:0/3601947757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c15cda0 msgr2=0x7f916c0c88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:05.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.583+0000 7f916bfff700 1 --2- 172.21.15.7:0/3601947757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c15cda0 0x7f916c0c88d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f916400c430 tx=0x7f916400c740 comp rx=0 tx=0).stop 2024-09-13T18:54:05.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.583+0000 7f916bfff700 1 -- 172.21.15.7:0/3601947757 shutdown_connections 2024-09-13T18:54:05.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.583+0000 7f916bfff700 1 --2- 172.21.15.7:0/3601947757 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9150066730 0x7f9150068bf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:05.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.583+0000 7f916bfff700 1 --2- 172.21.15.7:0/3601947757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f916c15cda0 0x7f916c0c88d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:05.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.583+0000 7f916bfff700 1 --2- 172.21.15.7:0/3601947757 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f916c06b900 0x7f916c1f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:05.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.583+0000 7f916bfff700 1 --2- 172.21.15.7:0/3601947757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f916c06a520 0x7f916c1f3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:05.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.583+0000 7f916bfff700 1 -- 172.21.15.7:0/3601947757 >> 172.21.15.7:0/3601947757 conn(0x7f916c0d2cb0 msgr2=0x7f916c0b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:05.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.583+0000 7f916bfff700 1 -- 172.21.15.7:0/3601947757 shutdown_connections 2024-09-13T18:54:05.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.583+0000 7f916bfff700 1 -- 172.21.15.7:0/3601947757 wait complete. 2024-09-13T18:54:05.594 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-09-13T18:54:05.610 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T18:54:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.723+0000 7f895adee700 1 Processor -- start 2024-09-13T18:54:05.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.723+0000 7f895adee700 1 -- start start 2024-09-13T18:54:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.723+0000 7f895adee700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8954160eb0 0x7f8954165590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.723+0000 7f895adee700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f89540b33d0 0x7f89540b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.723+0000 7f895adee700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89540b3d30 con 0x7f8954161290 2024-09-13T18:54:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.723+0000 7f895adee700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8954071f30 con 0x7f89540b33d0 2024-09-13T18:54:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.723+0000 7f895adee700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89541622e0 con 0x7f8954160eb0 2024-09-13T18:54:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.723+0000 7f89595eb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f89540b33d0 0x7f89540b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.723+0000 7f89595eb700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f89540b33d0 0x7f89540b37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39266/0 (socket says 172.21.15.7:39266) 2024-09-13T18:54:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.723+0000 7f89595eb700 1 -- 172.21.15.7:0/1829630597 learned_addr learned my addr 172.21.15.7:0/1829630597 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:05.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.723+0000 7f8959dec700 1 --2- 172.21.15.7:0/1829630597 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8954160eb0 0x7f8954165590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.723+0000 7f89595eb700 1 -- 172.21.15.7:0/1829630597 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8954160eb0 msgr2=0x7f8954165590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.723+0000 7f89595eb700 1 --2- 172.21.15.7:0/1829630597 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8954160eb0 0x7f8954165590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.723+0000 7f89595eb700 1 -- 172.21.15.7:0/1829630597 >> v1:172.21.15.171:6789/0 conn(0x7f8954161290 legacy=0x7f8954161740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.723+0000 7f89595eb700 1 -- 172.21.15.7:0/1829630597 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8954162000 con 0x7f89540b33d0 2024-09-13T18:54:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.723+0000 7f8958dea700 1 -- 172.21.15.7:0/1829630597 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130750260 0 0) 0x7f89540b3d30 con 0x7f8954161290 2024-09-13T18:54:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.727+0000 7f89595eb700 1 --2- 172.21.15.7:0/1829630597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89540b33d0 0x7f89540b37f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8950009910 tx=0x7f8950009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1823a0e537128d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.727+0000 7f8958dea700 1 -- 172.21.15.7:0/1829630597 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950011830 con 0x7f89540b33d0 2024-09-13T18:54:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.727+0000 7f8958dea700 1 -- 172.21.15.7:0/1829630597 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8950011990 con 0x7f89540b33d0 2024-09-13T18:54:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.727+0000 7f8958dea700 1 -- 172.21.15.7:0/1829630597 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950011d40 con 0x7f89540b33d0 2024-09-13T18:54:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.727+0000 7f895adee700 1 -- 172.21.15.7:0/1829630597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89540b33d0 msgr2=0x7f89540b37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.727+0000 7f895adee700 1 --2- 172.21.15.7:0/1829630597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89540b33d0 0x7f89540b37f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8950009910 tx=0x7f8950009c20 comp rx=0 tx=0).stop 2024-09-13T18:54:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.727+0000 7f895adee700 1 -- 172.21.15.7:0/1829630597 shutdown_connections 2024-09-13T18:54:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.727+0000 7f895adee700 1 --2- 172.21.15.7:0/1829630597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89540b33d0 0x7f89540b37f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.727+0000 7f895adee700 1 --2- 172.21.15.7:0/1829630597 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8954160eb0 0x7f8954165590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.727+0000 7f895adee700 1 -- 172.21.15.7:0/1829630597 >> 172.21.15.7:0/1829630597 conn(0x7f89540b0f00 msgr2=0x7f89540b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.727+0000 7f895adee700 1 -- 172.21.15.7:0/1829630597 shutdown_connections 2024-09-13T18:54:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.727+0000 7f895adee700 1 -- 172.21.15.7:0/1829630597 wait complete. 2024-09-13T18:54:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.727+0000 7f895adee700 1 Processor -- start 2024-09-13T18:54:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.727+0000 7f895adee700 1 -- start start 2024-09-13T18:54:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f895adee700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89540b33d0 0x7f89541f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f895adee700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8954160eb0 0x7f89540a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f895adee700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8954161290 0x7f89541f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f895adee700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89541f7880 con 0x7f8954161290 2024-09-13T18:54:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f895adee700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89540adcb0 con 0x7f89540b33d0 2024-09-13T18:54:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f895adee700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89540adf90 con 0x7f8954160eb0 2024-09-13T18:54:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f895a5ed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8954161290 0x7f89541f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f895a5ed700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8954161290 0x7f89541f5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39272/0 (socket says 172.21.15.7:39272) 2024-09-13T18:54:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f895a5ed700 1 -- 172.21.15.7:0/3076093060 learned_addr learned my addr 172.21.15.7:0/3076093060 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f89595eb700 1 --2- 172.21.15.7:0/3076093060 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8954160eb0 0x7f89540a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f8959dec700 1 --2- 172.21.15.7:0/3076093060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89540b33d0 0x7f89541f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f895a5ed700 1 -- 172.21.15.7:0/3076093060 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8954160eb0 msgr2=0x7f89540a91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f895a5ed700 1 --2- 172.21.15.7:0/3076093060 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8954160eb0 0x7f89540a91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f895a5ed700 1 -- 172.21.15.7:0/3076093060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89540b33d0 msgr2=0x7f89541f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f895a5ed700 1 --2- 172.21.15.7:0/3076093060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89540b33d0 0x7f89541f7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f895a5ed700 1 -- 172.21.15.7:0/3076093060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8950009570 con 0x7f8954161290 2024-09-13T18:54:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f895a5ed700 1 --2- 172.21.15.7:0/3076093060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8954161290 0x7f89541f5d30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f894c00cc10 tx=0x7f894c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f894a7fc700 1 -- 172.21.15.7:0/3076093060 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c003e50 con 0x7f8954161290 2024-09-13T18:54:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f894a7fc700 1 -- 172.21.15.7:0/3076093060 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c00a650 con 0x7f8954161290 2024-09-13T18:54:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f895adee700 1 -- 172.21.15.7:0/3076093060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89540ae270 con 0x7f8954161290 2024-09-13T18:54:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f894a7fc700 1 -- 172.21.15.7:0/3076093060 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c00aa00 con 0x7f8954161290 2024-09-13T18:54:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.731+0000 7f895adee700 1 -- 172.21.15.7:0/3076093060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f895415a9f0 con 0x7f8954161290 2024-09-13T18:54:05.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.735+0000 7f894a7fc700 1 -- 172.21.15.7:0/3076093060 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f894c00ab60 con 0x7f8954161290 2024-09-13T18:54:05.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.735+0000 7f895adee700 1 -- 172.21.15.7:0/3076093060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8954165590 con 0x7f8954161290 2024-09-13T18:54:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.735+0000 7f894a7fc700 1 --2- 172.21.15.7:0/3076093060 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8938066730 0x7f8938068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.735+0000 7f894a7fc700 1 -- 172.21.15.7:0/3076093060 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f894c0837c0 con 0x7f8954161290 2024-09-13T18:54:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.735+0000 7f8959dec700 1 --2- 172.21.15.7:0/3076093060 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8938066730 0x7f8938068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.735+0000 7f8959dec700 1 --2- 172.21.15.7:0/3076093060 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8938066730 0x7f8938068bf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8954071e00 tx=0x7f8944009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:05.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.739+0000 7f894a7fc700 1 -- 172.21.15.7:0/3076093060 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f894c050e70 con 0x7f8954161290 2024-09-13T18:54:05.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.971+0000 7f895adee700 1 -- 172.21.15.7:0/3076093060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f895406a370 con 0x7f8954161290 2024-09-13T18:54:05.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.971+0000 7f894a7fc700 1 -- 172.21.15.7:0/3076093060 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f894c054490 con 0x7f8954161290 2024-09-13T18:54:05.981 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:05.981 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:54:05.981 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-09-13T18:54:05.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.981 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-09-13T18:54:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRmluqnExAAsUgfqXyo3d0MsBU4pTUFKA==", 2024-09-13T18:54:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-13T18:54:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-13T18:54:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-13T18:54:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:54:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-09-13T18:54:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRmse2YGxAA1xevwPXbqkiFfH1c51CyUQ==", 2024-09-13T18:54:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-13T18:54:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-13T18:54:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-13T18:54:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:54:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-09-13T18:54:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRmovm+FxAAACx3SjaGO+6TwKcii8ZfcQ==", 2024-09-13T18:54:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-13T18:54:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-13T18:54:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-13T18:54:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:54:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-09-13T18:54:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRms9RQHxAAWQjRb5IFrsEkHgk1fFl1gA==", 2024-09-13T18:54:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-13T18:54:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-13T18:54:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-13T18:54:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:54:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-09-13T18:54:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCzieRmoweYHRAAJq4W0iwb4bNJtNoojDmYvA==", 2024-09-13T18:54:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-13T18:54:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-13T18:54:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:54:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-09-13T18:54:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQC2ieRmMlp6IBAA6UPBa8XtvToSEXOH7jRp2Q==", 2024-09-13T18:54:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-13T18:54:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-13T18:54:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:54:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-09-13T18:54:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQC5ieRm8PRwJxAAJ6u6O2CY9AB8ADXGUq3tHw==", 2024-09-13T18:54:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-13T18:54:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-13T18:54:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:54:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-09-13T18:54:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQC8ieRmpelUKxAAZkfxHmxH4AQs2+ZmYEMI5g==", 2024-09-13T18:54:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-13T18:54:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-13T18:54:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:54:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-09-13T18:54:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDCieRmt6JLERAAOSeTXcLflQO4UStq8/r4Kg==", 2024-09-13T18:54:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-13T18:54:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-13T18:54:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:54:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-09-13T18:54:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDFieRmnrQzFhAAbpEXq+QqxOuUEon2UfwRww==", 2024-09-13T18:54:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-13T18:54:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-13T18:54:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:54:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-09-13T18:54:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDIieRmbacMGhAAXeApS17wKjLK0KlAvOFMCw==", 2024-09-13T18:54:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-13T18:54:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-13T18:54:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:54:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-09-13T18:54:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDLieRmcxLrIRAAnXXU0UeSpdKkMzVuT2U9tA==", 2024-09-13T18:54:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-13T18:54:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-13T18:54:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:54:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-09-13T18:54:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRmcmOmIhAAoiL5c3tp8skQ5VcKF/6zVw==", 2024-09-13T18:54:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-13T18:54:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-09-13T18:54:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-13T18:54:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-09-13T18:54:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRmcNsbLBAAj36vmzv7emQ+ueYkP/2HtA==", 2024-09-13T18:54:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-13T18:54:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-13T18:54:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-13T18:54:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-13T18:54:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-09-13T18:54:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRmvonEKBAAefHhRIV+mux4L2yvJieukQ==", 2024-09-13T18:54:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-13T18:54:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-13T18:54:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-13T18:54:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-13T18:54:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-09-13T18:54:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRmh8G8MRAAeqibcXujnGSy9zs4XPf2hA==", 2024-09-13T18:54:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-13T18:54:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-13T18:54:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-13T18:54:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-13T18:54:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-09-13T18:54:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRm7nRBARAAInH7XqMLMDnVtQEezwkVhg==", 2024-09-13T18:54:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-13T18:54:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-13T18:54:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-13T18:54:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:54:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-09-13T18:54:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRm67fCDhAAm9sMp6tlCuJ3vwp57kczzA==", 2024-09-13T18:54:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-13T18:54:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-13T18:54:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:54:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-09-13T18:54:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQCyieRmJQ94CxAAIusXMD6x4ey4OLJO9t7n+w==", 2024-09-13T18:54:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-13T18:54:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-13T18:54:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-13T18:54:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-13T18:54:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-13T18:54:05.995 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:54:05.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.975+0000 7f895adee700 1 -- 172.21.15.7:0/3076093060 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8938066730 msgr2=0x7f8938068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:05.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.975+0000 7f895adee700 1 --2- 172.21.15.7:0/3076093060 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8938066730 0x7f8938068bf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8954071e00 tx=0x7f8944009240 comp rx=0 tx=0).stop 2024-09-13T18:54:05.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.975+0000 7f895adee700 1 -- 172.21.15.7:0/3076093060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8954161290 msgr2=0x7f89541f5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:05.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.975+0000 7f895adee700 1 --2- 172.21.15.7:0/3076093060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8954161290 0x7f89541f5d30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f894c00cc10 tx=0x7f894c00cf20 comp rx=0 tx=0).stop 2024-09-13T18:54:05.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.975+0000 7f895adee700 1 -- 172.21.15.7:0/3076093060 shutdown_connections 2024-09-13T18:54:05.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.975+0000 7f895adee700 1 --2- 172.21.15.7:0/3076093060 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8938066730 0x7f8938068bf0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:05.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.975+0000 7f895adee700 1 --2- 172.21.15.7:0/3076093060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8954161290 0x7f89541f5d30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:05.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.975+0000 7f895adee700 1 --2- 172.21.15.7:0/3076093060 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8954160eb0 0x7f89540a91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:05.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.975+0000 7f895adee700 1 --2- 172.21.15.7:0/3076093060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89540b33d0 0x7f89541f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:05.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.975+0000 7f895adee700 1 -- 172.21.15.7:0/3076093060 >> 172.21.15.7:0/3076093060 conn(0x7f89540b0f00 msgr2=0x7f895415dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:05.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.975+0000 7f895adee700 1 -- 172.21.15.7:0/3076093060 shutdown_connections 2024-09-13T18:54:05.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:05.975+0000 7f895adee700 1 -- 172.21.15.7:0/3076093060 wait complete. 2024-09-13T18:54:05.998 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:54:06.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1af55b700 1 Processor -- start 2024-09-13T18:54:06.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1af55b700 1 -- start start 2024-09-13T18:54:06.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1af55b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd1a80b3ae0 0x7fd1a806a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1af55b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1a806aaf0 0x7fd1a815e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1af55b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a813dcb0 con 0x7fd1a806bbf0 2024-09-13T18:54:06.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1af55b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a815be60 con 0x7fd1a80b3ae0 2024-09-13T18:54:06.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1af55b700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a813ffd0 con 0x7fd1a806aaf0 2024-09-13T18:54:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1add58700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd1a80b3ae0 0x7fd1a806a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1add58700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd1a80b3ae0 0x7fd1a806a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39274/0 (socket says 172.21.15.7:39274) 2024-09-13T18:54:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1add58700 1 -- 172.21.15.7:0/1793809917 learned_addr learned my addr 172.21.15.7:0/1793809917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1aed5a700 1 --2- 172.21.15.7:0/1793809917 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1a806aaf0 0x7fd1a815e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1add58700 1 -- 172.21.15.7:0/1793809917 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1a806aaf0 msgr2=0x7fd1a815e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1add58700 1 --2- 172.21.15.7:0/1793809917 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1a806aaf0 0x7fd1a815e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1add58700 1 -- 172.21.15.7:0/1793809917 >> v1:172.21.15.7:6789/0 conn(0x7fd1a806bbf0 legacy=0x7fd1a80b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1add58700 1 -- 172.21.15.7:0/1793809917 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a8140220 con 0x7fd1a80b3ae0 2024-09-13T18:54:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1add58700 1 --2- 172.21.15.7:0/1793809917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1a80b3ae0 0x7fd1a806a5b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009910 tx=0x7fd1a4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588d73757af9e1e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1ad557700 1 -- 172.21.15.7:0/1793809917 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4011830 con 0x7fd1a80b3ae0 2024-09-13T18:54:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1ad557700 1 -- 172.21.15.7:0/1793809917 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a4011990 con 0x7fd1a80b3ae0 2024-09-13T18:54:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.107+0000 7fd1ad557700 1 -- 172.21.15.7:0/1793809917 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4011d40 con 0x7fd1a80b3ae0 2024-09-13T18:54:06.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1af55b700 1 -- 172.21.15.7:0/1793809917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1a80b3ae0 msgr2=0x7fd1a806a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1af55b700 1 --2- 172.21.15.7:0/1793809917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1a80b3ae0 0x7fd1a806a5b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4009910 tx=0x7fd1a4009c20 comp rx=0 tx=0).stop 2024-09-13T18:54:06.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1af55b700 1 -- 172.21.15.7:0/1793809917 shutdown_connections 2024-09-13T18:54:06.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1af55b700 1 --2- 172.21.15.7:0/1793809917 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1a806aaf0 0x7fd1a815e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1af55b700 1 --2- 172.21.15.7:0/1793809917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1a80b3ae0 0x7fd1a806a5b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1af55b700 1 -- 172.21.15.7:0/1793809917 >> 172.21.15.7:0/1793809917 conn(0x7fd1a80b0f00 msgr2=0x7fd1a80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1af55b700 1 -- 172.21.15.7:0/1793809917 shutdown_connections 2024-09-13T18:54:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1af55b700 1 -- 172.21.15.7:0/1793809917 wait complete. 2024-09-13T18:54:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1af55b700 1 Processor -- start 2024-09-13T18:54:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1af55b700 1 -- start start 2024-09-13T18:54:06.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1af55b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1a806aaf0 0x7fd1a8156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1af55b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1a806bbf0 0x7fd1a80adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1af55b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1a80b3ae0 0x7fd1a8071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1af55b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a8156cf0 con 0x7fd1a806aaf0 2024-09-13T18:54:06.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1af55b700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a8156fd0 con 0x7fd1a80b3ae0 2024-09-13T18:54:06.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1af55b700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a81572b0 con 0x7fd1a806bbf0 2024-09-13T18:54:06.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1ae559700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1a806aaf0 0x7fd1a8156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1ae559700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1a806aaf0 0x7fd1a8156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39280/0 (socket says 172.21.15.7:39280) 2024-09-13T18:54:06.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1ae559700 1 -- 172.21.15.7:0/2055895584 learned_addr learned my addr 172.21.15.7:0/2055895584 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:06.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1aed5a700 1 --2- 172.21.15.7:0/2055895584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1a80b3ae0 0x7fd1a8071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.111+0000 7fd1add58700 1 --2- 172.21.15.7:0/2055895584 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1a806bbf0 0x7fd1a80adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.115+0000 7fd1ae559700 1 -- 172.21.15.7:0/2055895584 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1a806bbf0 msgr2=0x7fd1a80adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.115+0000 7fd1ae559700 1 --2- 172.21.15.7:0/2055895584 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1a806bbf0 0x7fd1a80adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.115+0000 7fd1ae559700 1 -- 172.21.15.7:0/2055895584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1a80b3ae0 msgr2=0x7fd1a8071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.115+0000 7fd1ae559700 1 --2- 172.21.15.7:0/2055895584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1a80b3ae0 0x7fd1a8071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.115+0000 7fd1ae559700 1 -- 172.21.15.7:0/2055895584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a4009570 con 0x7fd1a806aaf0 2024-09-13T18:54:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.115+0000 7fd1aed5a700 1 --2- 172.21.15.7:0/2055895584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1a80b3ae0 0x7fd1a8071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.115+0000 7fd1ae559700 1 --2- 172.21.15.7:0/2055895584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1a806aaf0 0x7fd1a8156710 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd19800c500 tx=0x7fd19800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.115+0000 7fd19effd700 1 -- 172.21.15.7:0/2055895584 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1980052d0 con 0x7fd1a806aaf0 2024-09-13T18:54:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.115+0000 7fd19effd700 1 -- 172.21.15.7:0/2055895584 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd198005430 con 0x7fd1a806aaf0 2024-09-13T18:54:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.115+0000 7fd19effd700 1 -- 172.21.15.7:0/2055895584 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1980055a0 con 0x7fd1a806aaf0 2024-09-13T18:54:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.115+0000 7fd1af55b700 1 -- 172.21.15.7:0/2055895584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a8157590 con 0x7fd1a806aaf0 2024-09-13T18:54:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.115+0000 7fd1af55b700 1 -- 172.21.15.7:0/2055895584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a813d2e0 con 0x7fd1a806aaf0 2024-09-13T18:54:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.115+0000 7fd1af55b700 1 -- 172.21.15.7:0/2055895584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a8157ab0 con 0x7fd1a806aaf0 2024-09-13T18:54:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.115+0000 7fd19effd700 1 -- 172.21.15.7:0/2055895584 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd1980038e0 con 0x7fd1a806aaf0 2024-09-13T18:54:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.119+0000 7fd19effd700 1 --2- 172.21.15.7:0/2055895584 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd18c066730 0x7fd18c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.119+0000 7fd19effd700 1 -- 172.21.15.7:0/2055895584 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd198083200 con 0x7fd1a806aaf0 2024-09-13T18:54:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.119+0000 7fd1add58700 1 --2- 172.21.15.7:0/2055895584 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd18c066730 0x7fd18c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.119+0000 7fd1add58700 1 --2- 172.21.15.7:0/2055895584 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd18c066730 0x7fd18c068bf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4000c00 tx=0x7fd1a4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:06.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.123+0000 7fd19effd700 1 -- 172.21.15.7:0/2055895584 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd1a8157ab0 con 0x7fd1a806aaf0 2024-09-13T18:54:06.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.299+0000 7fd1af55b700 1 -- 172.21.15.7:0/2055895584 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1a8157ab0 con 0x7fd1a806aaf0 2024-09-13T18:54:06.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.299+0000 7fd19effd700 1 -- 172.21.15.7:0/2055895584 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+12809 (secure 0 0 0) 0x7fd198050950 con 0x7fd1a806aaf0 2024-09-13T18:54:06.309 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:06.309 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":48,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:54:05.579655+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:54:06.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.303+0000 7fd1af55b700 1 -- 172.21.15.7:0/2055895584 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd18c066730 msgr2=0x7fd18c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.303+0000 7fd1af55b700 1 --2- 172.21.15.7:0/2055895584 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd18c066730 0x7fd18c068bf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd1a4000c00 tx=0x7fd1a4008030 comp rx=0 tx=0).stop 2024-09-13T18:54:06.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.303+0000 7fd1af55b700 1 -- 172.21.15.7:0/2055895584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1a806aaf0 msgr2=0x7fd1a8156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.303+0000 7fd1af55b700 1 --2- 172.21.15.7:0/2055895584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1a806aaf0 0x7fd1a8156710 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd19800c500 tx=0x7fd19800c810 comp rx=0 tx=0).stop 2024-09-13T18:54:06.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.303+0000 7fd1af55b700 1 -- 172.21.15.7:0/2055895584 shutdown_connections 2024-09-13T18:54:06.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.303+0000 7fd1af55b700 1 --2- 172.21.15.7:0/2055895584 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd18c066730 0x7fd18c068bf0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.303+0000 7fd1af55b700 1 --2- 172.21.15.7:0/2055895584 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd1a80b3ae0 0x7fd1a8071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.303+0000 7fd1af55b700 1 --2- 172.21.15.7:0/2055895584 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd1a806bbf0 0x7fd1a80adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.303+0000 7fd1af55b700 1 --2- 172.21.15.7:0/2055895584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1a806aaf0 0x7fd1a8156710 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.303+0000 7fd1af55b700 1 -- 172.21.15.7:0/2055895584 >> 172.21.15.7:0/2055895584 conn(0x7fd1a80b0f00 msgr2=0x7fd1a8158a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.303+0000 7fd1af55b700 1 -- 172.21.15.7:0/2055895584 shutdown_connections 2024-09-13T18:54:06.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.303+0000 7fd1af55b700 1 -- 172.21.15.7:0/2055895584 wait complete. 2024-09-13T18:54:06.324 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:54:06.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:54:06.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.439+0000 7ffa1a441700 1 Processor -- start 2024-09-13T18:54:06.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.439+0000 7ffa1a441700 1 -- start start 2024-09-13T18:54:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.439+0000 7ffa1a441700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffa14072390 0x7ffa14165740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.439+0000 7ffa1a441700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa140cdf30 con 0x7ffa140b4040 2024-09-13T18:54:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.439+0000 7ffa1a441700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa140b4540 con 0x7ffa14144590 2024-09-13T18:54:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.439+0000 7ffa1a441700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa140b3080 con 0x7ffa14072390 2024-09-13T18:54:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.439+0000 7ffa189de700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffa14072390 0x7ffa14165740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.439+0000 7ffa189de700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffa14072390 0x7ffa14165740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39288/0 (socket says 172.21.15.7:39288) 2024-09-13T18:54:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.439+0000 7ffa189de700 1 -- 172.21.15.7:0/3089712621 learned_addr learned my addr 172.21.15.7:0/3089712621 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.439+0000 7ffa189de700 1 -- 172.21.15.7:0/3089712621 >> v1:172.21.15.171:6789/0 conn(0x7ffa14144590 legacy=0x7ffa140b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.439+0000 7ffa189de700 1 -- 172.21.15.7:0/3089712621 >> v1:172.21.15.7:6789/0 conn(0x7ffa140b4040 legacy=0x7ffa14071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.439+0000 7ffa189de700 1 -- 172.21.15.7:0/3089712621 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa140b4820 con 0x7ffa14072390 2024-09-13T18:54:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.439+0000 7ffa189de700 1 --2- 172.21.15.7:0/3089712621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa14072390 0x7ffa14165740 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009e50 tx=0x7ffa0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f42ba7d8edf6cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:06.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.443+0000 7ffa12ffd700 1 -- 172.21.15.7:0/3089712621 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c00e070 con 0x7ffa14072390 2024-09-13T18:54:06.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.443+0000 7ffa12ffd700 1 -- 172.21.15.7:0/3089712621 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0c00b9c0 con 0x7ffa14072390 2024-09-13T18:54:06.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.443+0000 7ffa12ffd700 1 -- 172.21.15.7:0/3089712621 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c00bd70 con 0x7ffa14072390 2024-09-13T18:54:06.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.443+0000 7ffa1a441700 1 -- 172.21.15.7:0/3089712621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa14072390 msgr2=0x7ffa14165740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.443+0000 7ffa1a441700 1 --2- 172.21.15.7:0/3089712621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa14072390 0x7ffa14165740 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009e50 tx=0x7ffa0c00b330 comp rx=0 tx=0).stop 2024-09-13T18:54:06.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.443+0000 7ffa1a441700 1 -- 172.21.15.7:0/3089712621 shutdown_connections 2024-09-13T18:54:06.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.443+0000 7ffa1a441700 1 --2- 172.21.15.7:0/3089712621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa14072390 0x7ffa14165740 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.443+0000 7ffa1a441700 1 -- 172.21.15.7:0/3089712621 >> 172.21.15.7:0/3089712621 conn(0x7ffa140b1810 msgr2=0x7ffa140b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:06.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.443+0000 7ffa1a441700 1 -- 172.21.15.7:0/3089712621 shutdown_connections 2024-09-13T18:54:06.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.443+0000 7ffa1a441700 1 -- 172.21.15.7:0/3089712621 wait complete. 2024-09-13T18:54:06.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.443+0000 7ffa1a441700 1 Processor -- start 2024-09-13T18:54:06.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa1a441700 1 -- start start 2024-09-13T18:54:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa1a441700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffa14072390 0x7ffa1413b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa1a441700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa140b4040 0x7ffa140b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa1a441700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa14144590 0x7ffa14141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa1a441700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1413b9f0 con 0x7ffa140b4040 2024-09-13T18:54:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa1a441700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa140b1300 con 0x7ffa14144590 2024-09-13T18:54:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa1a441700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa140b1580 con 0x7ffa14072390 2024-09-13T18:54:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa137fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa140b4040 0x7ffa140b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa137fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa140b4040 0x7ffa140b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39292/0 (socket says 172.21.15.7:39292) 2024-09-13T18:54:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa137fe700 1 -- 172.21.15.7:0/3131770536 learned_addr learned my addr 172.21.15.7:0/3131770536 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:06.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa13fff700 1 --2- 172.21.15.7:0/3131770536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffa14072390 0x7ffa1413b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa189de700 1 --2- 172.21.15.7:0/3131770536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa14144590 0x7ffa14141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa137fe700 1 -- 172.21.15.7:0/3131770536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffa14072390 msgr2=0x7ffa1413b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa137fe700 1 --2- 172.21.15.7:0/3131770536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffa14072390 0x7ffa1413b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa137fe700 1 -- 172.21.15.7:0/3131770536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa14144590 msgr2=0x7ffa14141ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa137fe700 1 --2- 172.21.15.7:0/3131770536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa14144590 0x7ffa14141ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa137fe700 1 -- 172.21.15.7:0/3131770536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0c009b30 con 0x7ffa140b4040 2024-09-13T18:54:06.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa137fe700 1 --2- 172.21.15.7:0/3131770536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa140b4040 0x7ffa140b0dc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800c4d0 tx=0x7ffa0800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa10ff9700 1 -- 172.21.15.7:0/3131770536 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa080052d0 con 0x7ffa140b4040 2024-09-13T18:54:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa10ff9700 1 -- 172.21.15.7:0/3131770536 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa08005430 con 0x7ffa140b4040 2024-09-13T18:54:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa10ff9700 1 -- 172.21.15.7:0/3131770536 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa080055a0 con 0x7ffa140b4040 2024-09-13T18:54:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa1a441700 1 -- 172.21.15.7:0/3131770536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa14141110 con 0x7ffa140b4040 2024-09-13T18:54:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.447+0000 7ffa1a441700 1 -- 172.21.15.7:0/3131770536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa141416d0 con 0x7ffa140b4040 2024-09-13T18:54:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.451+0000 7ffa1a441700 1 -- 172.21.15.7:0/3131770536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa14071c80 con 0x7ffa140b4040 2024-09-13T18:54:06.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.451+0000 7ffa10ff9700 1 -- 172.21.15.7:0/3131770536 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ffa080038c0 con 0x7ffa140b4040 2024-09-13T18:54:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.451+0000 7ffa10ff9700 1 --2- 172.21.15.7:0/3131770536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ffa00066730 0x7ffa00068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.451+0000 7ffa10ff9700 1 -- 172.21.15.7:0/3131770536 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ffa080831c0 con 0x7ffa140b4040 2024-09-13T18:54:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.451+0000 7ffa13fff700 1 --2- 172.21.15.7:0/3131770536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ffa00066730 0x7ffa00068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.451+0000 7ffa13fff700 1 --2- 172.21.15.7:0/3131770536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ffa00066730 0x7ffa00068bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffa04007fd0 tx=0x7ffa04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:06.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.459+0000 7ffa10ff9700 1 -- 172.21.15.7:0/3131770536 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ffa08050800 con 0x7ffa140b4040 2024-09-13T18:54:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.635+0000 7ffa1a441700 1 -- 172.21.15.7:0/3131770536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffa140afff0 con 0x7ffa140b4040 2024-09-13T18:54:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.635+0000 7ffa10ff9700 1 -- 172.21.15.7:0/3131770536 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffa08081e60 con 0x7ffa140b4040 2024-09-13T18:54:06.643 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:54:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.635+0000 7ffa1a441700 1 -- 172.21.15.7:0/3131770536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ffa00066730 msgr2=0x7ffa00068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.635+0000 7ffa1a441700 1 --2- 172.21.15.7:0/3131770536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ffa00066730 0x7ffa00068bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffa04007fd0 tx=0x7ffa04009040 comp rx=0 tx=0).stop 2024-09-13T18:54:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.635+0000 7ffa1a441700 1 -- 172.21.15.7:0/3131770536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa140b4040 msgr2=0x7ffa140b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.635+0000 7ffa1a441700 1 --2- 172.21.15.7:0/3131770536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa140b4040 0x7ffa140b0dc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800c4d0 tx=0x7ffa0800c7e0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.635+0000 7ffa1a441700 1 -- 172.21.15.7:0/3131770536 shutdown_connections 2024-09-13T18:54:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.635+0000 7ffa1a441700 1 --2- 172.21.15.7:0/3131770536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ffa00066730 0x7ffa00068bf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.635+0000 7ffa1a441700 1 --2- 172.21.15.7:0/3131770536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ffa14144590 0x7ffa14141ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.635+0000 7ffa1a441700 1 --2- 172.21.15.7:0/3131770536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa140b4040 0x7ffa140b0dc0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.635+0000 7ffa1a441700 1 --2- 172.21.15.7:0/3131770536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ffa14072390 0x7ffa1413b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.635+0000 7ffa1a441700 1 -- 172.21.15.7:0/3131770536 >> 172.21.15.7:0/3131770536 conn(0x7ffa140b1810 msgr2=0x7ffa140b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.635+0000 7ffa1a441700 1 -- 172.21.15.7:0/3131770536 shutdown_connections 2024-09-13T18:54:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.635+0000 7ffa1a441700 1 -- 172.21.15.7:0/3131770536 wait complete. 2024-09-13T18:54:06.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T18:54:06.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.767+0000 7f6029e66700 1 Processor -- start 2024-09-13T18:54:06.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.767+0000 7f6029e66700 1 -- start start 2024-09-13T18:54:06.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f6029e66700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f60240b3ae0 0x7f602406cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f6029e66700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602406af40 con 0x7f602406aa90 2024-09-13T18:54:06.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f6029e66700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602406d690 con 0x7f6024159720 2024-09-13T18:54:06.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f6029e66700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60240cbfe0 con 0x7f60240b3ae0 2024-09-13T18:54:06.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f6029665700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f602406aa90 0x7f60240ccac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57870/0 (socket says 172.21.15.7:57870) 2024-09-13T18:54:06.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f6029665700 1 -- 172.21.15.7:0/3589035671 learned_addr learned my addr 172.21.15.7:0/3589035671 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:06.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f6023fff700 1 --2- 172.21.15.7:0/3589035671 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f60240b3ae0 0x7f602406cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f60237fe700 1 -- 172.21.15.7:0/3589035671 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228657167 0 0) 0x7f602406af40 con 0x7f602406aa90 2024-09-13T18:54:06.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f60237fe700 1 -- 172.21.15.7:0/3589035671 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60100035d0 con 0x7f602406aa90 2024-09-13T18:54:06.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f60237fe700 1 -- 172.21.15.7:0/3589035671 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2003695689 0 0) 0x7f602406d690 con 0x7f6024159720 2024-09-13T18:54:06.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f60237fe700 1 -- 172.21.15.7:0/3589035671 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f602406af40 con 0x7f6024159720 2024-09-13T18:54:06.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f60237fe700 1 -- 172.21.15.7:0/3589035671 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1791009236 0 0) 0x7f60100035d0 con 0x7f602406aa90 2024-09-13T18:54:06.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f6023fff700 1 -- 172.21.15.7:0/3589035671 >> v1:172.21.15.171:6789/0 conn(0x7f6024159720 legacy=0x7f60240b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f6023fff700 1 -- 172.21.15.7:0/3589035671 >> v1:172.21.15.7:6789/0 conn(0x7f602406aa90 legacy=0x7f60240ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f6023fff700 1 -- 172.21.15.7:0/3589035671 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60240cc2c0 con 0x7f60240b3ae0 2024-09-13T18:54:06.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f6023fff700 1 --2- 172.21.15.7:0/3589035671 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f60240b3ae0 0x7f602406cf90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f600c009880 tx=0x7f600c009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1830ff3110c031a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:06.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f60237fe700 1 -- 172.21.15.7:0/3589035671 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c0019f0 con 0x7f60240b3ae0 2024-09-13T18:54:06.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f60237fe700 1 -- 172.21.15.7:0/3589035671 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c004030 con 0x7f60240b3ae0 2024-09-13T18:54:06.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f60237fe700 1 -- 172.21.15.7:0/3589035671 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c0043a0 con 0x7f60240b3ae0 2024-09-13T18:54:06.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f6029e66700 1 -- 172.21.15.7:0/3589035671 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f60240b3ae0 msgr2=0x7f602406cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f6029e66700 1 --2- 172.21.15.7:0/3589035671 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f60240b3ae0 0x7f602406cf90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f600c009880 tx=0x7f600c009b90 comp rx=0 tx=0).stop 2024-09-13T18:54:06.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f6029e66700 1 -- 172.21.15.7:0/3589035671 shutdown_connections 2024-09-13T18:54:06.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f6029e66700 1 --2- 172.21.15.7:0/3589035671 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f60240b3ae0 0x7f602406cf90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f6029e66700 1 -- 172.21.15.7:0/3589035671 >> 172.21.15.7:0/3589035671 conn(0x7f60240b0f00 msgr2=0x7f60240b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:06.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f6029e66700 1 -- 172.21.15.7:0/3589035671 shutdown_connections 2024-09-13T18:54:06.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.771+0000 7f6029e66700 1 -- 172.21.15.7:0/3589035671 wait complete. 2024-09-13T18:54:06.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6029e66700 1 Processor -- start 2024-09-13T18:54:06.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6029e66700 1 -- start start 2024-09-13T18:54:06.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6029e66700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f602406aa90 0x7f602413c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6029e66700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60240b3ae0 0x7f60241e35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6029e66700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6024159720 0x7f602413e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6029e66700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602413e150 con 0x7f6024159720 2024-09-13T18:54:06.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6029e66700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602413ce50 con 0x7f60240b3ae0 2024-09-13T18:54:06.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6029e66700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602413d4d0 con 0x7f602406aa90 2024-09-13T18:54:06.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6029665700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6024159720 0x7f602413e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6029665700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6024159720 0x7f602413e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39302/0 (socket says 172.21.15.7:39302) 2024-09-13T18:54:06.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6029665700 1 -- 172.21.15.7:0/1000162843 learned_addr learned my addr 172.21.15.7:0/1000162843 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:06.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6023fff700 1 --2- 172.21.15.7:0/1000162843 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60240b3ae0 0x7f60241e35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6028e64700 1 --2- 172.21.15.7:0/1000162843 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f602406aa90 0x7f602413c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6029665700 1 -- 172.21.15.7:0/1000162843 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f602406aa90 msgr2=0x7f602413c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6029665700 1 --2- 172.21.15.7:0/1000162843 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f602406aa90 0x7f602413c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6029665700 1 -- 172.21.15.7:0/1000162843 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60240b3ae0 msgr2=0x7f60241e35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6029665700 1 --2- 172.21.15.7:0/1000162843 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60240b3ae0 0x7f60241e35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6029665700 1 -- 172.21.15.7:0/1000162843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c0094b0 con 0x7f6024159720 2024-09-13T18:54:06.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.775+0000 7f6029665700 1 --2- 172.21.15.7:0/1000162843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6024159720 0x7f602413e570 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f601c00cc70 tx=0x7f601c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:06.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.779+0000 7f60217fa700 1 -- 172.21.15.7:0/1000162843 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c00ad80 con 0x7f6024159720 2024-09-13T18:54:06.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.779+0000 7f60217fa700 1 -- 172.21.15.7:0/1000162843 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c00ce70 con 0x7f6024159720 2024-09-13T18:54:06.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.779+0000 7f6029e66700 1 -- 172.21.15.7:0/1000162843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602413d7b0 con 0x7f6024159720 2024-09-13T18:54:06.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.779+0000 7f60217fa700 1 -- 172.21.15.7:0/1000162843 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c0072f0 con 0x7f6024159720 2024-09-13T18:54:06.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.779+0000 7f6029e66700 1 -- 172.21.15.7:0/1000162843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602413dd70 con 0x7f6024159720 2024-09-13T18:54:06.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.779+0000 7f6029e66700 1 -- 172.21.15.7:0/1000162843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6024158ed0 con 0x7f6024159720 2024-09-13T18:54:06.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.779+0000 7f60217fa700 1 -- 172.21.15.7:0/1000162843 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f601c007450 con 0x7f6024159720 2024-09-13T18:54:06.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.779+0000 7f60217fa700 1 --2- 172.21.15.7:0/1000162843 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6004066730 0x7f6004068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:06.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.783+0000 7f60217fa700 1 -- 172.21.15.7:0/1000162843 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f601c083c60 con 0x7f6024159720 2024-09-13T18:54:06.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.783+0000 7f6028e64700 1 --2- 172.21.15.7:0/1000162843 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6004066730 0x7f6004068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:06.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.783+0000 7f6028e64700 1 --2- 172.21.15.7:0/1000162843 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6004066730 0x7f6004068bf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f60140011e0 tx=0x7f6014009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:06.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.787+0000 7f60217fa700 1 -- 172.21.15.7:0/1000162843 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f601c0513b0 con 0x7f6024159720 2024-09-13T18:54:06.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.959+0000 7f6029e66700 1 -- 172.21.15.7:0/1000162843 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f60240cb550 con 0x7f6024159720 2024-09-13T18:54:06.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.959+0000 7f60217fa700 1 -- 172.21.15.7:0/1000162843 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v48) v1 ==== 84+0+65 (secure 0 0 0) 0x7f601c018090 con 0x7f6024159720 2024-09-13T18:54:06.969 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:06.969 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-13T18:54:06.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-13T18:54:06.969 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-09-13T18:54:06.969 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-09-13T18:54:06.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:06.969 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-13T18:54:06.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.963+0000 7f6029e66700 1 -- 172.21.15.7:0/1000162843 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6004066730 msgr2=0x7f6004068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.963+0000 7f6029e66700 1 --2- 172.21.15.7:0/1000162843 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6004066730 0x7f6004068bf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f60140011e0 tx=0x7f6014009040 comp rx=0 tx=0).stop 2024-09-13T18:54:06.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.963+0000 7f6029e66700 1 -- 172.21.15.7:0/1000162843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6024159720 msgr2=0x7f602413e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:06.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.963+0000 7f6029e66700 1 --2- 172.21.15.7:0/1000162843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6024159720 0x7f602413e570 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f601c00cc70 tx=0x7f601c00a340 comp rx=0 tx=0).stop 2024-09-13T18:54:06.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.963+0000 7f6029e66700 1 -- 172.21.15.7:0/1000162843 shutdown_connections 2024-09-13T18:54:06.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.963+0000 7f6029e66700 1 --2- 172.21.15.7:0/1000162843 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6004066730 0x7f6004068bf0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.963+0000 7f6029e66700 1 --2- 172.21.15.7:0/1000162843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6024159720 0x7f602413e570 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.963+0000 7f6029e66700 1 --2- 172.21.15.7:0/1000162843 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60240b3ae0 0x7f60241e35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.963+0000 7f6029e66700 1 --2- 172.21.15.7:0/1000162843 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f602406aa90 0x7f602413c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:06.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.963+0000 7f6029e66700 1 -- 172.21.15.7:0/1000162843 >> 172.21.15.7:0/1000162843 conn(0x7f60240b0f00 msgr2=0x7f60241e6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:06.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.963+0000 7f6029e66700 1 -- 172.21.15.7:0/1000162843 shutdown_connections 2024-09-13T18:54:06.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:06.963+0000 7f6029e66700 1 -- 172.21.15.7:0/1000162843 wait complete. 2024-09-13T18:54:06.984 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T18:54:06.984 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T18:54:07.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.091+0000 7fcbaa7f5700 1 Processor -- start 2024-09-13T18:54:07.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.091+0000 7fcbaa7f5700 1 -- start start 2024-09-13T18:54:07.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.091+0000 7fcbaa7f5700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcba40711e0 0x7fcba40715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:07.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.091+0000 7fcbaa7f5700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fcba40cd750 0x7fcba40cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.091+0000 7fcbaa7f5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba415be20 con 0x7fcba406a310 2024-09-13T18:54:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.091+0000 7fcbaa7f5700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba413ff30 con 0x7fcba40cd750 2024-09-13T18:54:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.091+0000 7fcbaa7f5700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba4071b00 con 0x7fcba40711e0 2024-09-13T18:54:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.091+0000 7fcba9ff4700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcba406a310 0x7fcba406a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57878/0 (socket says 172.21.15.7:57878) 2024-09-13T18:54:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.091+0000 7fcba9ff4700 1 -- 172.21.15.7:0/1367626413 learned_addr learned my addr 172.21.15.7:0/1367626413 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.091+0000 7fcba8ff2700 1 --2- 172.21.15.7:0/1367626413 >> v2:172.21.15.171:3300/0 conn(0x7fcba40cd750 0x7fcba40cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:07.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.091+0000 7fcba97f3700 1 --2- 172.21.15.7:0/1367626413 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcba40711e0 0x7fcba40715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.095+0000 7fcba8ff2700 1 -- 172.21.15.7:0/1367626413 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcba40711e0 msgr2=0x7fcba40715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.095+0000 7fcba8ff2700 1 --2- 172.21.15.7:0/1367626413 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcba40711e0 0x7fcba40715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.095+0000 7fcba8ff2700 1 -- 172.21.15.7:0/1367626413 >> v1:172.21.15.7:6789/0 conn(0x7fcba406a310 legacy=0x7fcba406a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.095+0000 7fcb97fff700 1 -- 172.21.15.7:0/1367626413 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455336803 0 0) 0x7fcba415be20 con 0x7fcba406a310 2024-09-13T18:54:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.095+0000 7fcba8ff2700 1 -- 172.21.15.7:0/1367626413 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba4140180 con 0x7fcba40cd750 2024-09-13T18:54:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.095+0000 7fcba8ff2700 1 --2- 172.21.15.7:0/1367626413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcba40cd750 0x7fcba40cdbb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009850 tx=0x7fcb98009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa3fc312f744ef52 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.095+0000 7fcb97fff700 1 -- 172.21.15.7:0/1367626413 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98011830 con 0x7fcba40cd750 2024-09-13T18:54:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.095+0000 7fcb97fff700 1 -- 172.21.15.7:0/1367626413 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb98011990 con 0x7fcba40cd750 2024-09-13T18:54:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.095+0000 7fcb97fff700 1 -- 172.21.15.7:0/1367626413 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb98011d40 con 0x7fcba40cd750 2024-09-13T18:54:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.095+0000 7fcbaa7f5700 1 -- 172.21.15.7:0/1367626413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcba40cd750 msgr2=0x7fcba40cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.095+0000 7fcbaa7f5700 1 --2- 172.21.15.7:0/1367626413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcba40cd750 0x7fcba40cdbb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009850 tx=0x7fcb98009b60 comp rx=0 tx=0).stop 2024-09-13T18:54:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.095+0000 7fcbaa7f5700 1 -- 172.21.15.7:0/1367626413 shutdown_connections 2024-09-13T18:54:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.095+0000 7fcbaa7f5700 1 --2- 172.21.15.7:0/1367626413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcba40cd750 0x7fcba40cdbb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.095+0000 7fcbaa7f5700 1 --2- 172.21.15.7:0/1367626413 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcba40711e0 0x7fcba40715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.095+0000 7fcbaa7f5700 1 -- 172.21.15.7:0/1367626413 >> 172.21.15.7:0/1367626413 conn(0x7fcba40b0cc0 msgr2=0x7fcba406fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.095+0000 7fcbaa7f5700 1 -- 172.21.15.7:0/1367626413 shutdown_connections 2024-09-13T18:54:07.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.095+0000 7fcbaa7f5700 1 -- 172.21.15.7:0/1367626413 wait complete. 2024-09-13T18:54:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.099+0000 7fcbaa7f5700 1 Processor -- start 2024-09-13T18:54:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.099+0000 7fcbaa7f5700 1 -- start start 2024-09-13T18:54:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.099+0000 7fcbaa7f5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcba406a310 0x7fcba415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.099+0000 7fcbaa7f5700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcba40711e0 0x7fcba41eff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.099+0000 7fcbaa7f5700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcba40cd750 0x7fcba40a8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.099+0000 7fcbaa7f5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba41f1650 con 0x7fcba406a310 2024-09-13T18:54:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.099+0000 7fcbaa7f5700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba41f0a20 con 0x7fcba40cd750 2024-09-13T18:54:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.099+0000 7fcbaa7f5700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba41f0d00 con 0x7fcba40711e0 2024-09-13T18:54:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.099+0000 7fcba97f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcba406a310 0x7fcba415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.099+0000 7fcba97f3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcba406a310 0x7fcba415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39316/0 (socket says 172.21.15.7:39316) 2024-09-13T18:54:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.099+0000 7fcba8ff2700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcba40711e0 0x7fcba41eff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.099+0000 7fcba97f3700 1 -- 172.21.15.7:0/366117213 learned_addr learned my addr 172.21.15.7:0/366117213 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.099+0000 7fcba9ff4700 1 --2- 172.21.15.7:0/366117213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcba40cd750 0x7fcba40a8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:07.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.099+0000 7fcba97f3700 1 -- 172.21.15.7:0/366117213 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcba40711e0 msgr2=0x7fcba41eff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:07.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.099+0000 7fcba97f3700 1 --2- 172.21.15.7:0/366117213 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcba40711e0 0x7fcba41eff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:07.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.099+0000 7fcba97f3700 1 -- 172.21.15.7:0/366117213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcba40cd750 msgr2=0x7fcba40a8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:07.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.099+0000 7fcba97f3700 1 --2- 172.21.15.7:0/366117213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcba40cd750 0x7fcba40a8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:07.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.099+0000 7fcba97f3700 1 -- 172.21.15.7:0/366117213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb980094b0 con 0x7fcba406a310 2024-09-13T18:54:07.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.103+0000 7fcba9ff4700 1 --2- 172.21.15.7:0/366117213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcba40cd750 0x7fcba40a8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.103+0000 7fcba97f3700 1 --2- 172.21.15.7:0/366117213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcba406a310 0x7fcba415d5d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcba000e9d0 tx=0x7fcba000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.103+0000 7fcb95ffb700 1 -- 172.21.15.7:0/366117213 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0009720 con 0x7fcba406a310 2024-09-13T18:54:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.103+0000 7fcb95ffb700 1 -- 172.21.15.7:0/366117213 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba000ee70 con 0x7fcba406a310 2024-09-13T18:54:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.103+0000 7fcb95ffb700 1 -- 172.21.15.7:0/366117213 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba000ca50 con 0x7fcba406a310 2024-09-13T18:54:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.103+0000 7fcbaa7f5700 1 -- 172.21.15.7:0/366117213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba41f12e0 con 0x7fcba406a310 2024-09-13T18:54:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.103+0000 7fcbaa7f5700 1 -- 172.21.15.7:0/366117213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba41f0fe0 con 0x7fcba406a310 2024-09-13T18:54:07.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.103+0000 7fcb95ffb700 1 -- 172.21.15.7:0/366117213 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fcba000cbb0 con 0x7fcba406a310 2024-09-13T18:54:07.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.103+0000 7fcbaa7f5700 1 -- 172.21.15.7:0/366117213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba406bea0 con 0x7fcba406a310 2024-09-13T18:54:07.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.103+0000 7fcb95ffb700 1 --2- 172.21.15.7:0/366117213 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb88066760 0x7fcb88068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:07.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.103+0000 7fcb95ffb700 1 -- 172.21.15.7:0/366117213 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcba0083dd0 con 0x7fcba406a310 2024-09-13T18:54:07.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.103+0000 7fcba8ff2700 1 --2- 172.21.15.7:0/366117213 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb88066760 0x7fcb88068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:07.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.107+0000 7fcba8ff2700 1 --2- 172.21.15.7:0/366117213 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb88066760 0x7fcb88068c20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009850 tx=0x7fcb98008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.111+0000 7fcb95ffb700 1 -- 172.21.15.7:0/366117213 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcba0051520 con 0x7fcba406a310 2024-09-13T18:54:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.283+0000 7fcbaa7f5700 1 -- 172.21.15.7:0/366117213 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcba40ae850 con 0x7fcba406a310 2024-09-13T18:54:07.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.603+0000 7fcb95ffb700 1 -- 172.21.15.7:0/366117213 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v49) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcba000ce60 con 0x7fcba406a310 2024-09-13T18:54:07.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.611+0000 7fcbaa7f5700 1 -- 172.21.15.7:0/366117213 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb88066760 msgr2=0x7fcb88068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:07.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.611+0000 7fcbaa7f5700 1 --2- 172.21.15.7:0/366117213 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb88066760 0x7fcb88068c20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009850 tx=0x7fcb98008030 comp rx=0 tx=0).stop 2024-09-13T18:54:07.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.611+0000 7fcbaa7f5700 1 -- 172.21.15.7:0/366117213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcba406a310 msgr2=0x7fcba415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:07.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.611+0000 7fcbaa7f5700 1 --2- 172.21.15.7:0/366117213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcba406a310 0x7fcba415d5d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcba000e9d0 tx=0x7fcba000ece0 comp rx=0 tx=0).stop 2024-09-13T18:54:07.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.611+0000 7fcbaa7f5700 1 -- 172.21.15.7:0/366117213 shutdown_connections 2024-09-13T18:54:07.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.611+0000 7fcbaa7f5700 1 --2- 172.21.15.7:0/366117213 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fcb88066760 0x7fcb88068c20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:07.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.611+0000 7fcbaa7f5700 1 --2- 172.21.15.7:0/366117213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcba40cd750 0x7fcba40a8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:07.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.611+0000 7fcbaa7f5700 1 --2- 172.21.15.7:0/366117213 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fcba40711e0 0x7fcba41eff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:07.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.611+0000 7fcbaa7f5700 1 --2- 172.21.15.7:0/366117213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcba406a310 0x7fcba415d5d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:07.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.611+0000 7fcbaa7f5700 1 -- 172.21.15.7:0/366117213 >> 172.21.15.7:0/366117213 conn(0x7fcba40b0cc0 msgr2=0x7fcba413e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:07.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.611+0000 7fcbaa7f5700 1 -- 172.21.15.7:0/366117213 shutdown_connections 2024-09-13T18:54:07.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.611+0000 7fcbaa7f5700 1 -- 172.21.15.7:0/366117213 wait complete. 2024-09-13T18:54:07.621 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-09-13T18:54:07.632 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T18:54:07.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.743+0000 7f5f628fe700 1 Processor -- start 2024-09-13T18:54:07.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.743+0000 7f5f628fe700 1 -- start start 2024-09-13T18:54:07.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f628fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5f5c0b4850 0x7f5f5c06d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:07.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f628fe700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f5c0ce770 con 0x7f5f5c158f50 2024-09-13T18:54:07.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f628fe700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f5c0ce1b0 con 0x7f5f5c06d780 2024-09-13T18:54:07.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f628fe700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f5c06aa20 con 0x7f5f5c0b4850 2024-09-13T18:54:07.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f5b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5f5c0b4850 0x7f5f5c06d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:07.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f5bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5f5c158f50 0x7f5f5c06a310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57888/0 (socket says 172.21.15.7:57888) 2024-09-13T18:54:07.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f5bfff700 1 -- 172.21.15.7:0/4281665028 learned_addr learned my addr 172.21.15.7:0/4281665028 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:07.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f5b7fe700 1 -- 172.21.15.7:0/4281665028 >> v1:172.21.15.171:6789/0 conn(0x7f5f5c06d780 legacy=0x7f5f5c0cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:07.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f5b7fe700 1 -- 172.21.15.7:0/4281665028 >> v1:172.21.15.7:6789/0 conn(0x7f5f5c158f50 legacy=0x7f5f5c06a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:07.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f5affd700 1 -- 172.21.15.7:0/4281665028 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 312370686 0 0) 0x7f5f5c0ce770 con 0x7f5f5c158f50 2024-09-13T18:54:07.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f5b7fe700 1 -- 172.21.15.7:0/4281665028 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c0ce490 con 0x7f5f5c0b4850 2024-09-13T18:54:07.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f5b7fe700 1 --2- 172.21.15.7:0/4281665028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f5c0b4850 0x7f5f5c06d240 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c00a9f0 tx=0x7f5f4c00ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f05983b5487241 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:07.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f5affd700 1 -- 172.21.15.7:0/4281665028 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c00cb70 con 0x7f5f5c0b4850 2024-09-13T18:54:07.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f5affd700 1 -- 172.21.15.7:0/4281665028 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4c00ccd0 con 0x7f5f5c0b4850 2024-09-13T18:54:07.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f5affd700 1 -- 172.21.15.7:0/4281665028 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c018620 con 0x7f5f5c0b4850 2024-09-13T18:54:07.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f628fe700 1 -- 172.21.15.7:0/4281665028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f5c0b4850 msgr2=0x7f5f5c06d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:07.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f628fe700 1 --2- 172.21.15.7:0/4281665028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f5c0b4850 0x7f5f5c06d240 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c00a9f0 tx=0x7f5f4c00ad00 comp rx=0 tx=0).stop 2024-09-13T18:54:07.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f628fe700 1 -- 172.21.15.7:0/4281665028 shutdown_connections 2024-09-13T18:54:07.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f628fe700 1 --2- 172.21.15.7:0/4281665028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f5c0b4850 0x7f5f5c06d240 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:07.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f628fe700 1 -- 172.21.15.7:0/4281665028 >> 172.21.15.7:0/4281665028 conn(0x7f5f5c0af620 msgr2=0x7f5f5c0afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:07.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f628fe700 1 -- 172.21.15.7:0/4281665028 shutdown_connections 2024-09-13T18:54:07.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.747+0000 7f5f628fe700 1 -- 172.21.15.7:0/4281665028 wait complete. 2024-09-13T18:54:07.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f628fe700 1 Processor -- start 2024-09-13T18:54:07.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f628fe700 1 -- start start 2024-09-13T18:54:07.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f628fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f5c06d780 0x7f5f5c1f4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:07.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f628fe700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5f5c0b4850 0x7f5f5c1548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:07.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f628fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f5c158f50 0x7f5f5c0a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:07.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f628fe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c1f3fc0 con 0x7f5f5c06d780 2024-09-13T18:54:07.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f628fe700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c1f4990 con 0x7f5f5c158f50 2024-09-13T18:54:07.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f628fe700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c1f3a00 con 0x7f5f5c0b4850 2024-09-13T18:54:07.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f5bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f5c06d780 0x7f5f5c1f4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:07.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f5bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f5c06d780 0x7f5f5c1f4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39322/0 (socket says 172.21.15.7:39322) 2024-09-13T18:54:07.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f5bfff700 1 -- 172.21.15.7:0/3503444687 learned_addr learned my addr 172.21.15.7:0/3503444687 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:07.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f5b7fe700 1 --2- 172.21.15.7:0/3503444687 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5f5c0b4850 0x7f5f5c1548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:07.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f60e9b700 1 --2- 172.21.15.7:0/3503444687 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f5c158f50 0x7f5f5c0a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:07.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f5bfff700 1 -- 172.21.15.7:0/3503444687 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5f5c0b4850 msgr2=0x7f5f5c1548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:07.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f5bfff700 1 --2- 172.21.15.7:0/3503444687 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5f5c0b4850 0x7f5f5c1548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:07.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f5bfff700 1 -- 172.21.15.7:0/3503444687 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f5c158f50 msgr2=0x7f5f5c0a84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:07.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f5bfff700 1 --2- 172.21.15.7:0/3503444687 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f5c158f50 0x7f5f5c0a84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:07.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f5bfff700 1 -- 172.21.15.7:0/3503444687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c00a650 con 0x7f5f5c06d780 2024-09-13T18:54:07.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f5bfff700 1 --2- 172.21.15.7:0/3503444687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f5c06d780 0x7f5f5c1f4450 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400c4a0 tx=0x7f5f4400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:07.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f58ff9700 1 -- 172.21.15.7:0/3503444687 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44004ca0 con 0x7f5f5c06d780 2024-09-13T18:54:07.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f58ff9700 1 -- 172.21.15.7:0/3503444687 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f44004e00 con 0x7f5f5c06d780 2024-09-13T18:54:07.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f628fe700 1 -- 172.21.15.7:0/3503444687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5c0a8f00 con 0x7f5f5c06d780 2024-09-13T18:54:07.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f58ff9700 1 -- 172.21.15.7:0/3503444687 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44007310 con 0x7f5f5c06d780 2024-09-13T18:54:07.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.751+0000 7f5f628fe700 1 -- 172.21.15.7:0/3503444687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5c1f3c50 con 0x7f5f5c06d780 2024-09-13T18:54:07.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.755+0000 7f5f628fe700 1 -- 172.21.15.7:0/3503444687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f5c06d240 con 0x7f5f5c06d780 2024-09-13T18:54:07.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.755+0000 7f5f58ff9700 1 -- 172.21.15.7:0/3503444687 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5f44007470 con 0x7f5f5c06d780 2024-09-13T18:54:07.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.755+0000 7f5f58ff9700 1 --2- 172.21.15.7:0/3503444687 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5f3c066730 0x7f5f3c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:07.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.755+0000 7f5f58ff9700 1 -- 172.21.15.7:0/3503444687 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f5f44083b80 con 0x7f5f5c06d780 2024-09-13T18:54:07.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.755+0000 7f5f5b7fe700 1 --2- 172.21.15.7:0/3503444687 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5f3c066730 0x7f5f3c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:07.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.759+0000 7f5f5b7fe700 1 --2- 172.21.15.7:0/3503444687 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5f3c066730 0x7f5f3c068bf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c00a1c0 tx=0x7f5f4c01b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:07.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.763+0000 7f5f58ff9700 1 -- 172.21.15.7:0/3503444687 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5f44051030 con 0x7f5f5c06d780 2024-09-13T18:54:07.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:07.935+0000 7f5f628fe700 1 -- 172.21.15.7:0/3503444687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5f5c152d30 con 0x7f5f5c06d780 2024-09-13T18:54:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.615+0000 7f5f58ff9700 1 -- 172.21.15.7:0/3503444687 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v50) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5f44083e60 con 0x7f5f5c06d780 2024-09-13T18:54:08.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.631+0000 7f5f628fe700 1 -- 172.21.15.7:0/3503444687 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5f3c066730 msgr2=0x7f5f3c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:08.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.631+0000 7f5f628fe700 1 --2- 172.21.15.7:0/3503444687 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5f3c066730 0x7f5f3c068bf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c00a1c0 tx=0x7f5f4c01b040 comp rx=0 tx=0).stop 2024-09-13T18:54:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.631+0000 7f5f628fe700 1 -- 172.21.15.7:0/3503444687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f5c06d780 msgr2=0x7f5f5c1f4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.631+0000 7f5f628fe700 1 --2- 172.21.15.7:0/3503444687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f5c06d780 0x7f5f5c1f4450 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400c4a0 tx=0x7f5f4400c7b0 comp rx=0 tx=0).stop 2024-09-13T18:54:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.631+0000 7f5f628fe700 1 -- 172.21.15.7:0/3503444687 shutdown_connections 2024-09-13T18:54:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.631+0000 7f5f628fe700 1 --2- 172.21.15.7:0/3503444687 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5f3c066730 0x7f5f3c068bf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:08.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.631+0000 7f5f628fe700 1 --2- 172.21.15.7:0/3503444687 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5f5c158f50 0x7f5f5c0a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.631+0000 7f5f628fe700 1 --2- 172.21.15.7:0/3503444687 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5f5c0b4850 0x7f5f5c1548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.631+0000 7f5f628fe700 1 --2- 172.21.15.7:0/3503444687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f5c06d780 0x7f5f5c1f4450 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.631+0000 7f5f628fe700 1 -- 172.21.15.7:0/3503444687 >> 172.21.15.7:0/3503444687 conn(0x7f5f5c0af620 msgr2=0x7f5f5c0b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.631+0000 7f5f628fe700 1 -- 172.21.15.7:0/3503444687 shutdown_connections 2024-09-13T18:54:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.631+0000 7f5f628fe700 1 -- 172.21.15.7:0/3503444687 wait complete. 2024-09-13T18:54:08.642 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-09-13T18:54:08.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T18:54:08.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.763+0000 7f1cbbfff700 1 Processor -- start 2024-09-13T18:54:08.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.763+0000 7f1cbbfff700 1 -- start start 2024-09-13T18:54:08.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cbbfff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f1cbc06b900 0x7f1cbc0b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:08.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cbbfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cbc0b8430 con 0x7f1cbc15cda0 2024-09-13T18:54:08.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cbbfff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cbc06bd60 con 0x7f1cbc06a520 2024-09-13T18:54:08.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cbbfff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cbc0cf350 con 0x7f1cbc06b900 2024-09-13T18:54:08.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cbaffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1cbc15cda0 0x7f1cbc0ceb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57892/0 (socket says 172.21.15.7:57892) 2024-09-13T18:54:08.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cbaffd700 1 -- 172.21.15.7:0/2359932984 learned_addr learned my addr 172.21.15.7:0/2359932984 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:08.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cba7fc700 1 --2- 172.21.15.7:0/2359932984 >> v2:172.21.15.171:3300/0 conn(0x7f1cbc06b900 0x7f1cbc0b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:08.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cb9ffb700 1 -- 172.21.15.7:0/2359932984 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213559399 0 0) 0x7f1cbc0b8430 con 0x7f1cbc15cda0 2024-09-13T18:54:08.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cb9ffb700 1 -- 172.21.15.7:0/2359932984 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ca80035d0 con 0x7f1cbc15cda0 2024-09-13T18:54:08.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cb9ffb700 1 -- 172.21.15.7:0/2359932984 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1353231706 0 0) 0x7f1cbc06bd60 con 0x7f1cbc06a520 2024-09-13T18:54:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cb9ffb700 1 -- 172.21.15.7:0/2359932984 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cbc0b8430 con 0x7f1cbc06a520 2024-09-13T18:54:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cba7fc700 1 -- 172.21.15.7:0/2359932984 >> v1:172.21.15.171:6789/0 conn(0x7f1cbc06a520 legacy=0x7f1cbc06a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cba7fc700 1 -- 172.21.15.7:0/2359932984 >> v1:172.21.15.7:6789/0 conn(0x7f1cbc15cda0 legacy=0x7f1cbc0ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cba7fc700 1 -- 172.21.15.7:0/2359932984 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc0d0880 con 0x7f1cbc06b900 2024-09-13T18:54:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cba7fc700 1 --2- 172.21.15.7:0/2359932984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cbc06b900 0x7f1cbc0b5de0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009850 tx=0x7f1cac009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=576139c302aadb98 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cb9ffb700 1 -- 172.21.15.7:0/2359932984 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac011830 con 0x7f1cbc06b900 2024-09-13T18:54:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cb9ffb700 1 -- 172.21.15.7:0/2359932984 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cac011990 con 0x7f1cbc06b900 2024-09-13T18:54:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cb9ffb700 1 -- 172.21.15.7:0/2359932984 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac011d00 con 0x7f1cbc06b900 2024-09-13T18:54:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cbbfff700 1 -- 172.21.15.7:0/2359932984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cbc06b900 msgr2=0x7f1cbc0b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cbbfff700 1 --2- 172.21.15.7:0/2359932984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cbc06b900 0x7f1cbc0b5de0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009850 tx=0x7f1cac009b60 comp rx=0 tx=0).stop 2024-09-13T18:54:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cbbfff700 1 -- 172.21.15.7:0/2359932984 shutdown_connections 2024-09-13T18:54:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cbbfff700 1 --2- 172.21.15.7:0/2359932984 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cbc06b900 0x7f1cbc0b5de0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cbbfff700 1 -- 172.21.15.7:0/2359932984 >> 172.21.15.7:0/2359932984 conn(0x7f1cbc0d2cb0 msgr2=0x7f1cbc0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:08.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cbbfff700 1 -- 172.21.15.7:0/2359932984 shutdown_connections 2024-09-13T18:54:08.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.767+0000 7f1cbbfff700 1 -- 172.21.15.7:0/2359932984 wait complete. 2024-09-13T18:54:08.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cbbfff700 1 Processor -- start 2024-09-13T18:54:08.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cbbfff700 1 -- start start 2024-09-13T18:54:08.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cbbfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cbc06a520 0x7f1cbc1f3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:08.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cbbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cbc06b900 0x7f1cbc0c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:08.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cbbfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cbc15cda0 0x7f1cbc0c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:08.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cbbfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc0c6d20 con 0x7f1cbc06b900 2024-09-13T18:54:08.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cbbfff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc0c6f10 con 0x7f1cbc15cda0 2024-09-13T18:54:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cbbfff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc0c71f0 con 0x7f1cbc06a520 2024-09-13T18:54:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cba7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cbc06b900 0x7f1cbc0c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cba7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cbc06b900 0x7f1cbc0c67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39338/0 (socket says 172.21.15.7:39338) 2024-09-13T18:54:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cba7fc700 1 -- 172.21.15.7:0/3150767036 learned_addr learned my addr 172.21.15.7:0/3150767036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cbaffd700 1 --2- 172.21.15.7:0/3150767036 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cbc06a520 0x7f1cbc1f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cbb7fe700 1 --2- 172.21.15.7:0/3150767036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cbc15cda0 0x7f1cbc0c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cba7fc700 1 -- 172.21.15.7:0/3150767036 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cbc06a520 msgr2=0x7f1cbc1f3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cba7fc700 1 --2- 172.21.15.7:0/3150767036 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cbc06a520 0x7f1cbc1f3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cba7fc700 1 -- 172.21.15.7:0/3150767036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cbc15cda0 msgr2=0x7f1cbc0c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cba7fc700 1 --2- 172.21.15.7:0/3150767036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cbc15cda0 0x7f1cbc0c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cba7fc700 1 -- 172.21.15.7:0/3150767036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac0094b0 con 0x7f1cbc06b900 2024-09-13T18:54:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.771+0000 7f1cba7fc700 1 --2- 172.21.15.7:0/3150767036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cbc06b900 0x7f1cbc0c67e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0c67b0 tx=0x7f1cac004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.775+0000 7f1cb37fe700 1 -- 172.21.15.7:0/3150767036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac0045d0 con 0x7f1cbc06b900 2024-09-13T18:54:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.775+0000 7f1cb37fe700 1 -- 172.21.15.7:0/3150767036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cac01f070 con 0x7f1cbc06b900 2024-09-13T18:54:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.775+0000 7f1cbbfff700 1 -- 172.21.15.7:0/3150767036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cbc0c8c00 con 0x7f1cbc06b900 2024-09-13T18:54:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.775+0000 7f1cb37fe700 1 -- 172.21.15.7:0/3150767036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac0038c0 con 0x7f1cbc06b900 2024-09-13T18:54:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.775+0000 7f1cbbfff700 1 -- 172.21.15.7:0/3150767036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cbc0c88b0 con 0x7f1cbc06b900 2024-09-13T18:54:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.775+0000 7f1cbbfff700 1 -- 172.21.15.7:0/3150767036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cbc0c9120 con 0x7f1cbc06b900 2024-09-13T18:54:08.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.775+0000 7f1cb37fe700 1 -- 172.21.15.7:0/3150767036 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1cac003a20 con 0x7f1cbc06b900 2024-09-13T18:54:08.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.775+0000 7f1cb37fe700 1 --2- 172.21.15.7:0/3150767036 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1c9c066730 0x7f1c9c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:08.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.779+0000 7f1cb37fe700 1 -- 172.21.15.7:0/3150767036 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f1cac021070 con 0x7f1cbc06b900 2024-09-13T18:54:08.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.779+0000 7f1cbaffd700 1 --2- 172.21.15.7:0/3150767036 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1c9c066730 0x7f1c9c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:08.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.779+0000 7f1cbaffd700 1 --2- 172.21.15.7:0/3150767036 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1c9c066730 0x7f1c9c068bf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4000f60 tx=0x7f1ca4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:08.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.783+0000 7f1cb37fe700 1 -- 172.21.15.7:0/3150767036 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1cbc0c9120 con 0x7f1cbc06b900 2024-09-13T18:54:09.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:08.995+0000 7f1cbbfff700 1 -- 172.21.15.7:0/3150767036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1cbc0c9120 con 0x7f1cbc06b900 2024-09-13T18:54:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.011+0000 7f1cb37fe700 1 -- 172.21.15.7:0/3150767036 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v26) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1cbc0c9120 con 0x7f1cbc06b900 2024-09-13T18:54:09.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.027+0000 7f1cbbfff700 1 -- 172.21.15.7:0/3150767036 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1c9c066730 msgr2=0x7f1c9c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.027+0000 7f1cbbfff700 1 --2- 172.21.15.7:0/3150767036 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1c9c066730 0x7f1c9c068bf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4000f60 tx=0x7f1ca4009040 comp rx=0 tx=0).stop 2024-09-13T18:54:09.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.027+0000 7f1cbbfff700 1 -- 172.21.15.7:0/3150767036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cbc06b900 msgr2=0x7f1cbc0c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.027+0000 7f1cbbfff700 1 --2- 172.21.15.7:0/3150767036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cbc06b900 0x7f1cbc0c67e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0c67b0 tx=0x7f1cac004770 comp rx=0 tx=0).stop 2024-09-13T18:54:09.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.031+0000 7f1cbbfff700 1 -- 172.21.15.7:0/3150767036 shutdown_connections 2024-09-13T18:54:09.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.031+0000 7f1cbbfff700 1 --2- 172.21.15.7:0/3150767036 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1c9c066730 0x7f1c9c068bf0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.031+0000 7f1cbbfff700 1 --2- 172.21.15.7:0/3150767036 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cbc15cda0 0x7f1cbc0c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.031+0000 7f1cbbfff700 1 --2- 172.21.15.7:0/3150767036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cbc06b900 0x7f1cbc0c67e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.031+0000 7f1cbbfff700 1 --2- 172.21.15.7:0/3150767036 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1cbc06a520 0x7f1cbc1f3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.031+0000 7f1cbbfff700 1 -- 172.21.15.7:0/3150767036 >> 172.21.15.7:0/3150767036 conn(0x7f1cbc0d2cb0 msgr2=0x7f1cbc0b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:09.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.031+0000 7f1cbbfff700 1 -- 172.21.15.7:0/3150767036 shutdown_connections 2024-09-13T18:54:09.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.031+0000 7f1cbbfff700 1 -- 172.21.15.7:0/3150767036 wait complete. 2024-09-13T18:54:09.042 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 6 and data pool 7 2024-09-13T18:54:09.054 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:54:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53d6cb4700 1 Processor -- start 2024-09-13T18:54:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53d6cb4700 1 -- start start 2024-09-13T18:54:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53d6cb4700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f53d015d600 0x7f53d015da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53d6cb4700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f53d006fd60 0x7f53d0070170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53d6cb4700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d0154490 con 0x7f53d01540b0 2024-09-13T18:54:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53d6cb4700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d015dfb0 con 0x7f53d015d600 2024-09-13T18:54:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53d6cb4700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d015c260 con 0x7f53d006fd60 2024-09-13T18:54:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53cffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f53d015d600 0x7f53d015da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53cffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f53d015d600 0x7f53d015da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39344/0 (socket says 172.21.15.7:39344) 2024-09-13T18:54:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53d4a50700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f53d01540b0 0x7f53d015bb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:39882/0 (socket says 172.21.15.7:39882) 2024-09-13T18:54:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53cffff700 1 -- 172.21.15.7:0/3900760770 learned_addr learned my addr 172.21.15.7:0/3900760770 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53d5251700 1 --2- 172.21.15.7:0/3900760770 >> v2:172.21.15.171:3300/0 conn(0x7f53d006fd60 0x7f53d0070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53cffff700 1 -- 172.21.15.7:0/3900760770 >> v2:172.21.15.171:3300/0 conn(0x7f53d006fd60 msgr2=0x7f53d0070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53cffff700 1 --2- 172.21.15.7:0/3900760770 >> v2:172.21.15.171:3300/0 conn(0x7f53d006fd60 0x7f53d0070170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53cffff700 1 -- 172.21.15.7:0/3900760770 >> v1:172.21.15.171:6789/0 conn(0x7f53d01540b0 legacy=0x7f53d015bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53cffff700 1 -- 172.21.15.7:0/3900760770 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d015c530 con 0x7f53d015d600 2024-09-13T18:54:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53cffff700 1 --2- 172.21.15.7:0/3900760770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d015d600 0x7f53d015da70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f53d00ae5d0 tx=0x7f53c4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8019d0633d1b348f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53cf7fe700 1 -- 172.21.15.7:0/3900760770 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c400b8a0 con 0x7f53d015d600 2024-09-13T18:54:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53cf7fe700 1 -- 172.21.15.7:0/3900760770 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c400ba00 con 0x7f53d015d600 2024-09-13T18:54:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53cf7fe700 1 -- 172.21.15.7:0/3900760770 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c400bd90 con 0x7f53d015d600 2024-09-13T18:54:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53d6cb4700 1 -- 172.21.15.7:0/3900760770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d015d600 msgr2=0x7f53d015da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53d6cb4700 1 --2- 172.21.15.7:0/3900760770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d015d600 0x7f53d015da70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f53d00ae5d0 tx=0x7f53c4009bf0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53d6cb4700 1 -- 172.21.15.7:0/3900760770 shutdown_connections 2024-09-13T18:54:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53d6cb4700 1 --2- 172.21.15.7:0/3900760770 >> v2:172.21.15.171:3300/0 conn(0x7f53d006fd60 0x7f53d0070170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53d6cb4700 1 --2- 172.21.15.7:0/3900760770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d015d600 0x7f53d015da70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53d6cb4700 1 -- 172.21.15.7:0/3900760770 >> 172.21.15.7:0/3900760770 conn(0x7f53d00ce980 msgr2=0x7f53d00ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53d6cb4700 1 -- 172.21.15.7:0/3900760770 shutdown_connections 2024-09-13T18:54:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.147+0000 7f53d6cb4700 1 -- 172.21.15.7:0/3900760770 wait complete. 2024-09-13T18:54:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.151+0000 7f53d6cb4700 1 Processor -- start 2024-09-13T18:54:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.151+0000 7f53d6cb4700 1 -- start start 2024-09-13T18:54:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53d6cb4700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53d006fd60 0x7f53d0148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53d6cb4700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f53d01540b0 0x7f53d014cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53d6cb4700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d015d600 0x7f53d014e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53d6cb4700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d014e2b0 con 0x7f53d015d600 2024-09-13T18:54:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53d6cb4700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d014d6a0 con 0x7f53d006fd60 2024-09-13T18:54:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53d6cb4700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d014d980 con 0x7f53d01540b0 2024-09-13T18:54:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53d5251700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d015d600 0x7f53d014e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53d4a50700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53d006fd60 0x7f53d0148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53d5251700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d015d600 0x7f53d014e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39360/0 (socket says 172.21.15.7:39360) 2024-09-13T18:54:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53d5251700 1 -- 172.21.15.7:0/3958073061 learned_addr learned my addr 172.21.15.7:0/3958073061 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53cffff700 1 --2- 172.21.15.7:0/3958073061 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f53d01540b0 0x7f53d014cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53d5251700 1 -- 172.21.15.7:0/3958073061 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f53d01540b0 msgr2=0x7f53d014cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53d5251700 1 --2- 172.21.15.7:0/3958073061 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f53d01540b0 0x7f53d014cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53d5251700 1 -- 172.21.15.7:0/3958073061 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53d006fd60 msgr2=0x7f53d0148760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53d5251700 1 --2- 172.21.15.7:0/3958073061 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53d006fd60 0x7f53d0148760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53d5251700 1 -- 172.21.15.7:0/3958073061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c4009570 con 0x7f53d015d600 2024-09-13T18:54:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53d5251700 1 --2- 172.21.15.7:0/3958073061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d015d600 0x7f53d014e740 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f53c8007e40 tx=0x7f53c800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53cd7fa700 1 -- 172.21.15.7:0/3958073061 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c800ed30 con 0x7f53d015d600 2024-09-13T18:54:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53cd7fa700 1 -- 172.21.15.7:0/3958073061 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c8014420 con 0x7f53d015d600 2024-09-13T18:54:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53d6cb4700 1 -- 172.21.15.7:0/3958073061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d014dc60 con 0x7f53d015d600 2024-09-13T18:54:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53cd7fa700 1 -- 172.21.15.7:0/3958073061 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c80147d0 con 0x7f53d015d600 2024-09-13T18:54:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.155+0000 7f53d6cb4700 1 -- 172.21.15.7:0/3958073061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d0148ca0 con 0x7f53d015d600 2024-09-13T18:54:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.159+0000 7f53d6cb4700 1 -- 172.21.15.7:0/3958073061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d014e420 con 0x7f53d015d600 2024-09-13T18:54:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.159+0000 7f53cd7fa700 1 -- 172.21.15.7:0/3958073061 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f53c8014930 con 0x7f53d015d600 2024-09-13T18:54:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.159+0000 7f53cd7fa700 1 --2- 172.21.15.7:0/3958073061 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f53bc066730 0x7f53bc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.159+0000 7f53cd7fa700 1 -- 172.21.15.7:0/3958073061 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53c8084850 con 0x7f53d015d600 2024-09-13T18:54:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.159+0000 7f53d4a50700 1 --2- 172.21.15.7:0/3958073061 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f53bc066730 0x7f53bc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.159+0000 7f53d4a50700 1 --2- 172.21.15.7:0/3958073061 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f53bc066730 0x7f53bc068bf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f53c0000f60 tx=0x7f53c0007ef0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:09.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.167+0000 7f53cd7fa700 1 -- 172.21.15.7:0/3958073061 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f53c8051bc0 con 0x7f53d015d600 2024-09-13T18:54:09.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.343+0000 7f53d6cb4700 1 -- 172.21.15.7:0/3958073061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53d015e280 con 0x7f53d015d600 2024-09-13T18:54:09.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.347+0000 7f53cd7fa700 1 -- 172.21.15.7:0/3958073061 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16478 (secure 0 0 0) 0x7f53c80551e0 con 0x7f53d015d600 2024-09-13T18:54:09.355 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:09.356 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":51,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:54:09.003365+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:54:07.330160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T18:54:07.981407+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:54:09.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.347+0000 7f53d6cb4700 1 -- 172.21.15.7:0/3958073061 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f53bc066730 msgr2=0x7f53bc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.347+0000 7f53d6cb4700 1 --2- 172.21.15.7:0/3958073061 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f53bc066730 0x7f53bc068bf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f53c0000f60 tx=0x7f53c0007ef0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.347+0000 7f53d6cb4700 1 -- 172.21.15.7:0/3958073061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d015d600 msgr2=0x7f53d014e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.347+0000 7f53d6cb4700 1 --2- 172.21.15.7:0/3958073061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d015d600 0x7f53d014e740 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f53c8007e40 tx=0x7f53c800e3c0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.351+0000 7f53d6cb4700 1 -- 172.21.15.7:0/3958073061 shutdown_connections 2024-09-13T18:54:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.351+0000 7f53d6cb4700 1 --2- 172.21.15.7:0/3958073061 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f53bc066730 0x7f53bc068bf0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.351+0000 7f53d6cb4700 1 --2- 172.21.15.7:0/3958073061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f53d015d600 0x7f53d014e740 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.351+0000 7f53d6cb4700 1 --2- 172.21.15.7:0/3958073061 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f53d01540b0 0x7f53d014cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.351+0000 7f53d6cb4700 1 --2- 172.21.15.7:0/3958073061 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f53d006fd60 0x7f53d0148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.351+0000 7f53d6cb4700 1 -- 172.21.15.7:0/3958073061 >> 172.21.15.7:0/3958073061 conn(0x7f53d00ce980 msgr2=0x7f53d00cbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.351+0000 7f53d6cb4700 1 -- 172.21.15.7:0/3958073061 shutdown_connections 2024-09-13T18:54:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.351+0000 7f53d6cb4700 1 -- 172.21.15.7:0/3958073061 wait complete. 2024-09-13T18:54:09.371 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:54:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.471+0000 7fb2ba9cd700 1 Processor -- start 2024-09-13T18:54:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.471+0000 7fb2ba9cd700 1 -- start start 2024-09-13T18:54:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2ba9cd700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb2b415cec0 0x7fb2b40cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2ba9cd700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b40d21e0 con 0x7fb2b415cae0 2024-09-13T18:54:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2ba9cd700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b40d1940 con 0x7fb2b406b310 2024-09-13T18:54:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2ba9cd700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b40d1c20 con 0x7fb2b415cec0 2024-09-13T18:54:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2b99cb700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb2b415cae0 0x7fb2b406ac00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57904/0 (socket says 172.21.15.7:57904) 2024-09-13T18:54:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2b99cb700 1 -- 172.21.15.7:0/3087309509 learned_addr learned my addr 172.21.15.7:0/3087309509 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:09.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2b91ca700 1 --2- 172.21.15.7:0/3087309509 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb2b415cec0 0x7fb2b40cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2b89c9700 1 -- 172.21.15.7:0/3087309509 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1952732351 0 0) 0x7fb2b40d21e0 con 0x7fb2b415cae0 2024-09-13T18:54:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2b89c9700 1 -- 172.21.15.7:0/3087309509 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2a00035d0 con 0x7fb2b415cae0 2024-09-13T18:54:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2b89c9700 1 -- 172.21.15.7:0/3087309509 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4229549264 0 0) 0x7fb2b40d1940 con 0x7fb2b406b310 2024-09-13T18:54:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2b89c9700 1 -- 172.21.15.7:0/3087309509 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2b40d21e0 con 0x7fb2b406b310 2024-09-13T18:54:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2b91ca700 1 -- 172.21.15.7:0/3087309509 >> v1:172.21.15.171:6789/0 conn(0x7fb2b406b310 legacy=0x7fb2b40b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2b91ca700 1 -- 172.21.15.7:0/3087309509 >> v1:172.21.15.7:6789/0 conn(0x7fb2b415cae0 legacy=0x7fb2b406ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2b91ca700 1 -- 172.21.15.7:0/3087309509 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b40d1f00 con 0x7fb2b415cec0 2024-09-13T18:54:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2b91ca700 1 --2- 172.21.15.7:0/3087309509 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb2b415cec0 0x7fb2b40cf030 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009880 tx=0x7fb2a4009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecde313bab6f72cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2b89c9700 1 -- 172.21.15.7:0/3087309509 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4004230 con 0x7fb2b415cec0 2024-09-13T18:54:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2b89c9700 1 -- 172.21.15.7:0/3087309509 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a4004390 con 0x7fb2b415cec0 2024-09-13T18:54:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2b89c9700 1 -- 172.21.15.7:0/3087309509 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a40036c0 con 0x7fb2b415cec0 2024-09-13T18:54:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2ba9cd700 1 -- 172.21.15.7:0/3087309509 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb2b415cec0 msgr2=0x7fb2b40cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2ba9cd700 1 --2- 172.21.15.7:0/3087309509 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb2b415cec0 0x7fb2b40cf030 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009880 tx=0x7fb2a4009b90 comp rx=0 tx=0).stop 2024-09-13T18:54:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2ba9cd700 1 -- 172.21.15.7:0/3087309509 shutdown_connections 2024-09-13T18:54:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2ba9cd700 1 --2- 172.21.15.7:0/3087309509 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb2b415cec0 0x7fb2b40cf030 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.475+0000 7fb2ba9cd700 1 -- 172.21.15.7:0/3087309509 >> 172.21.15.7:0/3087309509 conn(0x7fb2b40d2c80 msgr2=0x7fb2b40d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.479+0000 7fb2ba9cd700 1 -- 172.21.15.7:0/3087309509 shutdown_connections 2024-09-13T18:54:09.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.479+0000 7fb2ba9cd700 1 -- 172.21.15.7:0/3087309509 wait complete. 2024-09-13T18:54:09.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.479+0000 7fb2ba9cd700 1 Processor -- start 2024-09-13T18:54:09.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.479+0000 7fb2ba9cd700 1 -- start start 2024-09-13T18:54:09.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2ba9cd700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb2b406b310 0x7fb2b41f6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:09.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2ba9cd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2b415cae0 0x7fb2b415fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:09.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2ba9cd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b415cec0 0x7fb2b4156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:09.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2ba9cd700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b41f69f0 con 0x7fb2b415cec0 2024-09-13T18:54:09.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2ba9cd700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b41f2410 con 0x7fb2b415cae0 2024-09-13T18:54:09.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2ba9cd700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b41f33e0 con 0x7fb2b406b310 2024-09-13T18:54:09.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2ba1cc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b415cec0 0x7fb2b4156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:09.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2ba1cc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b415cec0 0x7fb2b4156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39374/0 (socket says 172.21.15.7:39374) 2024-09-13T18:54:09.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2b91ca700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2b415cae0 0x7fb2b415fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:09.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2ba1cc700 1 -- 172.21.15.7:0/4143902572 learned_addr learned my addr 172.21.15.7:0/4143902572 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:09.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2b99cb700 1 --2- 172.21.15.7:0/4143902572 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb2b406b310 0x7fb2b41f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:09.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2ba1cc700 1 -- 172.21.15.7:0/4143902572 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb2b406b310 msgr2=0x7fb2b41f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2ba1cc700 1 --2- 172.21.15.7:0/4143902572 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb2b406b310 0x7fb2b41f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2ba1cc700 1 -- 172.21.15.7:0/4143902572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2b415cae0 msgr2=0x7fb2b415fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2ba1cc700 1 --2- 172.21.15.7:0/4143902572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2b415cae0 0x7fb2b415fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2ba1cc700 1 -- 172.21.15.7:0/4143902572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a40094b0 con 0x7fb2b415cec0 2024-09-13T18:54:09.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2b91ca700 1 --2- 172.21.15.7:0/4143902572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2b415cae0 0x7fb2b415fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:09.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2ba1cc700 1 --2- 172.21.15.7:0/4143902572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b415cec0 0x7fb2b4156e10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00cbf0 tx=0x7fb2ac00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:09.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2aa7fc700 1 -- 172.21.15.7:0/4143902572 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac003c20 con 0x7fb2b415cec0 2024-09-13T18:54:09.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2aa7fc700 1 -- 172.21.15.7:0/4143902572 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac003da0 con 0x7fb2b415cec0 2024-09-13T18:54:09.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2ba9cd700 1 -- 172.21.15.7:0/4143902572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b4160140 con 0x7fb2b415cec0 2024-09-13T18:54:09.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2aa7fc700 1 -- 172.21.15.7:0/4143902572 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac00a9e0 con 0x7fb2b415cec0 2024-09-13T18:54:09.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2ba9cd700 1 -- 172.21.15.7:0/4143902572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b41f2900 con 0x7fb2b415cec0 2024-09-13T18:54:09.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.483+0000 7fb2ba9cd700 1 -- 172.21.15.7:0/4143902572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b40cf030 con 0x7fb2b415cec0 2024-09-13T18:54:09.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.487+0000 7fb2aa7fc700 1 -- 172.21.15.7:0/4143902572 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb2ac00ab40 con 0x7fb2b415cec0 2024-09-13T18:54:09.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.487+0000 7fb2aa7fc700 1 --2- 172.21.15.7:0/4143902572 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb298066730 0x7fb298068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:09.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.487+0000 7fb2aa7fc700 1 -- 172.21.15.7:0/4143902572 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2ac084100 con 0x7fb2b415cec0 2024-09-13T18:54:09.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.487+0000 7fb2b99cb700 1 --2- 172.21.15.7:0/4143902572 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb298066730 0x7fb298068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:09.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.487+0000 7fb2b99cb700 1 --2- 172.21.15.7:0/4143902572 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb298066730 0x7fb298068bf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0000f60 tx=0x7fb2b0007e00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:09.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.499+0000 7fb2aa7fc700 1 -- 172.21.15.7:0/4143902572 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb2ac0513f0 con 0x7fb2b415cec0 2024-09-13T18:54:09.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.671+0000 7fb2ba9cd700 1 -- 172.21.15.7:0/4143902572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2b40d32b0 con 0x7fb2b415cec0 2024-09-13T18:54:09.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.675+0000 7fb2aa7fc700 1 -- 172.21.15.7:0/4143902572 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16478 (secure 0 0 0) 0x7fb2ac054a10 con 0x7fb2b415cec0 2024-09-13T18:54:09.684 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:09.684 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":51,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:54:09.003365+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:54:07.330160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T18:54:07.981407+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:54:09.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.679+0000 7fb2ba9cd700 1 -- 172.21.15.7:0/4143902572 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb298066730 msgr2=0x7fb298068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.679+0000 7fb2ba9cd700 1 --2- 172.21.15.7:0/4143902572 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb298066730 0x7fb298068bf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0000f60 tx=0x7fb2b0007e00 comp rx=0 tx=0).stop 2024-09-13T18:54:09.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.679+0000 7fb2ba9cd700 1 -- 172.21.15.7:0/4143902572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b415cec0 msgr2=0x7fb2b4156e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.679+0000 7fb2ba9cd700 1 --2- 172.21.15.7:0/4143902572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b415cec0 0x7fb2b4156e10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00cbf0 tx=0x7fb2ac00cf00 comp rx=0 tx=0).stop 2024-09-13T18:54:09.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.679+0000 7fb2ba9cd700 1 -- 172.21.15.7:0/4143902572 shutdown_connections 2024-09-13T18:54:09.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.679+0000 7fb2ba9cd700 1 --2- 172.21.15.7:0/4143902572 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb298066730 0x7fb298068bf0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.679+0000 7fb2ba9cd700 1 --2- 172.21.15.7:0/4143902572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2b415cec0 0x7fb2b4156e10 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.679+0000 7fb2ba9cd700 1 --2- 172.21.15.7:0/4143902572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb2b415cae0 0x7fb2b415fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.679+0000 7fb2ba9cd700 1 --2- 172.21.15.7:0/4143902572 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb2b406b310 0x7fb2b41f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.679+0000 7fb2ba9cd700 1 -- 172.21.15.7:0/4143902572 >> 172.21.15.7:0/4143902572 conn(0x7fb2b40d2c80 msgr2=0x7fb2b40cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:09.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.679+0000 7fb2ba9cd700 1 -- 172.21.15.7:0/4143902572 shutdown_connections 2024-09-13T18:54:09.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.679+0000 7fb2ba9cd700 1 -- 172.21.15.7:0/4143902572 wait complete. 2024-09-13T18:54:09.700 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T18:54:09.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa97e2d6700 1 Processor -- start 2024-09-13T18:54:09.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa97e2d6700 1 -- start start 2024-09-13T18:54:09.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa97e2d6700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa97806b900 0x7fa9780b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:09.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa97e2d6700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa97806a520 0x7fa97806a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:09.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa97e2d6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa97806aeb0 con 0x7fa97815cda0 2024-09-13T18:54:09.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa97e2d6700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9780b8430 con 0x7fa97806a520 2024-09-13T18:54:09.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa97e2d6700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97806bd60 con 0x7fa97806b900 2024-09-13T18:54:09.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa97d2d4700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa97815cda0 0x7fa9780ceb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57908/0 (socket says 172.21.15.7:57908) 2024-09-13T18:54:09.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa97d2d4700 1 -- 172.21.15.7:0/3173161621 learned_addr learned my addr 172.21.15.7:0/3173161621 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:09.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa97cad3700 1 --2- 172.21.15.7:0/3173161621 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa97806b900 0x7fa9780b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:09.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa97dad5700 1 --2- 172.21.15.7:0/3173161621 >> v2:172.21.15.171:3300/0 conn(0x7fa97806a520 0x7fa97806a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa96ffff700 1 -- 172.21.15.7:0/3173161621 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3848195182 0 0) 0x7fa974001110 con 0x7fa97815cda0 2024-09-13T18:54:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa96ffff700 1 -- 172.21.15.7:0/3173161621 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9640035d0 con 0x7fa97815cda0 2024-09-13T18:54:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa97cad3700 1 -- 172.21.15.7:0/3173161621 >> v2:172.21.15.171:3300/0 conn(0x7fa97806a520 msgr2=0x7fa97806a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa97cad3700 1 --2- 172.21.15.7:0/3173161621 >> v2:172.21.15.171:3300/0 conn(0x7fa97806a520 0x7fa97806a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa97cad3700 1 -- 172.21.15.7:0/3173161621 >> v1:172.21.15.7:6789/0 conn(0x7fa97815cda0 legacy=0x7fa9780ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa97cad3700 1 -- 172.21.15.7:0/3173161621 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9780cf350 con 0x7fa97806b900 2024-09-13T18:54:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa97dad5700 1 --2- 172.21.15.7:0/3173161621 >> v2:172.21.15.171:3300/0 conn(0x7fa97806a520 0x7fa97806a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa97cad3700 1 --2- 172.21.15.7:0/3173161621 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa97806b900 0x7fa9780b5de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa968009850 tx=0x7fa968009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ac0d8150a52eae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:09.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa96ffff700 1 -- 172.21.15.7:0/3173161621 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968011830 con 0x7fa97806b900 2024-09-13T18:54:09.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa96ffff700 1 -- 172.21.15.7:0/3173161621 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa968011990 con 0x7fa97806b900 2024-09-13T18:54:09.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.815+0000 7fa96ffff700 1 -- 172.21.15.7:0/3173161621 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968011d00 con 0x7fa97806b900 2024-09-13T18:54:09.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.819+0000 7fa97e2d6700 1 -- 172.21.15.7:0/3173161621 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa97806b900 msgr2=0x7fa9780b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.819+0000 7fa97e2d6700 1 --2- 172.21.15.7:0/3173161621 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa97806b900 0x7fa9780b5de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa968009850 tx=0x7fa968009b60 comp rx=0 tx=0).stop 2024-09-13T18:54:09.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.819+0000 7fa97e2d6700 1 -- 172.21.15.7:0/3173161621 shutdown_connections 2024-09-13T18:54:09.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.819+0000 7fa97e2d6700 1 --2- 172.21.15.7:0/3173161621 >> v2:172.21.15.171:3300/0 conn(0x7fa97806a520 0x7fa97806a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.819+0000 7fa97e2d6700 1 --2- 172.21.15.7:0/3173161621 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa97806b900 0x7fa9780b5de0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.819+0000 7fa97e2d6700 1 -- 172.21.15.7:0/3173161621 >> 172.21.15.7:0/3173161621 conn(0x7fa9780d2cb0 msgr2=0x7fa9780d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:09.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.819+0000 7fa97e2d6700 1 -- 172.21.15.7:0/3173161621 shutdown_connections 2024-09-13T18:54:09.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.819+0000 7fa97e2d6700 1 -- 172.21.15.7:0/3173161621 wait complete. 2024-09-13T18:54:09.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.819+0000 7fa97e2d6700 1 Processor -- start 2024-09-13T18:54:09.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.819+0000 7fa97e2d6700 1 -- start start 2024-09-13T18:54:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97e2d6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa97806a520 0x7fa978154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97e2d6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa97806b900 0x7fa9781521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97e2d6700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa97815cda0 0x7fa978152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97e2d6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa978152c50 con 0x7fa97806b900 2024-09-13T18:54:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97e2d6700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9781559b0 con 0x7fa97806a520 2024-09-13T18:54:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97e2d6700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9780c6880 con 0x7fa97815cda0 2024-09-13T18:54:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97cad3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa97806b900 0x7fa9781521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97cad3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa97806b900 0x7fa9781521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39378/0 (socket says 172.21.15.7:39378) 2024-09-13T18:54:09.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97cad3700 1 -- 172.21.15.7:0/1457151593 learned_addr learned my addr 172.21.15.7:0/1457151593 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97d2d4700 1 --2- 172.21.15.7:0/1457151593 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa97806a520 0x7fa978154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97dad5700 1 --2- 172.21.15.7:0/1457151593 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa97815cda0 0x7fa978152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97cad3700 1 -- 172.21.15.7:0/1457151593 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa97815cda0 msgr2=0x7fa978152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97cad3700 1 --2- 172.21.15.7:0/1457151593 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa97815cda0 0x7fa978152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97cad3700 1 -- 172.21.15.7:0/1457151593 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa97806a520 msgr2=0x7fa978154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97cad3700 1 --2- 172.21.15.7:0/1457151593 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa97806a520 0x7fa978154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97cad3700 1 -- 172.21.15.7:0/1457151593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa970009a70 con 0x7fa97806b900 2024-09-13T18:54:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97d2d4700 1 --2- 172.21.15.7:0/1457151593 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa97806a520 0x7fa978154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97cad3700 1 --2- 172.21.15.7:0/1457151593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa97806b900 0x7fa9781521d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa97806bea0 tx=0x7fa968004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:09.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa96dffb700 1 -- 172.21.15.7:0/1457151593 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9680045d0 con 0x7fa97806b900 2024-09-13T18:54:09.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa96dffb700 1 -- 172.21.15.7:0/1457151593 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96801f070 con 0x7fa97806b900 2024-09-13T18:54:09.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97e2d6700 1 -- 172.21.15.7:0/1457151593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9680094b0 con 0x7fa97806b900 2024-09-13T18:54:09.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa96dffb700 1 -- 172.21.15.7:0/1457151593 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968003900 con 0x7fa97806b900 2024-09-13T18:54:09.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97e2d6700 1 -- 172.21.15.7:0/1457151593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9780c6dd0 con 0x7fa97806b900 2024-09-13T18:54:09.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.823+0000 7fa97e2d6700 1 -- 172.21.15.7:0/1457151593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9780cce60 con 0x7fa97806b900 2024-09-13T18:54:09.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.827+0000 7fa96dffb700 1 -- 172.21.15.7:0/1457151593 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa968003a60 con 0x7fa97806b900 2024-09-13T18:54:09.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.827+0000 7fa96dffb700 1 --2- 172.21.15.7:0/1457151593 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa95c066730 0x7fa95c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:09.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.827+0000 7fa96dffb700 1 -- 172.21.15.7:0/1457151593 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa968021070 con 0x7fa97806b900 2024-09-13T18:54:09.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.827+0000 7fa97d2d4700 1 --2- 172.21.15.7:0/1457151593 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa95c066730 0x7fa95c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:09.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.827+0000 7fa97d2d4700 1 --2- 172.21.15.7:0/1457151593 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa95c066730 0x7fa95c068bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa974005930 tx=0x7fa974009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:09.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:09.831+0000 7fa96dffb700 1 -- 172.21.15.7:0/1457151593 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa968052ce0 con 0x7fa97806b900 2024-09-13T18:54:10.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:10.043+0000 7fa97e2d6700 1 -- 172.21.15.7:0/1457151593 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa9780c6fc0 con 0x7fa97806b900 2024-09-13T18:54:11.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.023+0000 7fa96dffb700 1 -- 172.21.15.7:0/1457151593 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v29) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa9680150b0 con 0x7fa97806b900 2024-09-13T18:54:11.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.027+0000 7fa97e2d6700 1 -- 172.21.15.7:0/1457151593 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa95c066730 msgr2=0x7fa95c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:11.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.027+0000 7fa97e2d6700 1 --2- 172.21.15.7:0/1457151593 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa95c066730 0x7fa95c068bf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa974005930 tx=0x7fa974009040 comp rx=0 tx=0).stop 2024-09-13T18:54:11.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.027+0000 7fa97e2d6700 1 -- 172.21.15.7:0/1457151593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa97806b900 msgr2=0x7fa9781521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:11.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.027+0000 7fa97e2d6700 1 --2- 172.21.15.7:0/1457151593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa97806b900 0x7fa9781521d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa97806bea0 tx=0x7fa968004770 comp rx=0 tx=0).stop 2024-09-13T18:54:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.027+0000 7fa97e2d6700 1 -- 172.21.15.7:0/1457151593 shutdown_connections 2024-09-13T18:54:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.027+0000 7fa97e2d6700 1 --2- 172.21.15.7:0/1457151593 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa95c066730 0x7fa95c068bf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.027+0000 7fa97e2d6700 1 --2- 172.21.15.7:0/1457151593 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa97815cda0 0x7fa978152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.027+0000 7fa97e2d6700 1 --2- 172.21.15.7:0/1457151593 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa97806b900 0x7fa9781521d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.027+0000 7fa97e2d6700 1 --2- 172.21.15.7:0/1457151593 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa97806a520 0x7fa978154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.027+0000 7fa97e2d6700 1 -- 172.21.15.7:0/1457151593 >> 172.21.15.7:0/1457151593 conn(0x7fa9780d2cb0 msgr2=0x7fa97806b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.027+0000 7fa97e2d6700 1 -- 172.21.15.7:0/1457151593 shutdown_connections 2024-09-13T18:54:11.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.027+0000 7fa97e2d6700 1 -- 172.21.15.7:0/1457151593 wait complete. 2024-09-13T18:54:11.053 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T18:54:11.053 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T18:54:11.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.167+0000 7fa2e5c78700 1 Processor -- start 2024-09-13T18:54:11.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.167+0000 7fa2e5c78700 1 -- start start 2024-09-13T18:54:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2e5c78700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa2e006f3a0 0x7fa2e015d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2e5c78700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e006f780 0x7fa2e0142c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2e5c78700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2e01431c0 con 0x7fa2e01401e0 2024-09-13T18:54:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2e5c78700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2e00cc340 con 0x7fa2e006f3a0 2024-09-13T18:54:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2e5c78700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e015d760 con 0x7fa2e006f780 2024-09-13T18:54:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2dffff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e006f780 0x7fa2e0142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2e5477700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7fa2e01401e0 0x7fa2e00b2ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:40712/0 (socket says 172.21.15.7:40712) 2024-09-13T18:54:11.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2dffff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e006f780 0x7fa2e0142c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:47218/0 (socket says 172.21.15.7:47218) 2024-09-13T18:54:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2e5477700 1 -- 172.21.15.7:0/2164322635 learned_addr learned my addr 172.21.15.7:0/2164322635 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2e4c76700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fa2e006f3a0 0x7fa2e015d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2df7fe700 1 -- 172.21.15.7:0/2164322635 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222559463 0 0) 0x7fa2d8001490 con 0x7fa2e01401e0 2024-09-13T18:54:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2df7fe700 1 -- 172.21.15.7:0/2164322635 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c80035d0 con 0x7fa2e01401e0 2024-09-13T18:54:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2dffff700 1 -- 172.21.15.7:0/2164322635 >> v2:172.21.15.171:3300/0 conn(0x7fa2e006f3a0 msgr2=0x7fa2e015d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2dffff700 1 --2- 172.21.15.7:0/2164322635 >> v2:172.21.15.171:3300/0 conn(0x7fa2e006f3a0 0x7fa2e015d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2dffff700 1 -- 172.21.15.7:0/2164322635 >> v1:172.21.15.171:6789/0 conn(0x7fa2e01401e0 legacy=0x7fa2e00b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2dffff700 1 -- 172.21.15.7:0/2164322635 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e015da40 con 0x7fa2e006f780 2024-09-13T18:54:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2dffff700 1 --2- 172.21.15.7:0/2164322635 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e006f780 0x7fa2e0142c80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009850 tx=0x7fa2d0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66032e5f1da7b558 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2df7fe700 1 -- 172.21.15.7:0/2164322635 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0011830 con 0x7fa2e006f780 2024-09-13T18:54:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2df7fe700 1 -- 172.21.15.7:0/2164322635 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d0011990 con 0x7fa2e006f780 2024-09-13T18:54:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2df7fe700 1 -- 172.21.15.7:0/2164322635 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0011d00 con 0x7fa2e006f780 2024-09-13T18:54:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2e5c78700 1 -- 172.21.15.7:0/2164322635 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e006f780 msgr2=0x7fa2e0142c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2e5c78700 1 --2- 172.21.15.7:0/2164322635 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e006f780 0x7fa2e0142c80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009850 tx=0x7fa2d0009b60 comp rx=0 tx=0).stop 2024-09-13T18:54:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2e5c78700 1 -- 172.21.15.7:0/2164322635 shutdown_connections 2024-09-13T18:54:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2e5c78700 1 --2- 172.21.15.7:0/2164322635 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e006f780 0x7fa2e0142c80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2e5c78700 1 --2- 172.21.15.7:0/2164322635 >> v2:172.21.15.171:3300/0 conn(0x7fa2e006f3a0 0x7fa2e015d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2e5c78700 1 -- 172.21.15.7:0/2164322635 >> 172.21.15.7:0/2164322635 conn(0x7fa2e0071ce0 msgr2=0x7fa2e00720c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2e5c78700 1 -- 172.21.15.7:0/2164322635 shutdown_connections 2024-09-13T18:54:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.171+0000 7fa2e5c78700 1 -- 172.21.15.7:0/2164322635 wait complete. 2024-09-13T18:54:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.175+0000 7fa2e5c78700 1 Processor -- start 2024-09-13T18:54:11.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.175+0000 7fa2e5c78700 1 -- start start 2024-09-13T18:54:11.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.175+0000 7fa2e5c78700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2e006f3a0 0x7fa2e01f6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:11.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.175+0000 7fa2e5c78700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e006f780 0x7fa2e0156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:11.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.175+0000 7fa2e5c78700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e01401e0 0x7fa2e013d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:11.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.175+0000 7fa2e5c78700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e01f5bc0 con 0x7fa2e006f780 2024-09-13T18:54:11.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.175+0000 7fa2e5c78700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e013dd90 con 0x7fa2e006f3a0 2024-09-13T18:54:11.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.175+0000 7fa2e5c78700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e0157100 con 0x7fa2e01401e0 2024-09-13T18:54:11.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.179+0000 7fa2dffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e006f780 0x7fa2e0156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:11.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.179+0000 7fa2dffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e006f780 0x7fa2e0156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55374/0 (socket says 172.21.15.7:55374) 2024-09-13T18:54:11.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.179+0000 7fa2dffff700 1 -- 172.21.15.7:0/3476987485 learned_addr learned my addr 172.21.15.7:0/3476987485 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:11.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.179+0000 7fa2e5477700 1 --2- 172.21.15.7:0/3476987485 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e01401e0 0x7fa2e013d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:11.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.179+0000 7fa2e4c76700 1 --2- 172.21.15.7:0/3476987485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2e006f3a0 0x7fa2e01f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:11.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.179+0000 7fa2dffff700 1 -- 172.21.15.7:0/3476987485 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e01401e0 msgr2=0x7fa2e013d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:11.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.179+0000 7fa2dffff700 1 --2- 172.21.15.7:0/3476987485 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e01401e0 0x7fa2e013d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:11.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.179+0000 7fa2dffff700 1 -- 172.21.15.7:0/3476987485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2e006f3a0 msgr2=0x7fa2e01f6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:11.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.179+0000 7fa2dffff700 1 --2- 172.21.15.7:0/3476987485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2e006f3a0 0x7fa2e01f6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:11.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.179+0000 7fa2dffff700 1 -- 172.21.15.7:0/3476987485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d00094b0 con 0x7fa2e006f780 2024-09-13T18:54:11.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.179+0000 7fa2dffff700 1 --2- 172.21.15.7:0/3476987485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e006f780 0x7fa2e0156a20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00090b0 tx=0x7fa2d0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:11.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.179+0000 7fa2dd7fa700 1 -- 172.21.15.7:0/3476987485 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00044d0 con 0x7fa2e006f780 2024-09-13T18:54:11.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.179+0000 7fa2dd7fa700 1 -- 172.21.15.7:0/3476987485 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d001f070 con 0x7fa2e006f780 2024-09-13T18:54:11.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.179+0000 7fa2e5c78700 1 -- 172.21.15.7:0/3476987485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e01407d0 con 0x7fa2e006f780 2024-09-13T18:54:11.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.179+0000 7fa2dd7fa700 1 -- 172.21.15.7:0/3476987485 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0003900 con 0x7fa2e006f780 2024-09-13T18:54:11.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.179+0000 7fa2e5c78700 1 -- 172.21.15.7:0/3476987485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e01f5e10 con 0x7fa2e006f780 2024-09-13T18:54:11.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.179+0000 7fa2e5c78700 1 -- 172.21.15.7:0/3476987485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e0157620 con 0x7fa2e006f780 2024-09-13T18:54:11.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.183+0000 7fa2dd7fa700 1 -- 172.21.15.7:0/3476987485 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa2d0003a60 con 0x7fa2e006f780 2024-09-13T18:54:11.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.183+0000 7fa2dd7fa700 1 --2- 172.21.15.7:0/3476987485 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa2cc066730 0x7fa2cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:11.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.183+0000 7fa2dd7fa700 1 -- 172.21.15.7:0/3476987485 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2d0021070 con 0x7fa2e006f780 2024-09-13T18:54:11.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.183+0000 7fa2e4c76700 1 --2- 172.21.15.7:0/3476987485 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa2cc066730 0x7fa2cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:11.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.183+0000 7fa2e4c76700 1 --2- 172.21.15.7:0/3476987485 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa2cc066730 0x7fa2cc068bf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009bf0 tx=0x7fa2d4009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:11.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.187+0000 7fa2dd7fa700 1 -- 172.21.15.7:0/3476987485 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa2d0052c70 con 0x7fa2e006f780 2024-09-13T18:54:11.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:11.399+0000 7fa2e5c78700 1 -- 172.21.15.7:0/3476987485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa2e015d110 con 0x7fa2e006f780 2024-09-13T18:54:12.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.031+0000 7fa2dd7fa700 1 -- 172.21.15.7:0/3476987485 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa2d00150c0 con 0x7fa2e006f780 2024-09-13T18:54:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.035+0000 7fa2e5c78700 1 -- 172.21.15.7:0/3476987485 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa2cc066730 msgr2=0x7fa2cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.035+0000 7fa2e5c78700 1 --2- 172.21.15.7:0/3476987485 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa2cc066730 0x7fa2cc068bf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009bf0 tx=0x7fa2d4009240 comp rx=0 tx=0).stop 2024-09-13T18:54:12.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.035+0000 7fa2e5c78700 1 -- 172.21.15.7:0/3476987485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e006f780 msgr2=0x7fa2e0156a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.035+0000 7fa2e5c78700 1 --2- 172.21.15.7:0/3476987485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e006f780 0x7fa2e0156a20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00090b0 tx=0x7fa2d0004770 comp rx=0 tx=0).stop 2024-09-13T18:54:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.035+0000 7fa2e5c78700 1 -- 172.21.15.7:0/3476987485 shutdown_connections 2024-09-13T18:54:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.035+0000 7fa2e5c78700 1 --2- 172.21.15.7:0/3476987485 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa2cc066730 0x7fa2cc068bf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.035+0000 7fa2e5c78700 1 --2- 172.21.15.7:0/3476987485 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa2e01401e0 0x7fa2e013d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.035+0000 7fa2e5c78700 1 --2- 172.21.15.7:0/3476987485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2e006f780 0x7fa2e0156a20 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.035+0000 7fa2e5c78700 1 --2- 172.21.15.7:0/3476987485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa2e006f3a0 0x7fa2e01f6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.035+0000 7fa2e5c78700 1 -- 172.21.15.7:0/3476987485 >> 172.21.15.7:0/3476987485 conn(0x7fa2e0071ce0 msgr2=0x7fa2e0141750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.035+0000 7fa2e5c78700 1 -- 172.21.15.7:0/3476987485 shutdown_connections 2024-09-13T18:54:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.035+0000 7fa2e5c78700 1 -- 172.21.15.7:0/3476987485 wait complete. 2024-09-13T18:54:12.062 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T18:54:12.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:12.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.171+0000 7effc5320700 1 Processor -- start 2024-09-13T18:54:12.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.171+0000 7effc5320700 1 -- start start 2024-09-13T18:54:12.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.171+0000 7effc5320700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effc00b3ae0 0x7effc0165590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.171+0000 7effc5320700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7effc0161290 0x7effc01616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.171+0000 7effc5320700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc0071f30 con 0x7effc0160eb0 2024-09-13T18:54:12.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.171+0000 7effc5320700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc01622e0 con 0x7effc00b3ae0 2024-09-13T18:54:12.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.171+0000 7effc5320700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc0162000 con 0x7effc0161290 2024-09-13T18:54:12.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.171+0000 7effbf7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effc00b3ae0 0x7effc0165590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.171+0000 7effbf7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7effc00b3ae0 0x7effc0165590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55382/0 (socket says 172.21.15.7:55382) 2024-09-13T18:54:12.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.171+0000 7effbf7fe700 1 -- 172.21.15.7:0/3766626240 learned_addr learned my addr 172.21.15.7:0/3766626240 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:12.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.171+0000 7effbffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7effc0160eb0 0x7effc00b33d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36668/0 (socket says 172.21.15.7:36668) 2024-09-13T18:54:12.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.171+0000 7effc4b1f700 1 --2- 172.21.15.7:0/3766626240 >> v2:172.21.15.171:3300/0 conn(0x7effc0161290 0x7effc01616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.171+0000 7effbf7fe700 1 -- 172.21.15.7:0/3766626240 >> v2:172.21.15.171:3300/0 conn(0x7effc0161290 msgr2=0x7effc01616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:12.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.171+0000 7effbf7fe700 1 --2- 172.21.15.7:0/3766626240 >> v2:172.21.15.171:3300/0 conn(0x7effc0161290 0x7effc01616c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.171+0000 7effbf7fe700 1 -- 172.21.15.7:0/3766626240 >> v1:172.21.15.7:6789/0 conn(0x7effc0160eb0 legacy=0x7effc00b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:12.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.171+0000 7effbeffd700 1 -- 172.21.15.7:0/3766626240 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 95057458 0 0) 0x7effc0071f30 con 0x7effc0160eb0 2024-09-13T18:54:12.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.171+0000 7effbf7fe700 1 -- 172.21.15.7:0/3766626240 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc0164ab0 con 0x7effc00b3ae0 2024-09-13T18:54:12.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.171+0000 7effbf7fe700 1 --2- 172.21.15.7:0/3766626240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effc00b3ae0 0x7effc0165590 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7effc00cd640 tx=0x7effa8009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72039c384dfdcf07 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:12.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.175+0000 7effbeffd700 1 -- 172.21.15.7:0/3766626240 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8011830 con 0x7effc00b3ae0 2024-09-13T18:54:12.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.175+0000 7effbeffd700 1 -- 172.21.15.7:0/3766626240 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa8011990 con 0x7effc00b3ae0 2024-09-13T18:54:12.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.175+0000 7effbeffd700 1 -- 172.21.15.7:0/3766626240 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8011d20 con 0x7effc00b3ae0 2024-09-13T18:54:12.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.175+0000 7effc5320700 1 -- 172.21.15.7:0/3766626240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effc00b3ae0 msgr2=0x7effc0165590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:12.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.175+0000 7effc5320700 1 --2- 172.21.15.7:0/3766626240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effc00b3ae0 0x7effc0165590 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7effc00cd640 tx=0x7effa8009bf0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.175+0000 7effc5320700 1 -- 172.21.15.7:0/3766626240 shutdown_connections 2024-09-13T18:54:12.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.175+0000 7effc5320700 1 --2- 172.21.15.7:0/3766626240 >> v2:172.21.15.171:3300/0 conn(0x7effc0161290 0x7effc01616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.175+0000 7effc5320700 1 --2- 172.21.15.7:0/3766626240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effc00b3ae0 0x7effc0165590 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.175+0000 7effc5320700 1 -- 172.21.15.7:0/3766626240 >> 172.21.15.7:0/3766626240 conn(0x7effc00b0f00 msgr2=0x7effc00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:12.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.175+0000 7effc5320700 1 -- 172.21.15.7:0/3766626240 shutdown_connections 2024-09-13T18:54:12.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.175+0000 7effc5320700 1 -- 172.21.15.7:0/3766626240 wait complete. 2024-09-13T18:54:12.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effc5320700 1 Processor -- start 2024-09-13T18:54:12.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effc5320700 1 -- start start 2024-09-13T18:54:12.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effc5320700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effc00b3ae0 0x7effc0158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effc5320700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effc0160eb0 0x7effc01f9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effc5320700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effc0161290 0x7effc00ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effc5320700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc01fae90 con 0x7effc0161290 2024-09-13T18:54:12.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effc5320700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc01fa5f0 con 0x7effc0160eb0 2024-09-13T18:54:12.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effc5320700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc01fa8d0 con 0x7effc00b3ae0 2024-09-13T18:54:12.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effc4b1f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effc0161290 0x7effc00ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effc4b1f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effc0161290 0x7effc00ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55386/0 (socket says 172.21.15.7:55386) 2024-09-13T18:54:12.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effc4b1f700 1 -- 172.21.15.7:0/755501948 learned_addr learned my addr 172.21.15.7:0/755501948 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:12.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effbf7fe700 1 --2- 172.21.15.7:0/755501948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effc0160eb0 0x7effc01f9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effbffff700 1 --2- 172.21.15.7:0/755501948 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effc00b3ae0 0x7effc0158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effc4b1f700 1 -- 172.21.15.7:0/755501948 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effc00b3ae0 msgr2=0x7effc0158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:12.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effc4b1f700 1 --2- 172.21.15.7:0/755501948 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effc00b3ae0 0x7effc0158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effc4b1f700 1 -- 172.21.15.7:0/755501948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effc0160eb0 msgr2=0x7effc01f9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:12.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effc4b1f700 1 --2- 172.21.15.7:0/755501948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effc0160eb0 0x7effc01f9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effc4b1f700 1 -- 172.21.15.7:0/755501948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa8009570 con 0x7effc0161290 2024-09-13T18:54:12.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effc4b1f700 1 --2- 172.21.15.7:0/755501948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effc0161290 0x7effc00ccae0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7effb800efc0 tx=0x7effb800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:12.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effbcff9700 1 -- 172.21.15.7:0/755501948 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8009ba0 con 0x7effc0161290 2024-09-13T18:54:12.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effbcff9700 1 -- 172.21.15.7:0/755501948 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb800f040 con 0x7effc0161290 2024-09-13T18:54:12.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.179+0000 7effc5320700 1 -- 172.21.15.7:0/755501948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc01fabb0 con 0x7effc0161290 2024-09-13T18:54:12.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.183+0000 7effbcff9700 1 -- 172.21.15.7:0/755501948 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8014480 con 0x7effc0161290 2024-09-13T18:54:12.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.183+0000 7effc5320700 1 -- 172.21.15.7:0/755501948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc01f7a00 con 0x7effc0161290 2024-09-13T18:54:12.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.183+0000 7effc5320700 1 -- 172.21.15.7:0/755501948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc015e480 con 0x7effc0161290 2024-09-13T18:54:12.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.183+0000 7effbcff9700 1 -- 172.21.15.7:0/755501948 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7effb80145e0 con 0x7effc0161290 2024-09-13T18:54:12.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.183+0000 7effbcff9700 1 --2- 172.21.15.7:0/755501948 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7effa0066730 0x7effa0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.183+0000 7effbcff9700 1 -- 172.21.15.7:0/755501948 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effb8084a20 con 0x7effc0161290 2024-09-13T18:54:12.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.183+0000 7effbffff700 1 --2- 172.21.15.7:0/755501948 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7effa0066730 0x7effa0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.187+0000 7effbffff700 1 --2- 172.21.15.7:0/755501948 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7effa0066730 0x7effa0068bf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7effb0000f60 tx=0x7effb0007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:12.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.195+0000 7effbcff9700 1 -- 172.21.15.7:0/755501948 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7effb8051d90 con 0x7effc0161290 2024-09-13T18:54:12.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.399+0000 7effc5320700 1 -- 172.21.15.7:0/755501948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effc0165590 con 0x7effc0161290 2024-09-13T18:54:12.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.399+0000 7effbcff9700 1 -- 172.21.15.7:0/755501948 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4734 (secure 0 0 0) 0x7effb8018020 con 0x7effc0161290 2024-09-13T18:54:12.409 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:12.409 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":30,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6836/1527871720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1527871720},{"type":"v1","addr":"172.21.15.171:6836","nonce":1527871720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5252,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/4126503064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6835","nonce":4126503064},{"type":"v1","addr":"172.21.15.171:6837","nonce":4126503064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5629,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1132382369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1132382369},{"type":"v1","addr":"172.21.15.7:6833","nonce":1132382369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:12.030791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3168373873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3168373873},{"type":"v1","addr":"172.21.15.7:6835","nonce":3168373873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:12.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.403+0000 7effc5320700 1 -- 172.21.15.7:0/755501948 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7effa0066730 msgr2=0x7effa0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:12.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.403+0000 7effc5320700 1 --2- 172.21.15.7:0/755501948 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7effa0066730 0x7effa0068bf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7effb0000f60 tx=0x7effb0007c00 comp rx=0 tx=0).stop 2024-09-13T18:54:12.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.403+0000 7effc5320700 1 -- 172.21.15.7:0/755501948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effc0161290 msgr2=0x7effc00ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:12.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.403+0000 7effc5320700 1 --2- 172.21.15.7:0/755501948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effc0161290 0x7effc00ccae0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7effb800efc0 tx=0x7effb800c3a0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.403+0000 7effc5320700 1 -- 172.21.15.7:0/755501948 shutdown_connections 2024-09-13T18:54:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.403+0000 7effc5320700 1 --2- 172.21.15.7:0/755501948 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7effa0066730 0x7effa0068bf0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.403+0000 7effc5320700 1 --2- 172.21.15.7:0/755501948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effc0161290 0x7effc00ccae0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.403+0000 7effc5320700 1 --2- 172.21.15.7:0/755501948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effc0160eb0 0x7effc01f9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.403+0000 7effc5320700 1 --2- 172.21.15.7:0/755501948 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effc00b3ae0 0x7effc0158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.403+0000 7effc5320700 1 -- 172.21.15.7:0/755501948 >> 172.21.15.7:0/755501948 conn(0x7effc00b0f00 msgr2=0x7effc0070280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.403+0000 7effc5320700 1 -- 172.21.15.7:0/755501948 shutdown_connections 2024-09-13T18:54:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.403+0000 7effc5320700 1 -- 172.21.15.7:0/755501948 wait complete. 2024-09-13T18:54:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-13T18:54:12.424 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:54:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13bfff700 1 Processor -- start 2024-09-13T18:54:12.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13bfff700 1 -- start start 2024-09-13T18:54:12.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13bfff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fc13c160eb0 0x7fc13c165590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13bfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc13c0b33d0 0x7fc13c0b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc13c161290 0x7fc13c161740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13bfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc13c0b3d30 con 0x7fc13c161290 2024-09-13T18:54:12.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13bfff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc13c071f30 con 0x7fc13c160eb0 2024-09-13T18:54:12.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13bfff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c1622e0 con 0x7fc13c0b33d0 2024-09-13T18:54:12.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc13c161290 0x7fc13c161740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13a7fc700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc13c0b33d0 0x7fc13c0b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13affd700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fc13c160eb0 0x7fc13c165590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc13c161290 0x7fc13c161740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55400/0 (socket says 172.21.15.7:55400) 2024-09-13T18:54:12.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13b7fe700 1 -- 172.21.15.7:0/3182487673 learned_addr learned my addr 172.21.15.7:0/3182487673 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:12.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13b7fe700 1 -- 172.21.15.7:0/3182487673 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc13c0b33d0 msgr2=0x7fc13c0b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:12.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13b7fe700 1 --2- 172.21.15.7:0/3182487673 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc13c0b33d0 0x7fc13c0b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13b7fe700 1 -- 172.21.15.7:0/3182487673 >> v2:172.21.15.171:3300/0 conn(0x7fc13c160eb0 msgr2=0x7fc13c165590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:12.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13b7fe700 1 --2- 172.21.15.7:0/3182487673 >> v2:172.21.15.171:3300/0 conn(0x7fc13c160eb0 0x7fc13c165590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13b7fe700 1 -- 172.21.15.7:0/3182487673 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c162000 con 0x7fc13c161290 2024-09-13T18:54:12.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13affd700 1 --2- 172.21.15.7:0/3182487673 >> v2:172.21.15.171:3300/0 conn(0x7fc13c160eb0 0x7fc13c165590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:12.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13a7fc700 1 --2- 172.21.15.7:0/3182487673 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc13c0b33d0 0x7fc13c0b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:12.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc13b7fe700 1 --2- 172.21.15.7:0/3182487673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc13c161290 0x7fc13c161740 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0b33a0 tx=0x7fc12c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3782340fade5de server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:12.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc139ffb700 1 -- 172.21.15.7:0/3182487673 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c011830 con 0x7fc13c161290 2024-09-13T18:54:12.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc139ffb700 1 -- 172.21.15.7:0/3182487673 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12c011990 con 0x7fc13c161290 2024-09-13T18:54:12.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.535+0000 7fc139ffb700 1 -- 172.21.15.7:0/3182487673 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c011d20 con 0x7fc13c161290 2024-09-13T18:54:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.539+0000 7fc13bfff700 1 -- 172.21.15.7:0/3182487673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc13c161290 msgr2=0x7fc13c161740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.539+0000 7fc13bfff700 1 --2- 172.21.15.7:0/3182487673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc13c161290 0x7fc13c161740 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0b33a0 tx=0x7fc12c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.539+0000 7fc13bfff700 1 -- 172.21.15.7:0/3182487673 shutdown_connections 2024-09-13T18:54:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.539+0000 7fc13bfff700 1 --2- 172.21.15.7:0/3182487673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc13c161290 0x7fc13c161740 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.539+0000 7fc13bfff700 1 --2- 172.21.15.7:0/3182487673 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc13c0b33d0 0x7fc13c0b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.539+0000 7fc13bfff700 1 --2- 172.21.15.7:0/3182487673 >> v2:172.21.15.171:3300/0 conn(0x7fc13c160eb0 0x7fc13c165590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.539+0000 7fc13bfff700 1 -- 172.21.15.7:0/3182487673 >> 172.21.15.7:0/3182487673 conn(0x7fc13c0b0f00 msgr2=0x7fc13c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.539+0000 7fc13bfff700 1 -- 172.21.15.7:0/3182487673 shutdown_connections 2024-09-13T18:54:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.539+0000 7fc13bfff700 1 -- 172.21.15.7:0/3182487673 wait complete. 2024-09-13T18:54:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.539+0000 7fc13bfff700 1 Processor -- start 2024-09-13T18:54:12.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.539+0000 7fc13bfff700 1 -- start start 2024-09-13T18:54:12.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc13c0b33d0 0x7fc13c0ca980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13bfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc13c160eb0 0x7fc13c14e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13bfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc13c161290 0x7fc13c0a9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13bfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c154ba0 con 0x7fc13c0b33d0 2024-09-13T18:54:12.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc13c0b33d0 0x7fc13c0ca980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13bfff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c0adca0 con 0x7fc13c160eb0 2024-09-13T18:54:12.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc13c0b33d0 0x7fc13c0ca980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55406/0 (socket says 172.21.15.7:55406) 2024-09-13T18:54:12.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13affd700 1 -- 172.21.15.7:0/2731750777 learned_addr learned my addr 172.21.15.7:0/2731750777 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:12.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13bfff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c0a90d0 con 0x7fc13c161290 2024-09-13T18:54:12.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13a7fc700 1 --2- 172.21.15.7:0/2731750777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc13c160eb0 0x7fc13c14e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13b7fe700 1 --2- 172.21.15.7:0/2731750777 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc13c161290 0x7fc13c0a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13affd700 1 -- 172.21.15.7:0/2731750777 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc13c161290 msgr2=0x7fc13c0a9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:12.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13affd700 1 --2- 172.21.15.7:0/2731750777 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc13c161290 0x7fc13c0a9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13affd700 1 -- 172.21.15.7:0/2731750777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc13c160eb0 msgr2=0x7fc13c14e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:12.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13affd700 1 --2- 172.21.15.7:0/2731750777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc13c160eb0 0x7fc13c14e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13affd700 1 -- 172.21.15.7:0/2731750777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc134009a70 con 0x7fc13c0b33d0 2024-09-13T18:54:12.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13affd700 1 --2- 172.21.15.7:0/2731750777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc13c0b33d0 0x7fc13c0ca980 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc12400ec10 tx=0x7fc12400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:12.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc133fff700 1 -- 172.21.15.7:0/2731750777 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12400c850 con 0x7fc13c0b33d0 2024-09-13T18:54:12.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc133fff700 1 -- 172.21.15.7:0/2731750777 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12400c9b0 con 0x7fc13c0b33d0 2024-09-13T18:54:12.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13bfff700 1 -- 172.21.15.7:0/2731750777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1240094b0 con 0x7fc13c0b33d0 2024-09-13T18:54:12.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc133fff700 1 -- 172.21.15.7:0/2731750777 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12400cd40 con 0x7fc13c0b33d0 2024-09-13T18:54:12.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.543+0000 7fc13bfff700 1 -- 172.21.15.7:0/2731750777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12c009570 con 0x7fc13c0b33d0 2024-09-13T18:54:12.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.547+0000 7fc13bfff700 1 -- 172.21.15.7:0/2731750777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc13c165590 con 0x7fc13c0b33d0 2024-09-13T18:54:12.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.547+0000 7fc133fff700 1 -- 172.21.15.7:0/2731750777 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc124013470 con 0x7fc13c0b33d0 2024-09-13T18:54:12.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.547+0000 7fc133fff700 1 --2- 172.21.15.7:0/2731750777 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc120066730 0x7fc120068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.547+0000 7fc133fff700 1 -- 172.21.15.7:0/2731750777 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc124084450 con 0x7fc13c0b33d0 2024-09-13T18:54:12.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.547+0000 7fc13a7fc700 1 --2- 172.21.15.7:0/2731750777 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc120066730 0x7fc120068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.547+0000 7fc13a7fc700 1 --2- 172.21.15.7:0/2731750777 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc120066730 0x7fc120068bf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc134009fd0 tx=0x7fc13400d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:12.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.555+0000 7fc133fff700 1 -- 172.21.15.7:0/2731750777 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc124051850 con 0x7fc13c0b33d0 2024-09-13T18:54:12.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.723+0000 7fc13bfff700 1 -- 172.21.15.7:0/2731750777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc13c0b2be0 con 0x7fc13c0b33d0 2024-09-13T18:54:12.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.727+0000 7fc133fff700 1 -- 172.21.15.7:0/2731750777 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16465 (secure 0 0 0) 0x7fc12400cea0 con 0x7fc13c0b33d0 2024-09-13T18:54:12.735 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:12.735 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":52,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:54:10.006861+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:54:07.330160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T18:54:07.981407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:54:12.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.727+0000 7fc13bfff700 1 -- 172.21.15.7:0/2731750777 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc120066730 msgr2=0x7fc120068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:12.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.727+0000 7fc13bfff700 1 --2- 172.21.15.7:0/2731750777 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc120066730 0x7fc120068bf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc134009fd0 tx=0x7fc13400d040 comp rx=0 tx=0).stop 2024-09-13T18:54:12.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.727+0000 7fc13bfff700 1 -- 172.21.15.7:0/2731750777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc13c0b33d0 msgr2=0x7fc13c0ca980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:12.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.727+0000 7fc13bfff700 1 --2- 172.21.15.7:0/2731750777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc13c0b33d0 0x7fc13c0ca980 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc12400ec10 tx=0x7fc12400ef20 comp rx=0 tx=0).stop 2024-09-13T18:54:12.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.727+0000 7fc13bfff700 1 -- 172.21.15.7:0/2731750777 shutdown_connections 2024-09-13T18:54:12.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.727+0000 7fc13bfff700 1 --2- 172.21.15.7:0/2731750777 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc120066730 0x7fc120068bf0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.727+0000 7fc13bfff700 1 --2- 172.21.15.7:0/2731750777 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc13c161290 0x7fc13c0a9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.727+0000 7fc13bfff700 1 --2- 172.21.15.7:0/2731750777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc13c160eb0 0x7fc13c14e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.727+0000 7fc13bfff700 1 --2- 172.21.15.7:0/2731750777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc13c0b33d0 0x7fc13c0ca980 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.727+0000 7fc13bfff700 1 -- 172.21.15.7:0/2731750777 >> 172.21.15.7:0/2731750777 conn(0x7fc13c0b0f00 msgr2=0x7fc13c0aed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:12.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.727+0000 7fc13bfff700 1 -- 172.21.15.7:0/2731750777 shutdown_connections 2024-09-13T18:54:12.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.727+0000 7fc13bfff700 1 -- 172.21.15.7:0/2731750777 wait complete. 2024-09-13T18:54:12.751 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T18:54:12.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:54:12.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f8471047700 1 Processor -- start 2024-09-13T18:54:12.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f8471047700 1 -- start start 2024-09-13T18:54:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f8471047700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f846c15cae0 0x7f846c0cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f8471047700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f846c15cec0 0x7f846c0d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f8471047700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846c0d21e0 con 0x7f846c06ac00 2024-09-13T18:54:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f8471047700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846c0d1750 con 0x7f846c15cae0 2024-09-13T18:54:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f8471047700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846c0d1940 con 0x7f846c15cec0 2024-09-13T18:54:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f846bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f846c15cae0 0x7f846c0cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f846bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f846c15cae0 0x7f846c0cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55418/0 (socket says 172.21.15.7:55418) 2024-09-13T18:54:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f846bfff700 1 -- 172.21.15.7:0/1836911409 learned_addr learned my addr 172.21.15.7:0/1836911409 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f846b7fe700 1 --2- 172.21.15.7:0/1836911409 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f846c15cec0 0x7f846c0d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f846bfff700 1 -- 172.21.15.7:0/1836911409 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f846c15cec0 msgr2=0x7f846c0d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f846bfff700 1 --2- 172.21.15.7:0/1836911409 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f846c15cec0 0x7f846c0d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f846bfff700 1 -- 172.21.15.7:0/1836911409 >> v1:172.21.15.7:6789/0 conn(0x7f846c06ac00 legacy=0x7f846c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f846bfff700 1 -- 172.21.15.7:0/1836911409 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846c0d1c20 con 0x7f846c15cae0 2024-09-13T18:54:12.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f846bfff700 1 --2- 172.21.15.7:0/1836911409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f846c15cae0 0x7f846c0cf030 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8454009910 tx=0x7f8454009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40862b9d4f58438 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f846affd700 1 -- 172.21.15.7:0/1836911409 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845400ba60 con 0x7f846c15cae0 2024-09-13T18:54:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f846affd700 1 -- 172.21.15.7:0/1836911409 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845400bbc0 con 0x7f846c15cae0 2024-09-13T18:54:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f846affd700 1 -- 172.21.15.7:0/1836911409 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8454009e60 con 0x7f846c15cae0 2024-09-13T18:54:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f8471047700 1 -- 172.21.15.7:0/1836911409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f846c15cae0 msgr2=0x7f846c0cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f8471047700 1 --2- 172.21.15.7:0/1836911409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f846c15cae0 0x7f846c0cf030 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8454009910 tx=0x7f8454009c20 comp rx=0 tx=0).stop 2024-09-13T18:54:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f8471047700 1 -- 172.21.15.7:0/1836911409 shutdown_connections 2024-09-13T18:54:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f8471047700 1 --2- 172.21.15.7:0/1836911409 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f846c15cec0 0x7f846c0d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f8471047700 1 --2- 172.21.15.7:0/1836911409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f846c15cae0 0x7f846c0cf030 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f8471047700 1 -- 172.21.15.7:0/1836911409 >> 172.21.15.7:0/1836911409 conn(0x7f846c0d2c80 msgr2=0x7f846c0d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f8471047700 1 -- 172.21.15.7:0/1836911409 shutdown_connections 2024-09-13T18:54:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.855+0000 7f8471047700 1 -- 172.21.15.7:0/1836911409 wait complete. 2024-09-13T18:54:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.859+0000 7f8471047700 1 Processor -- start 2024-09-13T18:54:12.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.859+0000 7f8471047700 1 -- start start 2024-09-13T18:54:12.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.859+0000 7f8471047700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f846c06ac00 0x7f846c0cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.859+0000 7f8471047700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f846c15cae0 0x7f846c0a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.859+0000 7f8471047700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f846c15cec0 0x7f846c14e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.859+0000 7f8471047700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c0ccf90 con 0x7f846c15cae0 2024-09-13T18:54:12.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.859+0000 7f8471047700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c0a9b20 con 0x7f846c06ac00 2024-09-13T18:54:12.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.859+0000 7f8471047700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846c14eed0 con 0x7f846c15cec0 2024-09-13T18:54:12.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.859+0000 7f846b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f846c15cae0 0x7f846c0a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.859+0000 7f846b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f846c15cae0 0x7f846c0a95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55426/0 (socket says 172.21.15.7:55426) 2024-09-13T18:54:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.859+0000 7f846b7fe700 1 -- 172.21.15.7:0/3237429374 learned_addr learned my addr 172.21.15.7:0/3237429374 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.859+0000 7f846bfff700 1 --2- 172.21.15.7:0/3237429374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f846c06ac00 0x7f846c0cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.859+0000 7f8470846700 1 --2- 172.21.15.7:0/3237429374 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f846c15cec0 0x7f846c14e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.859+0000 7f846b7fe700 1 -- 172.21.15.7:0/3237429374 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f846c15cec0 msgr2=0x7f846c14e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.859+0000 7f846b7fe700 1 --2- 172.21.15.7:0/3237429374 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f846c15cec0 0x7f846c14e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.859+0000 7f846b7fe700 1 -- 172.21.15.7:0/3237429374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f846c06ac00 msgr2=0x7f846c0cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.859+0000 7f846b7fe700 1 --2- 172.21.15.7:0/3237429374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f846c06ac00 0x7f846c0cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.859+0000 7f846b7fe700 1 -- 172.21.15.7:0/3237429374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8454009570 con 0x7f846c15cae0 2024-09-13T18:54:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.863+0000 7f846b7fe700 1 --2- 172.21.15.7:0/3237429374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f846c15cae0 0x7f846c0a95e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8464007ef0 tx=0x7f846400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.863+0000 7f8468ff9700 1 -- 172.21.15.7:0/3237429374 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846400ecd0 con 0x7f846c15cae0 2024-09-13T18:54:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.863+0000 7f8468ff9700 1 -- 172.21.15.7:0/3237429374 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846400ee30 con 0x7f846c15cae0 2024-09-13T18:54:12.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.863+0000 7f8471047700 1 -- 172.21.15.7:0/3237429374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846c14e500 con 0x7f846c15cae0 2024-09-13T18:54:12.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.863+0000 7f8468ff9700 1 -- 172.21.15.7:0/3237429374 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464014660 con 0x7f846c15cae0 2024-09-13T18:54:12.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.863+0000 7f8471047700 1 -- 172.21.15.7:0/3237429374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846c1f6620 con 0x7f846c15cae0 2024-09-13T18:54:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.863+0000 7f8468ff9700 1 -- 172.21.15.7:0/3237429374 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f84640147c0 con 0x7f846c15cae0 2024-09-13T18:54:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.863+0000 7f8471047700 1 -- 172.21.15.7:0/3237429374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f846c14e690 con 0x7f846c15cae0 2024-09-13T18:54:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.863+0000 7f8468ff9700 1 --2- 172.21.15.7:0/3237429374 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f844c066730 0x7f844c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.863+0000 7f8468ff9700 1 -- 172.21.15.7:0/3237429374 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8464084780 con 0x7f846c15cae0 2024-09-13T18:54:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.863+0000 7f846bfff700 1 --2- 172.21.15.7:0/3237429374 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f844c066730 0x7f844c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:12.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.867+0000 7f846bfff700 1 --2- 172.21.15.7:0/3237429374 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f844c066730 0x7f844c068bf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8454000c00 tx=0x7f8454008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:12.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:12.875+0000 7f8468ff9700 1 -- 172.21.15.7:0/3237429374 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8464051af0 con 0x7f846c15cae0 2024-09-13T18:54:13.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.043+0000 7f8471047700 1 -- 172.21.15.7:0/3237429374 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f846c0b5d90 con 0x7f844c066730 2024-09-13T18:54:13.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.059+0000 7f8468ff9700 1 -- 172.21.15.7:0/3237429374 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287710 (secure 0 0 0) 0x7f845400b860 con 0x7f844c066730 2024-09-13T18:54:13.073 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.063+0000 7f8471047700 1 -- 172.21.15.7:0/3237429374 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f844c066730 msgr2=0x7f844c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.067+0000 7f8471047700 1 --2- 172.21.15.7:0/3237429374 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f844c066730 0x7f844c068bf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8454000c00 tx=0x7f8454008030 comp rx=0 tx=0).stop 2024-09-13T18:54:13.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.067+0000 7f8471047700 1 -- 172.21.15.7:0/3237429374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f846c15cae0 msgr2=0x7f846c0a95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.067+0000 7f8471047700 1 --2- 172.21.15.7:0/3237429374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f846c15cae0 0x7f846c0a95e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8464007ef0 tx=0x7f846400e3c0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.067+0000 7f8471047700 1 -- 172.21.15.7:0/3237429374 shutdown_connections 2024-09-13T18:54:13.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.067+0000 7f8471047700 1 --2- 172.21.15.7:0/3237429374 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f844c066730 0x7f844c068bf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.067+0000 7f8471047700 1 --2- 172.21.15.7:0/3237429374 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f846c15cec0 0x7f846c14e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.067+0000 7f8471047700 1 --2- 172.21.15.7:0/3237429374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f846c15cae0 0x7f846c0a95e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.067+0000 7f8471047700 1 --2- 172.21.15.7:0/3237429374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f846c06ac00 0x7f846c0cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.067+0000 7f8471047700 1 -- 172.21.15.7:0/3237429374 >> 172.21.15.7:0/3237429374 conn(0x7f846c0d2c80 msgr2=0x7f846c0cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:13.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.067+0000 7f8471047700 1 -- 172.21.15.7:0/3237429374 shutdown_connections 2024-09-13T18:54:13.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.067+0000 7f8471047700 1 -- 172.21.15.7:0/3237429374 wait complete. 2024-09-13T18:54:13.077 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-13T18:54:13.093 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-09-13T18:54:12.249701+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246712,"kb_used_data":5956,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754728008,"statfs":{"total":773094113280,"available":772841480192,"internally_reserved":0,"allocated":6098944,"data_stored":3189312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":246402617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.644508"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512349+0000","last_change":"2024-09-13T18:54:09.034972+0000","last_active":"2024-09-13T18:54:10.512349+0000","last_peered":"2024-09-13T18:54:10.512349+0000","last_clean":"2024-09-13T18:54:10.512349+0000","last_became_active":"2024-09-13T18:54:09.034650+0000","last_became_peered":"2024-09-13T18:54:09.034650+0000","last_unstale":"2024-09-13T18:54:10.512349+0000","last_undegraded":"2024-09-13T18:54:10.512349+0000","last_fullsized":"2024-09-13T18:54:10.512349+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:14.924883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513549+0000","last_change":"2024-09-13T18:54:09.042105+0000","last_active":"2024-09-13T18:54:10.513549+0000","last_peered":"2024-09-13T18:54:10.513549+0000","last_clean":"2024-09-13T18:54:10.513549+0000","last_became_active":"2024-09-13T18:54:09.041856+0000","last_became_peered":"2024-09-13T18:54:09.041856+0000","last_unstale":"2024-09-13T18:54:10.513549+0000","last_undegraded":"2024-09-13T18:54:10.513549+0000","last_fullsized":"2024-09-13T18:54:10.513549+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:20.469932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017744+0000","last_change":"2024-09-13T18:54:09.038863+0000","last_active":"2024-09-13T18:54:10.017744+0000","last_peered":"2024-09-13T18:54:10.017744+0000","last_clean":"2024-09-13T18:54:10.017744+0000","last_became_active":"2024-09-13T18:54:09.038546+0000","last_became_peered":"2024-09-13T18:54:09.038546+0000","last_unstale":"2024-09-13T18:54:10.017744+0000","last_undegraded":"2024-09-13T18:54:10.017744+0000","last_fullsized":"2024-09-13T18:54:10.017744+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:25.641687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510925+0000","last_change":"2024-09-13T18:54:09.042297+0000","last_active":"2024-09-13T18:54:10.510925+0000","last_peered":"2024-09-13T18:54:10.510925+0000","last_clean":"2024-09-13T18:54:10.510925+0000","last_became_active":"2024-09-13T18:54:09.040112+0000","last_became_peered":"2024-09-13T18:54:09.040112+0000","last_unstale":"2024-09-13T18:54:10.510925+0000","last_undegraded":"2024-09-13T18:54:10.510925+0000","last_fullsized":"2024-09-13T18:54:10.510925+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:50:57.913889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513249+0000","last_change":"2024-09-13T18:54:09.037266+0000","last_active":"2024-09-13T18:54:10.513249+0000","last_peered":"2024-09-13T18:54:10.513249+0000","last_clean":"2024-09-13T18:54:10.513249+0000","last_became_active":"2024-09-13T18:54:09.036806+0000","last_became_peered":"2024-09-13T18:54:09.036806+0000","last_unstale":"2024-09-13T18:54:10.513249+0000","last_undegraded":"2024-09-13T18:54:10.513249+0000","last_fullsized":"2024-09-13T18:54:10.513249+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:19.657183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.012934+0000","last_change":"2024-09-13T18:54:09.029648+0000","last_active":"2024-09-13T18:54:10.012934+0000","last_peered":"2024-09-13T18:54:10.012934+0000","last_clean":"2024-09-13T18:54:10.012934+0000","last_became_active":"2024-09-13T18:54:09.029072+0000","last_became_peered":"2024-09-13T18:54:09.029072+0000","last_unstale":"2024-09-13T18:54:10.012934+0000","last_undegraded":"2024-09-13T18:54:10.012934+0000","last_fullsized":"2024-09-13T18:54:10.012934+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:39.825546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512028+0000","last_change":"2024-09-13T18:54:09.030924+0000","last_active":"2024-09-13T18:54:10.512028+0000","last_peered":"2024-09-13T18:54:10.512028+0000","last_clean":"2024-09-13T18:54:10.512028+0000","last_became_active":"2024-09-13T18:54:09.030608+0000","last_became_peered":"2024-09-13T18:54:09.030608+0000","last_unstale":"2024-09-13T18:54:10.512028+0000","last_undegraded":"2024-09-13T18:54:10.512028+0000","last_fullsized":"2024-09-13T18:54:10.512028+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:48.528143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510136+0000","last_change":"2024-09-13T18:54:09.042608+0000","last_active":"2024-09-13T18:54:10.510136+0000","last_peered":"2024-09-13T18:54:10.510136+0000","last_clean":"2024-09-13T18:54:10.510136+0000","last_became_active":"2024-09-13T18:54:09.042325+0000","last_became_peered":"2024-09-13T18:54:09.042325+0000","last_unstale":"2024-09-13T18:54:10.510136+0000","last_undegraded":"2024-09-13T18:54:10.510136+0000","last_fullsized":"2024-09-13T18:54:10.510136+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:56.280764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511799+0000","last_change":"2024-09-13T18:54:09.036445+0000","last_active":"2024-09-13T18:54:10.511799+0000","last_peered":"2024-09-13T18:54:10.511799+0000","last_clean":"2024-09-13T18:54:10.511799+0000","last_became_active":"2024-09-13T18:54:09.035594+0000","last_became_peered":"2024-09-13T18:54:09.035594+0000","last_unstale":"2024-09-13T18:54:10.511799+0000","last_undegraded":"2024-09-13T18:54:10.511799+0000","last_fullsized":"2024-09-13T18:54:10.511799+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:14.620991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511633+0000","last_change":"2024-09-13T18:54:09.039914+0000","last_active":"2024-09-13T18:54:10.511633+0000","last_peered":"2024-09-13T18:54:10.511633+0000","last_clean":"2024-09-13T18:54:10.511633+0000","last_became_active":"2024-09-13T18:54:09.039689+0000","last_became_peered":"2024-09-13T18:54:09.039689+0000","last_unstale":"2024-09-13T18:54:10.511633+0000","last_undegraded":"2024-09-13T18:54:10.511633+0000","last_fullsized":"2024-09-13T18:54:10.511633+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:38:33.066000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511237+0000","last_change":"2024-09-13T18:54:09.036054+0000","last_active":"2024-09-13T18:54:10.511237+0000","last_peered":"2024-09-13T18:54:10.511237+0000","last_clean":"2024-09-13T18:54:10.511237+0000","last_became_active":"2024-09-13T18:54:09.035758+0000","last_became_peered":"2024-09-13T18:54:09.035758+0000","last_unstale":"2024-09-13T18:54:10.511237+0000","last_undegraded":"2024-09-13T18:54:10.511237+0000","last_fullsized":"2024-09-13T18:54:10.511237+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:46:59.751215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513287+0000","last_change":"2024-09-13T18:54:09.042977+0000","last_active":"2024-09-13T18:54:10.513287+0000","last_peered":"2024-09-13T18:54:10.513287+0000","last_clean":"2024-09-13T18:54:10.513287+0000","last_became_active":"2024-09-13T18:54:09.042707+0000","last_became_peered":"2024-09-13T18:54:09.042707+0000","last_unstale":"2024-09-13T18:54:10.513287+0000","last_undegraded":"2024-09-13T18:54:10.513287+0000","last_fullsized":"2024-09-13T18:54:10.513287+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:56.280764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511117+0000","last_change":"2024-09-13T18:54:09.022273+0000","last_active":"2024-09-13T18:54:10.511117+0000","last_peered":"2024-09-13T18:54:10.511117+0000","last_clean":"2024-09-13T18:54:10.511117+0000","last_became_active":"2024-09-13T18:54:09.022056+0000","last_became_peered":"2024-09-13T18:54:09.022056+0000","last_unstale":"2024-09-13T18:54:10.511117+0000","last_undegraded":"2024-09-13T18:54:10.511117+0000","last_fullsized":"2024-09-13T18:54:10.511117+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:55:36.061890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510846+0000","last_change":"2024-09-13T18:54:09.035753+0000","last_active":"2024-09-13T18:54:10.510846+0000","last_peered":"2024-09-13T18:54:10.510846+0000","last_clean":"2024-09-13T18:54:10.510846+0000","last_became_active":"2024-09-13T18:54:09.035138+0000","last_became_peered":"2024-09-13T18:54:09.035138+0000","last_unstale":"2024-09-13T18:54:10.510846+0000","last_undegraded":"2024-09-13T18:54:10.510846+0000","last_fullsized":"2024-09-13T18:54:10.510846+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:50.659206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013062+0000","last_change":"2024-09-13T18:54:09.025685+0000","last_active":"2024-09-13T18:54:10.013062+0000","last_peered":"2024-09-13T18:54:10.013062+0000","last_clean":"2024-09-13T18:54:10.013062+0000","last_became_active":"2024-09-13T18:54:09.025400+0000","last_became_peered":"2024-09-13T18:54:09.025400+0000","last_unstale":"2024-09-13T18:54:10.013062+0000","last_undegraded":"2024-09-13T18:54:10.013062+0000","last_fullsized":"2024-09-13T18:54:10.013062+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:53:54.004733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.015423+0000","last_change":"2024-09-13T18:54:09.041537+0000","last_active":"2024-09-13T18:54:10.015423+0000","last_peered":"2024-09-13T18:54:10.015423+0000","last_clean":"2024-09-13T18:54:10.015423+0000","last_became_active":"2024-09-13T18:54:09.041240+0000","last_became_peered":"2024-09-13T18:54:09.041240+0000","last_unstale":"2024-09-13T18:54:10.015423+0000","last_undegraded":"2024-09-13T18:54:10.015423+0000","last_fullsized":"2024-09-13T18:54:10.015423+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:18.717450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.018196+0000","last_change":"2024-09-13T18:54:09.037869+0000","last_active":"2024-09-13T18:54:10.018196+0000","last_peered":"2024-09-13T18:54:10.018196+0000","last_clean":"2024-09-13T18:54:10.018196+0000","last_became_active":"2024-09-13T18:54:09.037649+0000","last_became_peered":"2024-09-13T18:54:09.037649+0000","last_unstale":"2024-09-13T18:54:10.018196+0000","last_undegraded":"2024-09-13T18:54:10.018196+0000","last_fullsized":"2024-09-13T18:54:10.018196+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:07.303524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017741+0000","last_change":"2024-09-13T18:54:09.041474+0000","last_active":"2024-09-13T18:54:10.017741+0000","last_peered":"2024-09-13T18:54:10.017741+0000","last_clean":"2024-09-13T18:54:10.017741+0000","last_became_active":"2024-09-13T18:54:09.041187+0000","last_became_peered":"2024-09-13T18:54:09.041187+0000","last_unstale":"2024-09-13T18:54:10.017741+0000","last_undegraded":"2024-09-13T18:54:10.017741+0000","last_fullsized":"2024-09-13T18:54:10.017741+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:16:58.895680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017017+0000","last_change":"2024-09-13T18:54:09.032137+0000","last_active":"2024-09-13T18:54:10.017017+0000","last_peered":"2024-09-13T18:54:10.017017+0000","last_clean":"2024-09-13T18:54:10.017017+0000","last_became_active":"2024-09-13T18:54:09.031750+0000","last_became_peered":"2024-09-13T18:54:09.031750+0000","last_unstale":"2024-09-13T18:54:10.017017+0000","last_undegraded":"2024-09-13T18:54:10.017017+0000","last_fullsized":"2024-09-13T18:54:10.017017+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:34:15.440622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511526+0000","last_change":"2024-09-13T18:54:09.035971+0000","last_active":"2024-09-13T18:54:10.511526+0000","last_peered":"2024-09-13T18:54:10.511526+0000","last_clean":"2024-09-13T18:54:10.511526+0000","last_became_active":"2024-09-13T18:54:09.035755+0000","last_became_peered":"2024-09-13T18:54:09.035755+0000","last_unstale":"2024-09-13T18:54:10.511526+0000","last_undegraded":"2024-09-13T18:54:10.511526+0000","last_fullsized":"2024-09-13T18:54:10.511526+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:25.641687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013166+0000","last_change":"2024-09-13T18:54:09.020556+0000","last_active":"2024-09-13T18:54:10.013166+0000","last_peered":"2024-09-13T18:54:10.013166+0000","last_clean":"2024-09-13T18:54:10.013166+0000","last_became_active":"2024-09-13T18:54:09.020163+0000","last_became_peered":"2024-09-13T18:54:09.020163+0000","last_unstale":"2024-09-13T18:54:10.013166+0000","last_undegraded":"2024-09-13T18:54:10.013166+0000","last_fullsized":"2024-09-13T18:54:10.013166+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:12:16.234055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511084+0000","last_change":"2024-09-13T18:54:09.040013+0000","last_active":"2024-09-13T18:54:10.511084+0000","last_peered":"2024-09-13T18:54:10.511084+0000","last_clean":"2024-09-13T18:54:10.511084+0000","last_became_active":"2024-09-13T18:54:09.039781+0000","last_became_peered":"2024-09-13T18:54:09.039781+0000","last_unstale":"2024-09-13T18:54:10.511084+0000","last_undegraded":"2024-09-13T18:54:10.511084+0000","last_fullsized":"2024-09-13T18:54:10.511084+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:20.469932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.015576+0000","last_change":"2024-09-13T18:54:09.034230+0000","last_active":"2024-09-13T18:54:10.015576+0000","last_peered":"2024-09-13T18:54:10.015576+0000","last_clean":"2024-09-13T18:54:10.015576+0000","last_became_active":"2024-09-13T18:54:09.033797+0000","last_became_peered":"2024-09-13T18:54:09.033797+0000","last_unstale":"2024-09-13T18:54:10.015576+0000","last_undegraded":"2024-09-13T18:54:10.015576+0000","last_fullsized":"2024-09-13T18:54:10.015576+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:36:45.544218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.015449+0000","last_change":"2024-09-13T18:54:09.034163+0000","last_active":"2024-09-13T18:54:10.015449+0000","last_peered":"2024-09-13T18:54:10.015449+0000","last_clean":"2024-09-13T18:54:10.015449+0000","last_became_active":"2024-09-13T18:54:09.033892+0000","last_became_peered":"2024-09-13T18:54:09.033892+0000","last_unstale":"2024-09-13T18:54:10.015449+0000","last_undegraded":"2024-09-13T18:54:10.015449+0000","last_fullsized":"2024-09-13T18:54:10.015449+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:11:22.207917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017445+0000","last_change":"2024-09-13T18:54:09.037297+0000","last_active":"2024-09-13T18:54:10.017445+0000","last_peered":"2024-09-13T18:54:10.017445+0000","last_clean":"2024-09-13T18:54:10.017445+0000","last_became_active":"2024-09-13T18:54:09.033915+0000","last_became_peered":"2024-09-13T18:54:09.033915+0000","last_unstale":"2024-09-13T18:54:10.017445+0000","last_undegraded":"2024-09-13T18:54:10.017445+0000","last_fullsized":"2024-09-13T18:54:10.017445+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:32:03.439867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512043+0000","last_change":"2024-09-13T18:54:09.036493+0000","last_active":"2024-09-13T18:54:10.512043+0000","last_peered":"2024-09-13T18:54:10.512043+0000","last_clean":"2024-09-13T18:54:10.512043+0000","last_became_active":"2024-09-13T18:54:09.035712+0000","last_became_peered":"2024-09-13T18:54:09.035712+0000","last_unstale":"2024-09-13T18:54:10.512043+0000","last_undegraded":"2024-09-13T18:54:10.512043+0000","last_fullsized":"2024-09-13T18:54:10.512043+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:18:23.698995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017770+0000","last_change":"2024-09-13T18:54:09.038968+0000","last_active":"2024-09-13T18:54:10.017770+0000","last_peered":"2024-09-13T18:54:10.017770+0000","last_clean":"2024-09-13T18:54:10.017770+0000","last_became_active":"2024-09-13T18:54:09.038709+0000","last_became_peered":"2024-09-13T18:54:09.038709+0000","last_unstale":"2024-09-13T18:54:10.017770+0000","last_undegraded":"2024-09-13T18:54:10.017770+0000","last_fullsized":"2024-09-13T18:54:10.017770+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:26:20.982783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513352+0000","last_change":"2024-09-13T18:54:09.042642+0000","last_active":"2024-09-13T18:54:10.513352+0000","last_peered":"2024-09-13T18:54:10.513352+0000","last_clean":"2024-09-13T18:54:10.513352+0000","last_became_active":"2024-09-13T18:54:09.042336+0000","last_became_peered":"2024-09-13T18:54:09.042336+0000","last_unstale":"2024-09-13T18:54:10.513352+0000","last_undegraded":"2024-09-13T18:54:10.513352+0000","last_fullsized":"2024-09-13T18:54:10.513352+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:17.507996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.016108+0000","last_change":"2024-09-13T18:54:09.034012+0000","last_active":"2024-09-13T18:54:10.016108+0000","last_peered":"2024-09-13T18:54:10.016108+0000","last_clean":"2024-09-13T18:54:10.016108+0000","last_became_active":"2024-09-13T18:54:09.033699+0000","last_became_peered":"2024-09-13T18:54:09.033699+0000","last_unstale":"2024-09-13T18:54:10.016108+0000","last_undegraded":"2024-09-13T18:54:10.016108+0000","last_fullsized":"2024-09-13T18:54:10.016108+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:33:12.532879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.016309+0000","last_change":"2024-09-13T18:54:09.031651+0000","last_active":"2024-09-13T18:54:10.016309+0000","last_peered":"2024-09-13T18:54:10.016309+0000","last_clean":"2024-09-13T18:54:10.016309+0000","last_became_active":"2024-09-13T18:54:09.031329+0000","last_became_peered":"2024-09-13T18:54:09.031329+0000","last_unstale":"2024-09-13T18:54:10.016309+0000","last_undegraded":"2024-09-13T18:54:10.016309+0000","last_fullsized":"2024-09-13T18:54:10.016309+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:46:42.484744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013003+0000","last_change":"2024-09-13T18:54:09.032424+0000","last_active":"2024-09-13T18:54:10.013003+0000","last_peered":"2024-09-13T18:54:10.013003+0000","last_clean":"2024-09-13T18:54:10.013003+0000","last_became_active":"2024-09-13T18:54:09.032219+0000","last_became_peered":"2024-09-13T18:54:09.032219+0000","last_unstale":"2024-09-13T18:54:10.013003+0000","last_undegraded":"2024-09-13T18:54:10.013003+0000","last_fullsized":"2024-09-13T18:54:10.013003+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:24:36.307754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510608+0000","last_change":"2024-09-13T18:54:09.031951+0000","last_active":"2024-09-13T18:54:10.510608+0000","last_peered":"2024-09-13T18:54:10.510608+0000","last_clean":"2024-09-13T18:54:10.510608+0000","last_became_active":"2024-09-13T18:54:09.031229+0000","last_became_peered":"2024-09-13T18:54:09.031229+0000","last_unstale":"2024-09-13T18:54:10.510608+0000","last_undegraded":"2024-09-13T18:54:10.510608+0000","last_fullsized":"2024-09-13T18:54:10.510608+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:21:06.044703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513255+0000","last_change":"2024-09-13T18:54:09.029453+0000","last_active":"2024-09-13T18:54:10.513255+0000","last_peered":"2024-09-13T18:54:10.513255+0000","last_clean":"2024-09-13T18:54:10.513255+0000","last_became_active":"2024-09-13T18:54:09.029203+0000","last_became_peered":"2024-09-13T18:54:09.029203+0000","last_unstale":"2024-09-13T18:54:10.513255+0000","last_undegraded":"2024-09-13T18:54:10.513255+0000","last_fullsized":"2024-09-13T18:54:10.513255+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:34:15.440622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"51'4","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013838+0000","last_change":"2024-09-13T18:54:08.641174+0000","last_active":"2024-09-13T18:54:10.013838+0000","last_peered":"2024-09-13T18:54:10.013838+0000","last_clean":"2024-09-13T18:54:10.013838+0000","last_became_active":"2024-09-13T18:54:08.640895+0000","last_became_peered":"2024-09-13T18:54:08.640895+0000","last_unstale":"2024-09-13T18:54:10.013838+0000","last_undegraded":"2024-09-13T18:54:10.013838+0000","last_fullsized":"2024-09-13T18:54:10.013838+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:46:58.739769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017448+0000","last_change":"2024-09-13T18:54:09.030151+0000","last_active":"2024-09-13T18:54:10.017448+0000","last_peered":"2024-09-13T18:54:10.017448+0000","last_clean":"2024-09-13T18:54:10.017448+0000","last_became_active":"2024-09-13T18:54:09.029846+0000","last_became_peered":"2024-09-13T18:54:09.029846+0000","last_unstale":"2024-09-13T18:54:10.017448+0000","last_undegraded":"2024-09-13T18:54:10.017448+0000","last_fullsized":"2024-09-13T18:54:10.017448+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:38.864994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"51'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.018235+0000","last_change":"2024-09-13T18:54:08.642329+0000","last_active":"2024-09-13T18:54:10.018235+0000","last_peered":"2024-09-13T18:54:10.018235+0000","last_clean":"2024-09-13T18:54:10.018235+0000","last_became_active":"2024-09-13T18:54:08.642016+0000","last_became_peered":"2024-09-13T18:54:08.642016+0000","last_unstale":"2024-09-13T18:54:10.018235+0000","last_undegraded":"2024-09-13T18:54:10.018235+0000","last_fullsized":"2024-09-13T18:54:10.018235+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:11.214204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513459+0000","last_change":"2024-09-13T18:54:09.034541+0000","last_active":"2024-09-13T18:54:10.513459+0000","last_peered":"2024-09-13T18:54:10.513459+0000","last_clean":"2024-09-13T18:54:10.513459+0000","last_became_active":"2024-09-13T18:54:09.033634+0000","last_became_peered":"2024-09-13T18:54:09.033634+0000","last_unstale":"2024-09-13T18:54:10.513459+0000","last_undegraded":"2024-09-13T18:54:10.513459+0000","last_fullsized":"2024-09-13T18:54:10.513459+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:50:57.913889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"51'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513511+0000","last_change":"2024-09-13T18:54:08.641770+0000","last_active":"2024-09-13T18:54:10.513511+0000","last_peered":"2024-09-13T18:54:10.513511+0000","last_clean":"2024-09-13T18:54:10.513511+0000","last_became_active":"2024-09-13T18:54:08.641394+0000","last_became_peered":"2024-09-13T18:54:08.641394+0000","last_unstale":"2024-09-13T18:54:10.513511+0000","last_undegraded":"2024-09-13T18:54:10.513511+0000","last_fullsized":"2024-09-13T18:54:10.513511+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:43.402295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513480+0000","last_change":"2024-09-13T18:54:09.038714+0000","last_active":"2024-09-13T18:54:10.513480+0000","last_peered":"2024-09-13T18:54:10.513480+0000","last_clean":"2024-09-13T18:54:10.513480+0000","last_became_active":"2024-09-13T18:54:09.038479+0000","last_became_peered":"2024-09-13T18:54:09.038479+0000","last_unstale":"2024-09-13T18:54:10.513480+0000","last_undegraded":"2024-09-13T18:54:10.513480+0000","last_fullsized":"2024-09-13T18:54:10.513480+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:59.323928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"51'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512032+0000","last_change":"2024-09-13T18:54:08.644572+0000","last_active":"2024-09-13T18:54:10.512032+0000","last_peered":"2024-09-13T18:54:10.512032+0000","last_clean":"2024-09-13T18:54:10.512032+0000","last_became_active":"2024-09-13T18:54:08.644201+0000","last_became_peered":"2024-09-13T18:54:08.644201+0000","last_unstale":"2024-09-13T18:54:10.512032+0000","last_undegraded":"2024-09-13T18:54:10.512032+0000","last_fullsized":"2024-09-13T18:54:10.512032+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:12.843255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.016863+0000","last_change":"2024-09-13T18:54:09.028991+0000","last_active":"2024-09-13T18:54:10.016863+0000","last_peered":"2024-09-13T18:54:10.016863+0000","last_clean":"2024-09-13T18:54:10.016863+0000","last_became_active":"2024-09-13T18:54:09.028639+0000","last_became_peered":"2024-09-13T18:54:09.028639+0000","last_unstale":"2024-09-13T18:54:10.016863+0000","last_undegraded":"2024-09-13T18:54:10.016863+0000","last_fullsized":"2024-09-13T18:54:10.016863+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:20.973711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"51'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013394+0000","last_change":"2024-09-13T18:54:08.640456+0000","last_active":"2024-09-13T18:54:10.013394+0000","last_peered":"2024-09-13T18:54:10.013394+0000","last_clean":"2024-09-13T18:54:10.013394+0000","last_became_active":"2024-09-13T18:54:08.640140+0000","last_became_peered":"2024-09-13T18:54:08.640140+0000","last_unstale":"2024-09-13T18:54:10.013394+0000","last_undegraded":"2024-09-13T18:54:10.013394+0000","last_fullsized":"2024-09-13T18:54:10.013394+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:27:57.708775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511491+0000","last_change":"2024-09-13T18:54:09.042642+0000","last_active":"2024-09-13T18:54:10.511491+0000","last_peered":"2024-09-13T18:54:10.511491+0000","last_clean":"2024-09-13T18:54:10.511491+0000","last_became_active":"2024-09-13T18:54:09.042346+0000","last_became_peered":"2024-09-13T18:54:09.042346+0000","last_unstale":"2024-09-13T18:54:10.511491+0000","last_undegraded":"2024-09-13T18:54:10.511491+0000","last_fullsized":"2024-09-13T18:54:10.511491+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:03.777920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"51'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017125+0000","last_change":"2024-09-13T18:54:08.637002+0000","last_active":"2024-09-13T18:54:10.017125+0000","last_peered":"2024-09-13T18:54:10.017125+0000","last_clean":"2024-09-13T18:54:10.017125+0000","last_became_active":"2024-09-13T18:54:08.636742+0000","last_became_peered":"2024-09-13T18:54:08.636742+0000","last_unstale":"2024-09-13T18:54:10.017125+0000","last_undegraded":"2024-09-13T18:54:10.017125+0000","last_fullsized":"2024-09-13T18:54:10.017125+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:16.055887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510378+0000","last_change":"2024-09-13T18:54:09.042484+0000","last_active":"2024-09-13T18:54:10.510378+0000","last_peered":"2024-09-13T18:54:10.510378+0000","last_clean":"2024-09-13T18:54:10.510378+0000","last_became_active":"2024-09-13T18:54:09.040094+0000","last_became_peered":"2024-09-13T18:54:09.040094+0000","last_unstale":"2024-09-13T18:54:10.510378+0000","last_undegraded":"2024-09-13T18:54:10.510378+0000","last_fullsized":"2024-09-13T18:54:10.510378+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:17.507996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512796+0000","last_change":"2024-09-13T18:54:08.638208+0000","last_active":"2024-09-13T18:54:10.512796+0000","last_peered":"2024-09-13T18:54:10.512796+0000","last_clean":"2024-09-13T18:54:10.512796+0000","last_became_active":"2024-09-13T18:54:08.636291+0000","last_became_peered":"2024-09-13T18:54:08.636291+0000","last_unstale":"2024-09-13T18:54:10.512796+0000","last_undegraded":"2024-09-13T18:54:10.512796+0000","last_fullsized":"2024-09-13T18:54:10.512796+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:46:58.739769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.015315+0000","last_change":"2024-09-13T18:54:09.033779+0000","last_active":"2024-09-13T18:54:10.015315+0000","last_peered":"2024-09-13T18:54:10.015315+0000","last_clean":"2024-09-13T18:54:10.015315+0000","last_became_active":"2024-09-13T18:54:09.033418+0000","last_became_peered":"2024-09-13T18:54:09.033418+0000","last_unstale":"2024-09-13T18:54:10.015315+0000","last_undegraded":"2024-09-13T18:54:10.015315+0000","last_fullsized":"2024-09-13T18:54:10.015315+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:35:24.626210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510353+0000","last_change":"2024-09-13T18:54:08.636504+0000","last_active":"2024-09-13T18:54:10.510353+0000","last_peered":"2024-09-13T18:54:10.510353+0000","last_clean":"2024-09-13T18:54:10.510353+0000","last_became_active":"2024-09-13T18:54:08.635726+0000","last_became_peered":"2024-09-13T18:54:08.635726+0000","last_unstale":"2024-09-13T18:54:10.510353+0000","last_undegraded":"2024-09-13T18:54:10.510353+0000","last_fullsized":"2024-09-13T18:54:10.510353+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:29.334736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017950+0000","last_change":"2024-09-13T18:54:09.031190+0000","last_active":"2024-09-13T18:54:10.017950+0000","last_peered":"2024-09-13T18:54:10.017950+0000","last_clean":"2024-09-13T18:54:10.017950+0000","last_became_active":"2024-09-13T18:54:09.030885+0000","last_became_peered":"2024-09-13T18:54:09.030885+0000","last_unstale":"2024-09-13T18:54:10.017950+0000","last_undegraded":"2024-09-13T18:54:10.017950+0000","last_fullsized":"2024-09-13T18:54:10.017950+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:44.533584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.018378+0000","last_change":"2024-09-13T18:54:08.633875+0000","last_active":"2024-09-13T18:54:10.018378+0000","last_peered":"2024-09-13T18:54:10.018378+0000","last_clean":"2024-09-13T18:54:10.018378+0000","last_became_active":"2024-09-13T18:54:08.633529+0000","last_became_peered":"2024-09-13T18:54:08.633529+0000","last_unstale":"2024-09-13T18:54:10.018378+0000","last_undegraded":"2024-09-13T18:54:10.018378+0000","last_fullsized":"2024-09-13T18:54:10.018378+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:05.014350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512195+0000","last_change":"2024-09-13T18:54:09.036011+0000","last_active":"2024-09-13T18:54:10.512195+0000","last_peered":"2024-09-13T18:54:10.512195+0000","last_clean":"2024-09-13T18:54:10.512195+0000","last_became_active":"2024-09-13T18:54:09.035739+0000","last_became_peered":"2024-09-13T18:54:09.035739+0000","last_unstale":"2024-09-13T18:54:10.512195+0000","last_undegraded":"2024-09-13T18:54:10.512195+0000","last_fullsized":"2024-09-13T18:54:10.512195+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:20.469932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511856+0000","last_change":"2024-09-13T18:54:08.641983+0000","last_active":"2024-09-13T18:54:10.511856+0000","last_peered":"2024-09-13T18:54:10.511856+0000","last_clean":"2024-09-13T18:54:10.511856+0000","last_became_active":"2024-09-13T18:54:08.641640+0000","last_became_peered":"2024-09-13T18:54:08.641640+0000","last_unstale":"2024-09-13T18:54:10.511856+0000","last_undegraded":"2024-09-13T18:54:10.511856+0000","last_fullsized":"2024-09-13T18:54:10.511856+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:43.402295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013434+0000","last_change":"2024-09-13T18:54:09.018322+0000","last_active":"2024-09-13T18:54:10.013434+0000","last_peered":"2024-09-13T18:54:10.013434+0000","last_clean":"2024-09-13T18:54:10.013434+0000","last_became_active":"2024-09-13T18:54:09.018017+0000","last_became_peered":"2024-09-13T18:54:09.018017+0000","last_unstale":"2024-09-13T18:54:10.013434+0000","last_undegraded":"2024-09-13T18:54:10.013434+0000","last_fullsized":"2024-09-13T18:54:10.013434+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:43:16.925363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"51'3","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513037+0000","last_change":"2024-09-13T18:54:08.643783+0000","last_active":"2024-09-13T18:54:10.513037+0000","last_peered":"2024-09-13T18:54:10.513037+0000","last_clean":"2024-09-13T18:54:10.513037+0000","last_became_active":"2024-09-13T18:54:08.643433+0000","last_became_peered":"2024-09-13T18:54:08.643433+0000","last_unstale":"2024-09-13T18:54:10.513037+0000","last_undegraded":"2024-09-13T18:54:10.513037+0000","last_fullsized":"2024-09-13T18:54:10.513037+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:47.725231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512611+0000","last_change":"2024-09-13T18:54:09.030914+0000","last_active":"2024-09-13T18:54:10.512611+0000","last_peered":"2024-09-13T18:54:10.512611+0000","last_clean":"2024-09-13T18:54:10.512611+0000","last_became_active":"2024-09-13T18:54:09.030639+0000","last_became_peered":"2024-09-13T18:54:09.030639+0000","last_unstale":"2024-09-13T18:54:10.512611+0000","last_undegraded":"2024-09-13T18:54:10.512611+0000","last_fullsized":"2024-09-13T18:54:10.512611+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:38:33.066000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"51'6","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511636+0000","last_change":"2024-09-13T18:54:08.632413+0000","last_active":"2024-09-13T18:54:10.511636+0000","last_peered":"2024-09-13T18:54:10.511636+0000","last_clean":"2024-09-13T18:54:10.511636+0000","last_became_active":"2024-09-13T18:54:08.632154+0000","last_became_peered":"2024-09-13T18:54:08.632154+0000","last_unstale":"2024-09-13T18:54:10.511636+0000","last_undegraded":"2024-09-13T18:54:10.511636+0000","last_fullsized":"2024-09-13T18:54:10.511636+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:05.014350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.016198+0000","last_change":"2024-09-13T18:54:09.031848+0000","last_active":"2024-09-13T18:54:10.016198+0000","last_peered":"2024-09-13T18:54:10.016198+0000","last_clean":"2024-09-13T18:54:10.016198+0000","last_became_active":"2024-09-13T18:54:09.031589+0000","last_became_peered":"2024-09-13T18:54:09.031589+0000","last_unstale":"2024-09-13T18:54:10.016198+0000","last_undegraded":"2024-09-13T18:54:10.016198+0000","last_fullsized":"2024-09-13T18:54:10.016198+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:50.905909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"51'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013554+0000","last_change":"2024-09-13T18:54:08.634808+0000","last_active":"2024-09-13T18:54:10.013554+0000","last_peered":"2024-09-13T18:54:10.013554+0000","last_clean":"2024-09-13T18:54:10.013554+0000","last_became_active":"2024-09-13T18:54:08.634448+0000","last_became_peered":"2024-09-13T18:54:08.634448+0000","last_unstale":"2024-09-13T18:54:10.013554+0000","last_undegraded":"2024-09-13T18:54:10.013554+0000","last_fullsized":"2024-09-13T18:54:10.013554+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:49.647759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510987+0000","last_change":"2024-09-13T18:54:09.038992+0000","last_active":"2024-09-13T18:54:10.510987+0000","last_peered":"2024-09-13T18:54:10.510987+0000","last_clean":"2024-09-13T18:54:10.510987+0000","last_became_active":"2024-09-13T18:54:09.036624+0000","last_became_peered":"2024-09-13T18:54:09.036624+0000","last_unstale":"2024-09-13T18:54:10.510987+0000","last_undegraded":"2024-09-13T18:54:10.510987+0000","last_fullsized":"2024-09-13T18:54:10.510987+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:48.528143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512212+0000","last_change":"2024-09-13T18:54:08.642806+0000","last_active":"2024-09-13T18:54:10.512212+0000","last_peered":"2024-09-13T18:54:10.512212+0000","last_clean":"2024-09-13T18:54:10.512212+0000","last_became_active":"2024-09-13T18:54:08.641339+0000","last_became_peered":"2024-09-13T18:54:08.641339+0000","last_unstale":"2024-09-13T18:54:10.512212+0000","last_undegraded":"2024-09-13T18:54:10.512212+0000","last_fullsized":"2024-09-13T18:54:10.512212+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:49.894463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.012954+0000","last_change":"2024-09-13T18:54:09.018040+0000","last_active":"2024-09-13T18:54:10.012954+0000","last_peered":"2024-09-13T18:54:10.012954+0000","last_clean":"2024-09-13T18:54:10.012954+0000","last_became_active":"2024-09-13T18:54:09.017747+0000","last_became_peered":"2024-09-13T18:54:09.017747+0000","last_unstale":"2024-09-13T18:54:10.012954+0000","last_undegraded":"2024-09-13T18:54:10.012954+0000","last_fullsized":"2024-09-13T18:54:10.012954+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:26.348636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013060+0000","last_change":"2024-09-13T18:54:08.633363+0000","last_active":"2024-09-13T18:54:10.013060+0000","last_peered":"2024-09-13T18:54:10.013060+0000","last_clean":"2024-09-13T18:54:10.013060+0000","last_became_active":"2024-09-13T18:54:08.633093+0000","last_became_peered":"2024-09-13T18:54:08.633093+0000","last_unstale":"2024-09-13T18:54:10.013060+0000","last_undegraded":"2024-09-13T18:54:10.013060+0000","last_fullsized":"2024-09-13T18:54:10.013060+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:20:21.015963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512167+0000","last_change":"2024-09-13T18:54:09.036997+0000","last_active":"2024-09-13T18:54:10.512167+0000","last_peered":"2024-09-13T18:54:10.512167+0000","last_clean":"2024-09-13T18:54:10.512167+0000","last_became_active":"2024-09-13T18:54:09.036719+0000","last_became_peered":"2024-09-13T18:54:09.036719+0000","last_unstale":"2024-09-13T18:54:10.512167+0000","last_undegraded":"2024-09-13T18:54:10.512167+0000","last_fullsized":"2024-09-13T18:54:10.512167+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:27:58.720221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"51'8","reported_seq":19,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511694+0000","last_change":"2024-09-13T18:54:08.641810+0000","last_active":"2024-09-13T18:54:10.511694+0000","last_peered":"2024-09-13T18:54:10.511694+0000","last_clean":"2024-09-13T18:54:10.511694+0000","last_became_active":"2024-09-13T18:54:08.641451+0000","last_became_peered":"2024-09-13T18:54:08.641451+0000","last_unstale":"2024-09-13T18:54:10.511694+0000","last_undegraded":"2024-09-13T18:54:10.511694+0000","last_fullsized":"2024-09-13T18:54:10.511694+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:16.425833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512299+0000","last_change":"2024-09-13T18:54:09.035525+0000","last_active":"2024-09-13T18:54:10.512299+0000","last_peered":"2024-09-13T18:54:10.512299+0000","last_clean":"2024-09-13T18:54:10.512299+0000","last_became_active":"2024-09-13T18:54:09.035270+0000","last_became_peered":"2024-09-13T18:54:09.035270+0000","last_unstale":"2024-09-13T18:54:10.512299+0000","last_undegraded":"2024-09-13T18:54:10.512299+0000","last_fullsized":"2024-09-13T18:54:10.512299+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:52.700687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"51'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512754+0000","last_change":"2024-09-13T18:54:08.632985+0000","last_active":"2024-09-13T18:54:10.512754+0000","last_peered":"2024-09-13T18:54:10.512754+0000","last_clean":"2024-09-13T18:54:10.512754+0000","last_became_active":"2024-09-13T18:54:08.632713+0000","last_became_peered":"2024-09-13T18:54:08.632713+0000","last_unstale":"2024-09-13T18:54:10.512754+0000","last_undegraded":"2024-09-13T18:54:10.512754+0000","last_fullsized":"2024-09-13T18:54:10.512754+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:34:32.208511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511928+0000","last_change":"2024-09-13T18:54:09.033978+0000","last_active":"2024-09-13T18:54:10.511928+0000","last_peered":"2024-09-13T18:54:10.511928+0000","last_clean":"2024-09-13T18:54:10.511928+0000","last_became_active":"2024-09-13T18:54:09.033706+0000","last_became_peered":"2024-09-13T18:54:09.033706+0000","last_unstale":"2024-09-13T18:54:10.511928+0000","last_undegraded":"2024-09-13T18:54:10.511928+0000","last_fullsized":"2024-09-13T18:54:10.511928+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:50.172893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"51'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.018040+0000","last_change":"2024-09-13T18:54:08.633785+0000","last_active":"2024-09-13T18:54:10.018040+0000","last_peered":"2024-09-13T18:54:10.018040+0000","last_clean":"2024-09-13T18:54:10.018040+0000","last_became_active":"2024-09-13T18:54:08.633479+0000","last_became_peered":"2024-09-13T18:54:08.633479+0000","last_unstale":"2024-09-13T18:54:10.018040+0000","last_undegraded":"2024-09-13T18:54:10.018040+0000","last_fullsized":"2024-09-13T18:54:10.018040+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:30.803360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.016406+0000","last_change":"2024-09-13T18:54:09.028606+0000","last_active":"2024-09-13T18:54:10.016406+0000","last_peered":"2024-09-13T18:54:10.016406+0000","last_clean":"2024-09-13T18:54:10.016406+0000","last_became_active":"2024-09-13T18:54:09.028354+0000","last_became_peered":"2024-09-13T18:54:09.028354+0000","last_unstale":"2024-09-13T18:54:10.016406+0000","last_undegraded":"2024-09-13T18:54:10.016406+0000","last_fullsized":"2024-09-13T18:54:10.016406+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:42:28.771252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"51'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513017+0000","last_change":"2024-09-13T18:54:08.638649+0000","last_active":"2024-09-13T18:54:10.513017+0000","last_peered":"2024-09-13T18:54:10.513017+0000","last_clean":"2024-09-13T18:54:10.513017+0000","last_became_active":"2024-09-13T18:54:08.638402+0000","last_became_peered":"2024-09-13T18:54:08.638402+0000","last_unstale":"2024-09-13T18:54:10.513017+0000","last_undegraded":"2024-09-13T18:54:10.513017+0000","last_fullsized":"2024-09-13T18:54:10.513017+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:36.231227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.016876+0000","last_change":"2024-09-13T18:54:09.034238+0000","last_active":"2024-09-13T18:54:10.016876+0000","last_peered":"2024-09-13T18:54:10.016876+0000","last_clean":"2024-09-13T18:54:10.016876+0000","last_became_active":"2024-09-13T18:54:09.033625+0000","last_became_peered":"2024-09-13T18:54:09.033625+0000","last_unstale":"2024-09-13T18:54:10.016876+0000","last_undegraded":"2024-09-13T18:54:10.016876+0000","last_fullsized":"2024-09-13T18:54:10.016876+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:59.323928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013499+0000","last_change":"2024-09-13T18:54:08.638582+0000","last_active":"2024-09-13T18:54:10.013499+0000","last_peered":"2024-09-13T18:54:10.013499+0000","last_clean":"2024-09-13T18:54:10.013499+0000","last_became_active":"2024-09-13T18:54:08.638301+0000","last_became_peered":"2024-09-13T18:54:08.638301+0000","last_unstale":"2024-09-13T18:54:10.013499+0000","last_undegraded":"2024-09-13T18:54:10.013499+0000","last_fullsized":"2024-09-13T18:54:10.013499+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:36.231227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.012943+0000","last_change":"2024-09-13T18:54:09.018585+0000","last_active":"2024-09-13T18:54:10.012943+0000","last_peered":"2024-09-13T18:54:10.012943+0000","last_clean":"2024-09-13T18:54:10.012943+0000","last_became_active":"2024-09-13T18:54:09.018330+0000","last_became_peered":"2024-09-13T18:54:09.018330+0000","last_unstale":"2024-09-13T18:54:10.012943+0000","last_undegraded":"2024-09-13T18:54:10.012943+0000","last_fullsized":"2024-09-13T18:54:10.012943+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:03:03.283695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"51'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013757+0000","last_change":"2024-09-13T18:54:08.635266+0000","last_active":"2024-09-13T18:54:10.013757+0000","last_peered":"2024-09-13T18:54:10.013757+0000","last_clean":"2024-09-13T18:54:10.013757+0000","last_became_active":"2024-09-13T18:54:08.634849+0000","last_became_peered":"2024-09-13T18:54:08.634849+0000","last_unstale":"2024-09-13T18:54:10.013757+0000","last_undegraded":"2024-09-13T18:54:10.013757+0000","last_fullsized":"2024-09-13T18:54:10.013757+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:49.161446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513218+0000","last_change":"2024-09-13T18:54:09.033331+0000","last_active":"2024-09-13T18:54:10.513218+0000","last_peered":"2024-09-13T18:54:10.513218+0000","last_clean":"2024-09-13T18:54:10.513218+0000","last_became_active":"2024-09-13T18:54:09.033036+0000","last_became_peered":"2024-09-13T18:54:09.033036+0000","last_unstale":"2024-09-13T18:54:10.513218+0000","last_undegraded":"2024-09-13T18:54:10.513218+0000","last_fullsized":"2024-09-13T18:54:10.513218+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:18.717450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512975+0000","last_change":"2024-09-13T18:54:08.642111+0000","last_active":"2024-09-13T18:54:10.512975+0000","last_peered":"2024-09-13T18:54:10.512975+0000","last_clean":"2024-09-13T18:54:10.512975+0000","last_became_active":"2024-09-13T18:54:08.641851+0000","last_became_peered":"2024-09-13T18:54:08.641851+0000","last_unstale":"2024-09-13T18:54:10.512975+0000","last_undegraded":"2024-09-13T18:54:10.512975+0000","last_fullsized":"2024-09-13T18:54:10.512975+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:06:58.916296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510841+0000","last_change":"2024-09-13T18:54:09.030933+0000","last_active":"2024-09-13T18:54:10.510841+0000","last_peered":"2024-09-13T18:54:10.510841+0000","last_clean":"2024-09-13T18:54:10.510841+0000","last_became_active":"2024-09-13T18:54:09.030668+0000","last_became_peered":"2024-09-13T18:54:09.030668+0000","last_unstale":"2024-09-13T18:54:10.510841+0000","last_undegraded":"2024-09-13T18:54:10.510841+0000","last_fullsized":"2024-09-13T18:54:10.510841+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:58.743317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510734+0000","last_change":"2024-09-13T18:54:08.641045+0000","last_active":"2024-09-13T18:54:10.510734+0000","last_peered":"2024-09-13T18:54:10.510734+0000","last_clean":"2024-09-13T18:54:10.510734+0000","last_became_active":"2024-09-13T18:54:08.640761+0000","last_became_peered":"2024-09-13T18:54:08.640761+0000","last_unstale":"2024-09-13T18:54:10.510734+0000","last_undegraded":"2024-09-13T18:54:10.510734+0000","last_fullsized":"2024-09-13T18:54:10.510734+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:52.033459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510385+0000","last_change":"2024-09-13T18:54:09.036139+0000","last_active":"2024-09-13T18:54:10.510385+0000","last_peered":"2024-09-13T18:54:10.510385+0000","last_clean":"2024-09-13T18:54:10.510385+0000","last_became_active":"2024-09-13T18:54:09.035869+0000","last_became_peered":"2024-09-13T18:54:09.035869+0000","last_unstale":"2024-09-13T18:54:10.510385+0000","last_undegraded":"2024-09-13T18:54:10.510385+0000","last_fullsized":"2024-09-13T18:54:10.510385+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:32:03.439867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511704+0000","last_change":"2024-09-13T18:54:08.642596+0000","last_active":"2024-09-13T18:54:10.511704+0000","last_peered":"2024-09-13T18:54:10.511704+0000","last_clean":"2024-09-13T18:54:10.511704+0000","last_became_active":"2024-09-13T18:54:08.641301+0000","last_became_peered":"2024-09-13T18:54:08.641301+0000","last_unstale":"2024-09-13T18:54:10.511704+0000","last_undegraded":"2024-09-13T18:54:10.511704+0000","last_fullsized":"2024-09-13T18:54:10.511704+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:37.853548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017550+0000","last_change":"2024-09-13T18:54:09.038480+0000","last_active":"2024-09-13T18:54:10.017550+0000","last_peered":"2024-09-13T18:54:10.017550+0000","last_clean":"2024-09-13T18:54:10.017550+0000","last_became_active":"2024-09-13T18:54:09.038209+0000","last_became_peered":"2024-09-13T18:54:09.038209+0000","last_unstale":"2024-09-13T18:54:10.017550+0000","last_undegraded":"2024-09-13T18:54:10.017550+0000","last_fullsized":"2024-09-13T18:54:10.017550+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:59.919515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013558+0000","last_change":"2024-09-13T18:54:08.632672+0000","last_active":"2024-09-13T18:54:10.013558+0000","last_peered":"2024-09-13T18:54:10.013558+0000","last_clean":"2024-09-13T18:54:10.013558+0000","last_became_active":"2024-09-13T18:54:08.632438+0000","last_became_peered":"2024-09-13T18:54:08.632438+0000","last_unstale":"2024-09-13T18:54:10.013558+0000","last_undegraded":"2024-09-13T18:54:10.013558+0000","last_fullsized":"2024-09-13T18:54:10.013558+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:13.609544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511169+0000","last_change":"2024-09-13T18:54:09.034818+0000","last_active":"2024-09-13T18:54:10.511169+0000","last_peered":"2024-09-13T18:54:10.511169+0000","last_clean":"2024-09-13T18:54:10.511169+0000","last_became_active":"2024-09-13T18:54:09.034547+0000","last_became_peered":"2024-09-13T18:54:09.034547+0000","last_unstale":"2024-09-13T18:54:10.511169+0000","last_undegraded":"2024-09-13T18:54:10.511169+0000","last_fullsized":"2024-09-13T18:54:10.511169+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:27:58.720221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513232+0000","last_change":"2024-09-13T18:54:08.644292+0000","last_active":"2024-09-13T18:54:10.513232+0000","last_peered":"2024-09-13T18:54:10.513232+0000","last_clean":"2024-09-13T18:54:10.513232+0000","last_became_active":"2024-09-13T18:54:08.644026+0000","last_became_peered":"2024-09-13T18:54:08.644026+0000","last_unstale":"2024-09-13T18:54:10.513232+0000","last_undegraded":"2024-09-13T18:54:10.513232+0000","last_fullsized":"2024-09-13T18:54:10.513232+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:58.908069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512830+0000","last_change":"2024-09-13T18:54:09.038442+0000","last_active":"2024-09-13T18:54:10.512830+0000","last_peered":"2024-09-13T18:54:10.512830+0000","last_clean":"2024-09-13T18:54:10.512830+0000","last_became_active":"2024-09-13T18:54:09.038196+0000","last_became_peered":"2024-09-13T18:54:09.038196+0000","last_unstale":"2024-09-13T18:54:10.512830+0000","last_undegraded":"2024-09-13T18:54:10.512830+0000","last_fullsized":"2024-09-13T18:54:10.512830+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:03.777920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513109+0000","last_change":"2024-09-13T18:54:08.636715+0000","last_active":"2024-09-13T18:54:10.513109+0000","last_peered":"2024-09-13T18:54:10.513109+0000","last_clean":"2024-09-13T18:54:10.513109+0000","last_became_active":"2024-09-13T18:54:08.636436+0000","last_became_peered":"2024-09-13T18:54:08.636436+0000","last_unstale":"2024-09-13T18:54:10.513109+0000","last_undegraded":"2024-09-13T18:54:10.513109+0000","last_fullsized":"2024-09-13T18:54:10.513109+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:52.033459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013171+0000","last_change":"2024-09-13T18:54:09.029802+0000","last_active":"2024-09-13T18:54:10.013171+0000","last_peered":"2024-09-13T18:54:10.013171+0000","last_clean":"2024-09-13T18:54:10.013171+0000","last_became_active":"2024-09-13T18:54:09.029417+0000","last_became_peered":"2024-09-13T18:54:09.029417+0000","last_unstale":"2024-09-13T18:54:10.013171+0000","last_undegraded":"2024-09-13T18:54:10.013171+0000","last_fullsized":"2024-09-13T18:54:10.013171+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:31:00.210734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512511+0000","last_change":"2024-09-13T18:54:08.643816+0000","last_active":"2024-09-13T18:54:10.512511+0000","last_peered":"2024-09-13T18:54:10.512511+0000","last_clean":"2024-09-13T18:54:10.512511+0000","last_became_active":"2024-09-13T18:54:08.643521+0000","last_became_peered":"2024-09-13T18:54:08.643521+0000","last_unstale":"2024-09-13T18:54:10.512511+0000","last_undegraded":"2024-09-13T18:54:10.512511+0000","last_fullsized":"2024-09-13T18:54:10.512511+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:32:02.428421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013343+0000","last_change":"2024-09-13T18:54:09.022602+0000","last_active":"2024-09-13T18:54:10.013343+0000","last_peered":"2024-09-13T18:54:10.013343+0000","last_clean":"2024-09-13T18:54:10.013343+0000","last_became_active":"2024-09-13T18:54:09.022295+0000","last_became_peered":"2024-09-13T18:54:09.022295+0000","last_unstale":"2024-09-13T18:54:10.013343+0000","last_undegraded":"2024-09-13T18:54:10.013343+0000","last_fullsized":"2024-09-13T18:54:10.013343+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:46.663258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017335+0000","last_change":"2024-09-13T18:54:08.630805+0000","last_active":"2024-09-13T18:54:10.017335+0000","last_peered":"2024-09-13T18:54:10.017335+0000","last_clean":"2024-09-13T18:54:10.017335+0000","last_became_active":"2024-09-13T18:54:08.630559+0000","last_became_peered":"2024-09-13T18:54:08.630559+0000","last_unstale":"2024-09-13T18:54:10.017335+0000","last_undegraded":"2024-09-13T18:54:10.017335+0000","last_fullsized":"2024-09-13T18:54:10.017335+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:42:10.101010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512829+0000","last_change":"2024-09-13T18:54:09.032997+0000","last_active":"2024-09-13T18:54:10.512829+0000","last_peered":"2024-09-13T18:54:10.512829+0000","last_clean":"2024-09-13T18:54:10.512829+0000","last_became_active":"2024-09-13T18:54:09.032715+0000","last_became_peered":"2024-09-13T18:54:09.032715+0000","last_unstale":"2024-09-13T18:54:10.512829+0000","last_undegraded":"2024-09-13T18:54:10.512829+0000","last_fullsized":"2024-09-13T18:54:10.512829+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:38:33.066000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512962+0000","last_change":"2024-09-13T18:54:08.638488+0000","last_active":"2024-09-13T18:54:10.512962+0000","last_peered":"2024-09-13T18:54:10.512962+0000","last_clean":"2024-09-13T18:54:10.512962+0000","last_became_active":"2024-09-13T18:54:08.638221+0000","last_became_peered":"2024-09-13T18:54:08.638221+0000","last_unstale":"2024-09-13T18:54:10.512962+0000","last_undegraded":"2024-09-13T18:54:10.512962+0000","last_fullsized":"2024-09-13T18:54:10.512962+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:29.334736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.016358+0000","last_change":"2024-09-13T18:54:09.034617+0000","last_active":"2024-09-13T18:54:10.016358+0000","last_peered":"2024-09-13T18:54:10.016358+0000","last_clean":"2024-09-13T18:54:10.016358+0000","last_became_active":"2024-09-13T18:54:09.034331+0000","last_became_peered":"2024-09-13T18:54:09.034331+0000","last_unstale":"2024-09-13T18:54:10.016358+0000","last_undegraded":"2024-09-13T18:54:10.016358+0000","last_fullsized":"2024-09-13T18:54:10.016358+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:52.700687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510617+0000","last_change":"2024-09-13T18:54:08.633373+0000","last_active":"2024-09-13T18:54:10.510617+0000","last_peered":"2024-09-13T18:54:10.510617+0000","last_clean":"2024-09-13T18:54:10.510617+0000","last_became_active":"2024-09-13T18:54:08.633068+0000","last_became_peered":"2024-09-13T18:54:08.633068+0000","last_unstale":"2024-09-13T18:54:10.510617+0000","last_undegraded":"2024-09-13T18:54:10.510617+0000","last_fullsized":"2024-09-13T18:54:10.510617+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:08:50.580470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511761+0000","last_change":"2024-09-13T18:54:09.039179+0000","last_active":"2024-09-13T18:54:10.511761+0000","last_peered":"2024-09-13T18:54:10.511761+0000","last_clean":"2024-09-13T18:54:10.511761+0000","last_became_active":"2024-09-13T18:54:09.036603+0000","last_became_peered":"2024-09-13T18:54:09.036603+0000","last_unstale":"2024-09-13T18:54:10.511761+0000","last_undegraded":"2024-09-13T18:54:10.511761+0000","last_fullsized":"2024-09-13T18:54:10.511761+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:14.924883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"51'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513272+0000","last_change":"2024-09-13T18:54:08.641300+0000","last_active":"2024-09-13T18:54:10.513272+0000","last_peered":"2024-09-13T18:54:10.513272+0000","last_clean":"2024-09-13T18:54:10.513272+0000","last_became_active":"2024-09-13T18:54:08.639819+0000","last_became_peered":"2024-09-13T18:54:08.639819+0000","last_unstale":"2024-09-13T18:54:10.513272+0000","last_undegraded":"2024-09-13T18:54:10.513272+0000","last_fullsized":"2024-09-13T18:54:10.513272+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:49.647759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":184,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-13T18:54:09.023777+0000","last_change":"2024-09-13T18:52:15.080080+0000","last_active":"2024-09-13T18:54:09.023777+0000","last_peered":"2024-09-13T18:54:09.023777+0000","last_clean":"2024-09-13T18:54:09.023777+0000","last_became_active":"2024-09-13T18:52:15.078100+0000","last_became_peered":"2024-09-13T18:52:15.078100+0000","last_unstale":"2024-09-13T18:54:09.023777+0000","last_undegraded":"2024-09-13T18:54:09.023777+0000","last_fullsized":"2024-09-13T18:54:09.023777+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_clean_scrub_stamp":"2024-09-13T18:52:13.196473+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:29:59.452757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28192,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343648,"statfs":{"total":96636764160,"available":96607895552,"internally_reserved":0,"allocated":1196032,"data_stored":837002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.64500000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.56100000000000005}]},{"osd":4,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.73999999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.46700000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.76500000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":618496,"data_stored":253516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.97199999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":1.204}]},{"osd":2,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.91700000000000004}]},{"osd":3,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":1.1000000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.171},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":1.0780000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.004},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":1.2589999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.0859999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.23}]}]},{"osd":5,"up_from":14,"seq":60129542172,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":614400,"data_stored":252914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.66600000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.70399999999999996}]},{"osd":2,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.996}]},{"osd":3,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.81699999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.91400000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.081}]},{"osd":7,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":3.4369999999999998,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":1.107}]}]},{"osd":4,"up_from":14,"seq":60129542171,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":622592,"data_stored":254408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.747}]},{"osd":1,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.71499999999999997}]},{"osd":2,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.878}]},{"osd":3,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.84799999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.022}]},{"osd":7,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.94499999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":622592,"data_stored":253482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.871},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.61799999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.70799999999999996}]},{"osd":2,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.081}]},{"osd":4,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.27},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.3140000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.1559999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.1200000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.996},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.391}]}]},{"osd":2,"up_from":14,"seq":60129542171,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35816,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94336024,"statfs":{"total":96636764160,"available":96600088576,"internally_reserved":0,"allocated":614400,"data_stored":248130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.93200000000000005}]},{"osd":3,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.82399999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.85199999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.1379999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":1.238}]},{"osd":7,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.2090000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":614400,"data_stored":252914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.786,"5min":2.786,"15min":2.786},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.78200000000000003}]},{"osd":2,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.113,"5min":3.113,"15min":3.113},"last":0.62},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.93700000000000006}]},{"osd":3,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.65300000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.052}]},{"osd":5,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.4780000000000002,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":0.90900000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.878}]},{"osd":7,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.4409999999999998,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.0029999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1196032,"data_stored":836946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.55100000000000005}]},{"osd":2,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.006}]},{"osd":3,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":1.036},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.73499999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.091}]},{"osd":5,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":1.131},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.98999999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.5329999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:54:13.099 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:54:13.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.203+0000 7f60b9eca700 1 Processor -- start 2024-09-13T18:54:13.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.203+0000 7f60b9eca700 1 -- start start 2024-09-13T18:54:13.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b9eca700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f60b406acc0 0x7f60b415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b9eca700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60b413dcb0 con 0x7f60b406bbf0 2024-09-13T18:54:13.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b9eca700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60b413ffd0 con 0x7f60b40b3ae0 2024-09-13T18:54:13.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b9eca700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60b406b110 con 0x7f60b406acc0 2024-09-13T18:54:13.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b96c9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f60b406acc0 0x7f60b415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b96c9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f60b406acc0 0x7f60b415ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55440/0 (socket says 172.21.15.7:55440) 2024-09-13T18:54:13.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b96c9700 1 -- 172.21.15.7:0/4058573858 learned_addr learned my addr 172.21.15.7:0/4058573858 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:13.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b8ec8700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f60b406bbf0 0x7f60b40b33d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36690/0 (socket says 172.21.15.7:36690) 2024-09-13T18:54:13.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b96c9700 1 -- 172.21.15.7:0/4058573858 >> v1:172.21.15.171:6789/0 conn(0x7f60b40b3ae0 legacy=0x7f60b406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b96c9700 1 -- 172.21.15.7:0/4058573858 >> v1:172.21.15.7:6789/0 conn(0x7f60b406bbf0 legacy=0x7f60b40b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b37fe700 1 -- 172.21.15.7:0/4058573858 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822512540 0 0) 0x7f60b413dcb0 con 0x7f60b406bbf0 2024-09-13T18:54:13.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b96c9700 1 -- 172.21.15.7:0/4058573858 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b4140220 con 0x7f60b406acc0 2024-09-13T18:54:13.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b96c9700 1 --2- 172.21.15.7:0/4058573858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b406acc0 0x7f60b415ae50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009e50 tx=0x7f60ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=227b007968ccb4a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:13.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b37fe700 1 -- 172.21.15.7:0/4058573858 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac00e070 con 0x7f60b406acc0 2024-09-13T18:54:13.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b37fe700 1 -- 172.21.15.7:0/4058573858 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ac00b9c0 con 0x7f60b406acc0 2024-09-13T18:54:13.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b37fe700 1 -- 172.21.15.7:0/4058573858 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac00bd70 con 0x7f60b406acc0 2024-09-13T18:54:13.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b9eca700 1 -- 172.21.15.7:0/4058573858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b406acc0 msgr2=0x7f60b415ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b9eca700 1 --2- 172.21.15.7:0/4058573858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b406acc0 0x7f60b415ae50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009e50 tx=0x7f60ac00b330 comp rx=0 tx=0).stop 2024-09-13T18:54:13.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b9eca700 1 -- 172.21.15.7:0/4058573858 shutdown_connections 2024-09-13T18:54:13.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b9eca700 1 --2- 172.21.15.7:0/4058573858 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b406acc0 0x7f60b415ae50 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b9eca700 1 -- 172.21.15.7:0/4058573858 >> 172.21.15.7:0/4058573858 conn(0x7f60b40b0f00 msgr2=0x7f60b40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:13.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b9eca700 1 -- 172.21.15.7:0/4058573858 shutdown_connections 2024-09-13T18:54:13.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.207+0000 7f60b9eca700 1 -- 172.21.15.7:0/4058573858 wait complete. 2024-09-13T18:54:13.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b9eca700 1 Processor -- start 2024-09-13T18:54:13.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b9eca700 1 -- start start 2024-09-13T18:54:13.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b9eca700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f60b406acc0 0x7f60b40addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b9eca700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60b406bbf0 0x7f60b413b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b9eca700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b40b3ae0 0x7f60b40ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b9eca700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b40cc190 con 0x7f60b40b3ae0 2024-09-13T18:54:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b9eca700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b40cc470 con 0x7f60b406bbf0 2024-09-13T18:54:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b9eca700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b40ad2d0 con 0x7f60b406acc0 2024-09-13T18:54:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b96c9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b40b3ae0 0x7f60b40ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b96c9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b40b3ae0 0x7f60b40ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55454/0 (socket says 172.21.15.7:55454) 2024-09-13T18:54:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b96c9700 1 -- 172.21.15.7:0/2919746834 learned_addr learned my addr 172.21.15.7:0/2919746834 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b8ec8700 1 --2- 172.21.15.7:0/2919746834 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f60b406acc0 0x7f60b40addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b3fff700 1 --2- 172.21.15.7:0/2919746834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60b406bbf0 0x7f60b413b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b96c9700 1 -- 172.21.15.7:0/2919746834 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f60b406acc0 msgr2=0x7f60b40addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b96c9700 1 --2- 172.21.15.7:0/2919746834 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f60b406acc0 0x7f60b40addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b96c9700 1 -- 172.21.15.7:0/2919746834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60b406bbf0 msgr2=0x7f60b413b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b96c9700 1 --2- 172.21.15.7:0/2919746834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60b406bbf0 0x7f60b413b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b96c9700 1 -- 172.21.15.7:0/2919746834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac009b30 con 0x7f60b40b3ae0 2024-09-13T18:54:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b96c9700 1 --2- 172.21.15.7:0/2919746834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b40b3ae0 0x7f60b40ccc60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f60b406b390 tx=0x7f60ac004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b17fa700 1 -- 172.21.15.7:0/2919746834 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac004900 con 0x7f60b40b3ae0 2024-09-13T18:54:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.211+0000 7f60b17fa700 1 -- 172.21.15.7:0/2919746834 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ac0099b0 con 0x7f60b40b3ae0 2024-09-13T18:54:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.215+0000 7f60b9eca700 1 -- 172.21.15.7:0/2919746834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b40ad5b0 con 0x7f60b40b3ae0 2024-09-13T18:54:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.215+0000 7f60b17fa700 1 -- 172.21.15.7:0/2919746834 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac01b670 con 0x7f60b40b3ae0 2024-09-13T18:54:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.215+0000 7f60b9eca700 1 -- 172.21.15.7:0/2919746834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b413b8f0 con 0x7f60b40b3ae0 2024-09-13T18:54:13.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.215+0000 7f60b9eca700 1 -- 172.21.15.7:0/2919746834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b40ada70 con 0x7f60b40b3ae0 2024-09-13T18:54:13.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.215+0000 7f60b17fa700 1 -- 172.21.15.7:0/2919746834 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f60ac01b7d0 con 0x7f60b40b3ae0 2024-09-13T18:54:13.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.215+0000 7f60b17fa700 1 --2- 172.21.15.7:0/2919746834 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6094066730 0x7f6094068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.215+0000 7f60b17fa700 1 -- 172.21.15.7:0/2919746834 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60ac085a70 con 0x7f60b40b3ae0 2024-09-13T18:54:13.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.215+0000 7f60b8ec8700 1 --2- 172.21.15.7:0/2919746834 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6094066730 0x7f6094068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.219+0000 7f60b8ec8700 1 --2- 172.21.15.7:0/2919746834 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6094066730 0x7f6094068bf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f609c007b30 tx=0x7f609c007a80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:13.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.223+0000 7f60b17fa700 1 -- 172.21.15.7:0/2919746834 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f60ac052e70 con 0x7f60b40b3ae0 2024-09-13T18:54:13.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.387+0000 7f60b9eca700 1 -- 172.21.15.7:0/2919746834 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60b4140ac0 con 0x7f6094066730 2024-09-13T18:54:13.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.403+0000 7f60b17fa700 1 -- 172.21.15.7:0/2919746834 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287710 (secure 0 0 0) 0x7f609c001350 con 0x7f6094066730 2024-09-13T18:54:13.414 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.411+0000 7f60b9eca700 1 -- 172.21.15.7:0/2919746834 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6094066730 msgr2=0x7f6094068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.411+0000 7f60b9eca700 1 --2- 172.21.15.7:0/2919746834 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6094066730 0x7f6094068bf0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f609c007b30 tx=0x7f609c007a80 comp rx=0 tx=0).stop 2024-09-13T18:54:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.411+0000 7f60b9eca700 1 -- 172.21.15.7:0/2919746834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b40b3ae0 msgr2=0x7f60b40ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.411+0000 7f60b9eca700 1 --2- 172.21.15.7:0/2919746834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b40b3ae0 0x7f60b40ccc60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f60b406b390 tx=0x7f60ac004af0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.411+0000 7f60b9eca700 1 -- 172.21.15.7:0/2919746834 shutdown_connections 2024-09-13T18:54:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.411+0000 7f60b9eca700 1 --2- 172.21.15.7:0/2919746834 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6094066730 0x7f6094068bf0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.411+0000 7f60b9eca700 1 --2- 172.21.15.7:0/2919746834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60b40b3ae0 0x7f60b40ccc60 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.411+0000 7f60b9eca700 1 --2- 172.21.15.7:0/2919746834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f60b406bbf0 0x7f60b413b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.411+0000 7f60b9eca700 1 --2- 172.21.15.7:0/2919746834 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f60b406acc0 0x7f60b40addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.411+0000 7f60b9eca700 1 -- 172.21.15.7:0/2919746834 >> 172.21.15.7:0/2919746834 conn(0x7f60b40b0f00 msgr2=0x7f60b40aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:13.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.411+0000 7f60b9eca700 1 -- 172.21.15.7:0/2919746834 shutdown_connections 2024-09-13T18:54:13.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.411+0000 7f60b9eca700 1 -- 172.21.15.7:0/2919746834 wait complete. 2024-09-13T18:54:13.421 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-13T18:54:13.435 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-09-13T18:54:12.249701+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":141,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246712,"kb_used_data":5956,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754728008,"statfs":{"total":773094113280,"available":772841480192,"internally_reserved":0,"allocated":6098944,"data_stored":3189312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":246402617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.644508"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512349+0000","last_change":"2024-09-13T18:54:09.034972+0000","last_active":"2024-09-13T18:54:10.512349+0000","last_peered":"2024-09-13T18:54:10.512349+0000","last_clean":"2024-09-13T18:54:10.512349+0000","last_became_active":"2024-09-13T18:54:09.034650+0000","last_became_peered":"2024-09-13T18:54:09.034650+0000","last_unstale":"2024-09-13T18:54:10.512349+0000","last_undegraded":"2024-09-13T18:54:10.512349+0000","last_fullsized":"2024-09-13T18:54:10.512349+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:14.924883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513549+0000","last_change":"2024-09-13T18:54:09.042105+0000","last_active":"2024-09-13T18:54:10.513549+0000","last_peered":"2024-09-13T18:54:10.513549+0000","last_clean":"2024-09-13T18:54:10.513549+0000","last_became_active":"2024-09-13T18:54:09.041856+0000","last_became_peered":"2024-09-13T18:54:09.041856+0000","last_unstale":"2024-09-13T18:54:10.513549+0000","last_undegraded":"2024-09-13T18:54:10.513549+0000","last_fullsized":"2024-09-13T18:54:10.513549+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:20.469932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017744+0000","last_change":"2024-09-13T18:54:09.038863+0000","last_active":"2024-09-13T18:54:10.017744+0000","last_peered":"2024-09-13T18:54:10.017744+0000","last_clean":"2024-09-13T18:54:10.017744+0000","last_became_active":"2024-09-13T18:54:09.038546+0000","last_became_peered":"2024-09-13T18:54:09.038546+0000","last_unstale":"2024-09-13T18:54:10.017744+0000","last_undegraded":"2024-09-13T18:54:10.017744+0000","last_fullsized":"2024-09-13T18:54:10.017744+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:25.641687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510925+0000","last_change":"2024-09-13T18:54:09.042297+0000","last_active":"2024-09-13T18:54:10.510925+0000","last_peered":"2024-09-13T18:54:10.510925+0000","last_clean":"2024-09-13T18:54:10.510925+0000","last_became_active":"2024-09-13T18:54:09.040112+0000","last_became_peered":"2024-09-13T18:54:09.040112+0000","last_unstale":"2024-09-13T18:54:10.510925+0000","last_undegraded":"2024-09-13T18:54:10.510925+0000","last_fullsized":"2024-09-13T18:54:10.510925+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:50:57.913889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513249+0000","last_change":"2024-09-13T18:54:09.037266+0000","last_active":"2024-09-13T18:54:10.513249+0000","last_peered":"2024-09-13T18:54:10.513249+0000","last_clean":"2024-09-13T18:54:10.513249+0000","last_became_active":"2024-09-13T18:54:09.036806+0000","last_became_peered":"2024-09-13T18:54:09.036806+0000","last_unstale":"2024-09-13T18:54:10.513249+0000","last_undegraded":"2024-09-13T18:54:10.513249+0000","last_fullsized":"2024-09-13T18:54:10.513249+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:37:19.657183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.012934+0000","last_change":"2024-09-13T18:54:09.029648+0000","last_active":"2024-09-13T18:54:10.012934+0000","last_peered":"2024-09-13T18:54:10.012934+0000","last_clean":"2024-09-13T18:54:10.012934+0000","last_became_active":"2024-09-13T18:54:09.029072+0000","last_became_peered":"2024-09-13T18:54:09.029072+0000","last_unstale":"2024-09-13T18:54:10.012934+0000","last_undegraded":"2024-09-13T18:54:10.012934+0000","last_fullsized":"2024-09-13T18:54:10.012934+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:25:39.825546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512028+0000","last_change":"2024-09-13T18:54:09.030924+0000","last_active":"2024-09-13T18:54:10.512028+0000","last_peered":"2024-09-13T18:54:10.512028+0000","last_clean":"2024-09-13T18:54:10.512028+0000","last_became_active":"2024-09-13T18:54:09.030608+0000","last_became_peered":"2024-09-13T18:54:09.030608+0000","last_unstale":"2024-09-13T18:54:10.512028+0000","last_undegraded":"2024-09-13T18:54:10.512028+0000","last_fullsized":"2024-09-13T18:54:10.512028+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:48.528143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510136+0000","last_change":"2024-09-13T18:54:09.042608+0000","last_active":"2024-09-13T18:54:10.510136+0000","last_peered":"2024-09-13T18:54:10.510136+0000","last_clean":"2024-09-13T18:54:10.510136+0000","last_became_active":"2024-09-13T18:54:09.042325+0000","last_became_peered":"2024-09-13T18:54:09.042325+0000","last_unstale":"2024-09-13T18:54:10.510136+0000","last_undegraded":"2024-09-13T18:54:10.510136+0000","last_fullsized":"2024-09-13T18:54:10.510136+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:56.280764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511799+0000","last_change":"2024-09-13T18:54:09.036445+0000","last_active":"2024-09-13T18:54:10.511799+0000","last_peered":"2024-09-13T18:54:10.511799+0000","last_clean":"2024-09-13T18:54:10.511799+0000","last_became_active":"2024-09-13T18:54:09.035594+0000","last_became_peered":"2024-09-13T18:54:09.035594+0000","last_unstale":"2024-09-13T18:54:10.511799+0000","last_undegraded":"2024-09-13T18:54:10.511799+0000","last_fullsized":"2024-09-13T18:54:10.511799+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:14.620991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511633+0000","last_change":"2024-09-13T18:54:09.039914+0000","last_active":"2024-09-13T18:54:10.511633+0000","last_peered":"2024-09-13T18:54:10.511633+0000","last_clean":"2024-09-13T18:54:10.511633+0000","last_became_active":"2024-09-13T18:54:09.039689+0000","last_became_peered":"2024-09-13T18:54:09.039689+0000","last_unstale":"2024-09-13T18:54:10.511633+0000","last_undegraded":"2024-09-13T18:54:10.511633+0000","last_fullsized":"2024-09-13T18:54:10.511633+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:38:33.066000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511237+0000","last_change":"2024-09-13T18:54:09.036054+0000","last_active":"2024-09-13T18:54:10.511237+0000","last_peered":"2024-09-13T18:54:10.511237+0000","last_clean":"2024-09-13T18:54:10.511237+0000","last_became_active":"2024-09-13T18:54:09.035758+0000","last_became_peered":"2024-09-13T18:54:09.035758+0000","last_unstale":"2024-09-13T18:54:10.511237+0000","last_undegraded":"2024-09-13T18:54:10.511237+0000","last_fullsized":"2024-09-13T18:54:10.511237+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:46:59.751215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513287+0000","last_change":"2024-09-13T18:54:09.042977+0000","last_active":"2024-09-13T18:54:10.513287+0000","last_peered":"2024-09-13T18:54:10.513287+0000","last_clean":"2024-09-13T18:54:10.513287+0000","last_became_active":"2024-09-13T18:54:09.042707+0000","last_became_peered":"2024-09-13T18:54:09.042707+0000","last_unstale":"2024-09-13T18:54:10.513287+0000","last_undegraded":"2024-09-13T18:54:10.513287+0000","last_fullsized":"2024-09-13T18:54:10.513287+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:56.280764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511117+0000","last_change":"2024-09-13T18:54:09.022273+0000","last_active":"2024-09-13T18:54:10.511117+0000","last_peered":"2024-09-13T18:54:10.511117+0000","last_clean":"2024-09-13T18:54:10.511117+0000","last_became_active":"2024-09-13T18:54:09.022056+0000","last_became_peered":"2024-09-13T18:54:09.022056+0000","last_unstale":"2024-09-13T18:54:10.511117+0000","last_undegraded":"2024-09-13T18:54:10.511117+0000","last_fullsized":"2024-09-13T18:54:10.511117+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:55:36.061890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510846+0000","last_change":"2024-09-13T18:54:09.035753+0000","last_active":"2024-09-13T18:54:10.510846+0000","last_peered":"2024-09-13T18:54:10.510846+0000","last_clean":"2024-09-13T18:54:10.510846+0000","last_became_active":"2024-09-13T18:54:09.035138+0000","last_became_peered":"2024-09-13T18:54:09.035138+0000","last_unstale":"2024-09-13T18:54:10.510846+0000","last_undegraded":"2024-09-13T18:54:10.510846+0000","last_fullsized":"2024-09-13T18:54:10.510846+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:50.659206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013062+0000","last_change":"2024-09-13T18:54:09.025685+0000","last_active":"2024-09-13T18:54:10.013062+0000","last_peered":"2024-09-13T18:54:10.013062+0000","last_clean":"2024-09-13T18:54:10.013062+0000","last_became_active":"2024-09-13T18:54:09.025400+0000","last_became_peered":"2024-09-13T18:54:09.025400+0000","last_unstale":"2024-09-13T18:54:10.013062+0000","last_undegraded":"2024-09-13T18:54:10.013062+0000","last_fullsized":"2024-09-13T18:54:10.013062+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:53:54.004733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.015423+0000","last_change":"2024-09-13T18:54:09.041537+0000","last_active":"2024-09-13T18:54:10.015423+0000","last_peered":"2024-09-13T18:54:10.015423+0000","last_clean":"2024-09-13T18:54:10.015423+0000","last_became_active":"2024-09-13T18:54:09.041240+0000","last_became_peered":"2024-09-13T18:54:09.041240+0000","last_unstale":"2024-09-13T18:54:10.015423+0000","last_undegraded":"2024-09-13T18:54:10.015423+0000","last_fullsized":"2024-09-13T18:54:10.015423+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:18.717450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.018196+0000","last_change":"2024-09-13T18:54:09.037869+0000","last_active":"2024-09-13T18:54:10.018196+0000","last_peered":"2024-09-13T18:54:10.018196+0000","last_clean":"2024-09-13T18:54:10.018196+0000","last_became_active":"2024-09-13T18:54:09.037649+0000","last_became_peered":"2024-09-13T18:54:09.037649+0000","last_unstale":"2024-09-13T18:54:10.018196+0000","last_undegraded":"2024-09-13T18:54:10.018196+0000","last_fullsized":"2024-09-13T18:54:10.018196+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:07.303524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017741+0000","last_change":"2024-09-13T18:54:09.041474+0000","last_active":"2024-09-13T18:54:10.017741+0000","last_peered":"2024-09-13T18:54:10.017741+0000","last_clean":"2024-09-13T18:54:10.017741+0000","last_became_active":"2024-09-13T18:54:09.041187+0000","last_became_peered":"2024-09-13T18:54:09.041187+0000","last_unstale":"2024-09-13T18:54:10.017741+0000","last_undegraded":"2024-09-13T18:54:10.017741+0000","last_fullsized":"2024-09-13T18:54:10.017741+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:16:58.895680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017017+0000","last_change":"2024-09-13T18:54:09.032137+0000","last_active":"2024-09-13T18:54:10.017017+0000","last_peered":"2024-09-13T18:54:10.017017+0000","last_clean":"2024-09-13T18:54:10.017017+0000","last_became_active":"2024-09-13T18:54:09.031750+0000","last_became_peered":"2024-09-13T18:54:09.031750+0000","last_unstale":"2024-09-13T18:54:10.017017+0000","last_undegraded":"2024-09-13T18:54:10.017017+0000","last_fullsized":"2024-09-13T18:54:10.017017+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:34:15.440622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511526+0000","last_change":"2024-09-13T18:54:09.035971+0000","last_active":"2024-09-13T18:54:10.511526+0000","last_peered":"2024-09-13T18:54:10.511526+0000","last_clean":"2024-09-13T18:54:10.511526+0000","last_became_active":"2024-09-13T18:54:09.035755+0000","last_became_peered":"2024-09-13T18:54:09.035755+0000","last_unstale":"2024-09-13T18:54:10.511526+0000","last_undegraded":"2024-09-13T18:54:10.511526+0000","last_fullsized":"2024-09-13T18:54:10.511526+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:25.641687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013166+0000","last_change":"2024-09-13T18:54:09.020556+0000","last_active":"2024-09-13T18:54:10.013166+0000","last_peered":"2024-09-13T18:54:10.013166+0000","last_clean":"2024-09-13T18:54:10.013166+0000","last_became_active":"2024-09-13T18:54:09.020163+0000","last_became_peered":"2024-09-13T18:54:09.020163+0000","last_unstale":"2024-09-13T18:54:10.013166+0000","last_undegraded":"2024-09-13T18:54:10.013166+0000","last_fullsized":"2024-09-13T18:54:10.013166+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:12:16.234055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511084+0000","last_change":"2024-09-13T18:54:09.040013+0000","last_active":"2024-09-13T18:54:10.511084+0000","last_peered":"2024-09-13T18:54:10.511084+0000","last_clean":"2024-09-13T18:54:10.511084+0000","last_became_active":"2024-09-13T18:54:09.039781+0000","last_became_peered":"2024-09-13T18:54:09.039781+0000","last_unstale":"2024-09-13T18:54:10.511084+0000","last_undegraded":"2024-09-13T18:54:10.511084+0000","last_fullsized":"2024-09-13T18:54:10.511084+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:20.469932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.015576+0000","last_change":"2024-09-13T18:54:09.034230+0000","last_active":"2024-09-13T18:54:10.015576+0000","last_peered":"2024-09-13T18:54:10.015576+0000","last_clean":"2024-09-13T18:54:10.015576+0000","last_became_active":"2024-09-13T18:54:09.033797+0000","last_became_peered":"2024-09-13T18:54:09.033797+0000","last_unstale":"2024-09-13T18:54:10.015576+0000","last_undegraded":"2024-09-13T18:54:10.015576+0000","last_fullsized":"2024-09-13T18:54:10.015576+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:36:45.544218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.015449+0000","last_change":"2024-09-13T18:54:09.034163+0000","last_active":"2024-09-13T18:54:10.015449+0000","last_peered":"2024-09-13T18:54:10.015449+0000","last_clean":"2024-09-13T18:54:10.015449+0000","last_became_active":"2024-09-13T18:54:09.033892+0000","last_became_peered":"2024-09-13T18:54:09.033892+0000","last_unstale":"2024-09-13T18:54:10.015449+0000","last_undegraded":"2024-09-13T18:54:10.015449+0000","last_fullsized":"2024-09-13T18:54:10.015449+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:11:22.207917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017445+0000","last_change":"2024-09-13T18:54:09.037297+0000","last_active":"2024-09-13T18:54:10.017445+0000","last_peered":"2024-09-13T18:54:10.017445+0000","last_clean":"2024-09-13T18:54:10.017445+0000","last_became_active":"2024-09-13T18:54:09.033915+0000","last_became_peered":"2024-09-13T18:54:09.033915+0000","last_unstale":"2024-09-13T18:54:10.017445+0000","last_undegraded":"2024-09-13T18:54:10.017445+0000","last_fullsized":"2024-09-13T18:54:10.017445+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:32:03.439867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512043+0000","last_change":"2024-09-13T18:54:09.036493+0000","last_active":"2024-09-13T18:54:10.512043+0000","last_peered":"2024-09-13T18:54:10.512043+0000","last_clean":"2024-09-13T18:54:10.512043+0000","last_became_active":"2024-09-13T18:54:09.035712+0000","last_became_peered":"2024-09-13T18:54:09.035712+0000","last_unstale":"2024-09-13T18:54:10.512043+0000","last_undegraded":"2024-09-13T18:54:10.512043+0000","last_fullsized":"2024-09-13T18:54:10.512043+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:18:23.698995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017770+0000","last_change":"2024-09-13T18:54:09.038968+0000","last_active":"2024-09-13T18:54:10.017770+0000","last_peered":"2024-09-13T18:54:10.017770+0000","last_clean":"2024-09-13T18:54:10.017770+0000","last_became_active":"2024-09-13T18:54:09.038709+0000","last_became_peered":"2024-09-13T18:54:09.038709+0000","last_unstale":"2024-09-13T18:54:10.017770+0000","last_undegraded":"2024-09-13T18:54:10.017770+0000","last_fullsized":"2024-09-13T18:54:10.017770+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:26:20.982783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513352+0000","last_change":"2024-09-13T18:54:09.042642+0000","last_active":"2024-09-13T18:54:10.513352+0000","last_peered":"2024-09-13T18:54:10.513352+0000","last_clean":"2024-09-13T18:54:10.513352+0000","last_became_active":"2024-09-13T18:54:09.042336+0000","last_became_peered":"2024-09-13T18:54:09.042336+0000","last_unstale":"2024-09-13T18:54:10.513352+0000","last_undegraded":"2024-09-13T18:54:10.513352+0000","last_fullsized":"2024-09-13T18:54:10.513352+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:17.507996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.016108+0000","last_change":"2024-09-13T18:54:09.034012+0000","last_active":"2024-09-13T18:54:10.016108+0000","last_peered":"2024-09-13T18:54:10.016108+0000","last_clean":"2024-09-13T18:54:10.016108+0000","last_became_active":"2024-09-13T18:54:09.033699+0000","last_became_peered":"2024-09-13T18:54:09.033699+0000","last_unstale":"2024-09-13T18:54:10.016108+0000","last_undegraded":"2024-09-13T18:54:10.016108+0000","last_fullsized":"2024-09-13T18:54:10.016108+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:33:12.532879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.016309+0000","last_change":"2024-09-13T18:54:09.031651+0000","last_active":"2024-09-13T18:54:10.016309+0000","last_peered":"2024-09-13T18:54:10.016309+0000","last_clean":"2024-09-13T18:54:10.016309+0000","last_became_active":"2024-09-13T18:54:09.031329+0000","last_became_peered":"2024-09-13T18:54:09.031329+0000","last_unstale":"2024-09-13T18:54:10.016309+0000","last_undegraded":"2024-09-13T18:54:10.016309+0000","last_fullsized":"2024-09-13T18:54:10.016309+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:46:42.484744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013003+0000","last_change":"2024-09-13T18:54:09.032424+0000","last_active":"2024-09-13T18:54:10.013003+0000","last_peered":"2024-09-13T18:54:10.013003+0000","last_clean":"2024-09-13T18:54:10.013003+0000","last_became_active":"2024-09-13T18:54:09.032219+0000","last_became_peered":"2024-09-13T18:54:09.032219+0000","last_unstale":"2024-09-13T18:54:10.013003+0000","last_undegraded":"2024-09-13T18:54:10.013003+0000","last_fullsized":"2024-09-13T18:54:10.013003+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:24:36.307754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510608+0000","last_change":"2024-09-13T18:54:09.031951+0000","last_active":"2024-09-13T18:54:10.510608+0000","last_peered":"2024-09-13T18:54:10.510608+0000","last_clean":"2024-09-13T18:54:10.510608+0000","last_became_active":"2024-09-13T18:54:09.031229+0000","last_became_peered":"2024-09-13T18:54:09.031229+0000","last_unstale":"2024-09-13T18:54:10.510608+0000","last_undegraded":"2024-09-13T18:54:10.510608+0000","last_fullsized":"2024-09-13T18:54:10.510608+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:21:06.044703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513255+0000","last_change":"2024-09-13T18:54:09.029453+0000","last_active":"2024-09-13T18:54:10.513255+0000","last_peered":"2024-09-13T18:54:10.513255+0000","last_clean":"2024-09-13T18:54:10.513255+0000","last_became_active":"2024-09-13T18:54:09.029203+0000","last_became_peered":"2024-09-13T18:54:09.029203+0000","last_unstale":"2024-09-13T18:54:10.513255+0000","last_undegraded":"2024-09-13T18:54:10.513255+0000","last_fullsized":"2024-09-13T18:54:10.513255+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:34:15.440622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"51'4","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013838+0000","last_change":"2024-09-13T18:54:08.641174+0000","last_active":"2024-09-13T18:54:10.013838+0000","last_peered":"2024-09-13T18:54:10.013838+0000","last_clean":"2024-09-13T18:54:10.013838+0000","last_became_active":"2024-09-13T18:54:08.640895+0000","last_became_peered":"2024-09-13T18:54:08.640895+0000","last_unstale":"2024-09-13T18:54:10.013838+0000","last_undegraded":"2024-09-13T18:54:10.013838+0000","last_fullsized":"2024-09-13T18:54:10.013838+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:46:58.739769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017448+0000","last_change":"2024-09-13T18:54:09.030151+0000","last_active":"2024-09-13T18:54:10.017448+0000","last_peered":"2024-09-13T18:54:10.017448+0000","last_clean":"2024-09-13T18:54:10.017448+0000","last_became_active":"2024-09-13T18:54:09.029846+0000","last_became_peered":"2024-09-13T18:54:09.029846+0000","last_unstale":"2024-09-13T18:54:10.017448+0000","last_undegraded":"2024-09-13T18:54:10.017448+0000","last_fullsized":"2024-09-13T18:54:10.017448+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:38.864994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"51'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.018235+0000","last_change":"2024-09-13T18:54:08.642329+0000","last_active":"2024-09-13T18:54:10.018235+0000","last_peered":"2024-09-13T18:54:10.018235+0000","last_clean":"2024-09-13T18:54:10.018235+0000","last_became_active":"2024-09-13T18:54:08.642016+0000","last_became_peered":"2024-09-13T18:54:08.642016+0000","last_unstale":"2024-09-13T18:54:10.018235+0000","last_undegraded":"2024-09-13T18:54:10.018235+0000","last_fullsized":"2024-09-13T18:54:10.018235+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:11.214204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513459+0000","last_change":"2024-09-13T18:54:09.034541+0000","last_active":"2024-09-13T18:54:10.513459+0000","last_peered":"2024-09-13T18:54:10.513459+0000","last_clean":"2024-09-13T18:54:10.513459+0000","last_became_active":"2024-09-13T18:54:09.033634+0000","last_became_peered":"2024-09-13T18:54:09.033634+0000","last_unstale":"2024-09-13T18:54:10.513459+0000","last_undegraded":"2024-09-13T18:54:10.513459+0000","last_fullsized":"2024-09-13T18:54:10.513459+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:50:57.913889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"51'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513511+0000","last_change":"2024-09-13T18:54:08.641770+0000","last_active":"2024-09-13T18:54:10.513511+0000","last_peered":"2024-09-13T18:54:10.513511+0000","last_clean":"2024-09-13T18:54:10.513511+0000","last_became_active":"2024-09-13T18:54:08.641394+0000","last_became_peered":"2024-09-13T18:54:08.641394+0000","last_unstale":"2024-09-13T18:54:10.513511+0000","last_undegraded":"2024-09-13T18:54:10.513511+0000","last_fullsized":"2024-09-13T18:54:10.513511+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:43.402295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513480+0000","last_change":"2024-09-13T18:54:09.038714+0000","last_active":"2024-09-13T18:54:10.513480+0000","last_peered":"2024-09-13T18:54:10.513480+0000","last_clean":"2024-09-13T18:54:10.513480+0000","last_became_active":"2024-09-13T18:54:09.038479+0000","last_became_peered":"2024-09-13T18:54:09.038479+0000","last_unstale":"2024-09-13T18:54:10.513480+0000","last_undegraded":"2024-09-13T18:54:10.513480+0000","last_fullsized":"2024-09-13T18:54:10.513480+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:59.323928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"51'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512032+0000","last_change":"2024-09-13T18:54:08.644572+0000","last_active":"2024-09-13T18:54:10.512032+0000","last_peered":"2024-09-13T18:54:10.512032+0000","last_clean":"2024-09-13T18:54:10.512032+0000","last_became_active":"2024-09-13T18:54:08.644201+0000","last_became_peered":"2024-09-13T18:54:08.644201+0000","last_unstale":"2024-09-13T18:54:10.512032+0000","last_undegraded":"2024-09-13T18:54:10.512032+0000","last_fullsized":"2024-09-13T18:54:10.512032+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:21:12.843255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.016863+0000","last_change":"2024-09-13T18:54:09.028991+0000","last_active":"2024-09-13T18:54:10.016863+0000","last_peered":"2024-09-13T18:54:10.016863+0000","last_clean":"2024-09-13T18:54:10.016863+0000","last_became_active":"2024-09-13T18:54:09.028639+0000","last_became_peered":"2024-09-13T18:54:09.028639+0000","last_unstale":"2024-09-13T18:54:10.016863+0000","last_undegraded":"2024-09-13T18:54:10.016863+0000","last_fullsized":"2024-09-13T18:54:10.016863+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:20.973711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"51'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013394+0000","last_change":"2024-09-13T18:54:08.640456+0000","last_active":"2024-09-13T18:54:10.013394+0000","last_peered":"2024-09-13T18:54:10.013394+0000","last_clean":"2024-09-13T18:54:10.013394+0000","last_became_active":"2024-09-13T18:54:08.640140+0000","last_became_peered":"2024-09-13T18:54:08.640140+0000","last_unstale":"2024-09-13T18:54:10.013394+0000","last_undegraded":"2024-09-13T18:54:10.013394+0000","last_fullsized":"2024-09-13T18:54:10.013394+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:27:57.708775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511491+0000","last_change":"2024-09-13T18:54:09.042642+0000","last_active":"2024-09-13T18:54:10.511491+0000","last_peered":"2024-09-13T18:54:10.511491+0000","last_clean":"2024-09-13T18:54:10.511491+0000","last_became_active":"2024-09-13T18:54:09.042346+0000","last_became_peered":"2024-09-13T18:54:09.042346+0000","last_unstale":"2024-09-13T18:54:10.511491+0000","last_undegraded":"2024-09-13T18:54:10.511491+0000","last_fullsized":"2024-09-13T18:54:10.511491+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:03.777920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"51'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017125+0000","last_change":"2024-09-13T18:54:08.637002+0000","last_active":"2024-09-13T18:54:10.017125+0000","last_peered":"2024-09-13T18:54:10.017125+0000","last_clean":"2024-09-13T18:54:10.017125+0000","last_became_active":"2024-09-13T18:54:08.636742+0000","last_became_peered":"2024-09-13T18:54:08.636742+0000","last_unstale":"2024-09-13T18:54:10.017125+0000","last_undegraded":"2024-09-13T18:54:10.017125+0000","last_fullsized":"2024-09-13T18:54:10.017125+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:12:16.055887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510378+0000","last_change":"2024-09-13T18:54:09.042484+0000","last_active":"2024-09-13T18:54:10.510378+0000","last_peered":"2024-09-13T18:54:10.510378+0000","last_clean":"2024-09-13T18:54:10.510378+0000","last_became_active":"2024-09-13T18:54:09.040094+0000","last_became_peered":"2024-09-13T18:54:09.040094+0000","last_unstale":"2024-09-13T18:54:10.510378+0000","last_undegraded":"2024-09-13T18:54:10.510378+0000","last_fullsized":"2024-09-13T18:54:10.510378+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:54:17.507996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512796+0000","last_change":"2024-09-13T18:54:08.638208+0000","last_active":"2024-09-13T18:54:10.512796+0000","last_peered":"2024-09-13T18:54:10.512796+0000","last_clean":"2024-09-13T18:54:10.512796+0000","last_became_active":"2024-09-13T18:54:08.636291+0000","last_became_peered":"2024-09-13T18:54:08.636291+0000","last_unstale":"2024-09-13T18:54:10.512796+0000","last_undegraded":"2024-09-13T18:54:10.512796+0000","last_fullsized":"2024-09-13T18:54:10.512796+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:46:58.739769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.015315+0000","last_change":"2024-09-13T18:54:09.033779+0000","last_active":"2024-09-13T18:54:10.015315+0000","last_peered":"2024-09-13T18:54:10.015315+0000","last_clean":"2024-09-13T18:54:10.015315+0000","last_became_active":"2024-09-13T18:54:09.033418+0000","last_became_peered":"2024-09-13T18:54:09.033418+0000","last_unstale":"2024-09-13T18:54:10.015315+0000","last_undegraded":"2024-09-13T18:54:10.015315+0000","last_fullsized":"2024-09-13T18:54:10.015315+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:35:24.626210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510353+0000","last_change":"2024-09-13T18:54:08.636504+0000","last_active":"2024-09-13T18:54:10.510353+0000","last_peered":"2024-09-13T18:54:10.510353+0000","last_clean":"2024-09-13T18:54:10.510353+0000","last_became_active":"2024-09-13T18:54:08.635726+0000","last_became_peered":"2024-09-13T18:54:08.635726+0000","last_unstale":"2024-09-13T18:54:10.510353+0000","last_undegraded":"2024-09-13T18:54:10.510353+0000","last_fullsized":"2024-09-13T18:54:10.510353+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:29.334736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017950+0000","last_change":"2024-09-13T18:54:09.031190+0000","last_active":"2024-09-13T18:54:10.017950+0000","last_peered":"2024-09-13T18:54:10.017950+0000","last_clean":"2024-09-13T18:54:10.017950+0000","last_became_active":"2024-09-13T18:54:09.030885+0000","last_became_peered":"2024-09-13T18:54:09.030885+0000","last_unstale":"2024-09-13T18:54:10.017950+0000","last_undegraded":"2024-09-13T18:54:10.017950+0000","last_fullsized":"2024-09-13T18:54:10.017950+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:44.533584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.018378+0000","last_change":"2024-09-13T18:54:08.633875+0000","last_active":"2024-09-13T18:54:10.018378+0000","last_peered":"2024-09-13T18:54:10.018378+0000","last_clean":"2024-09-13T18:54:10.018378+0000","last_became_active":"2024-09-13T18:54:08.633529+0000","last_became_peered":"2024-09-13T18:54:08.633529+0000","last_unstale":"2024-09-13T18:54:10.018378+0000","last_undegraded":"2024-09-13T18:54:10.018378+0000","last_fullsized":"2024-09-13T18:54:10.018378+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:05.014350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512195+0000","last_change":"2024-09-13T18:54:09.036011+0000","last_active":"2024-09-13T18:54:10.512195+0000","last_peered":"2024-09-13T18:54:10.512195+0000","last_clean":"2024-09-13T18:54:10.512195+0000","last_became_active":"2024-09-13T18:54:09.035739+0000","last_became_peered":"2024-09-13T18:54:09.035739+0000","last_unstale":"2024-09-13T18:54:10.512195+0000","last_undegraded":"2024-09-13T18:54:10.512195+0000","last_fullsized":"2024-09-13T18:54:10.512195+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:44:20.469932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511856+0000","last_change":"2024-09-13T18:54:08.641983+0000","last_active":"2024-09-13T18:54:10.511856+0000","last_peered":"2024-09-13T18:54:10.511856+0000","last_clean":"2024-09-13T18:54:10.511856+0000","last_became_active":"2024-09-13T18:54:08.641640+0000","last_became_peered":"2024-09-13T18:54:08.641640+0000","last_unstale":"2024-09-13T18:54:10.511856+0000","last_undegraded":"2024-09-13T18:54:10.511856+0000","last_fullsized":"2024-09-13T18:54:10.511856+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:43.402295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013434+0000","last_change":"2024-09-13T18:54:09.018322+0000","last_active":"2024-09-13T18:54:10.013434+0000","last_peered":"2024-09-13T18:54:10.013434+0000","last_clean":"2024-09-13T18:54:10.013434+0000","last_became_active":"2024-09-13T18:54:09.018017+0000","last_became_peered":"2024-09-13T18:54:09.018017+0000","last_unstale":"2024-09-13T18:54:10.013434+0000","last_undegraded":"2024-09-13T18:54:10.013434+0000","last_fullsized":"2024-09-13T18:54:10.013434+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:43:16.925363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"51'3","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513037+0000","last_change":"2024-09-13T18:54:08.643783+0000","last_active":"2024-09-13T18:54:10.513037+0000","last_peered":"2024-09-13T18:54:10.513037+0000","last_clean":"2024-09-13T18:54:10.513037+0000","last_became_active":"2024-09-13T18:54:08.643433+0000","last_became_peered":"2024-09-13T18:54:08.643433+0000","last_unstale":"2024-09-13T18:54:10.513037+0000","last_undegraded":"2024-09-13T18:54:10.513037+0000","last_fullsized":"2024-09-13T18:54:10.513037+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:47.725231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512611+0000","last_change":"2024-09-13T18:54:09.030914+0000","last_active":"2024-09-13T18:54:10.512611+0000","last_peered":"2024-09-13T18:54:10.512611+0000","last_clean":"2024-09-13T18:54:10.512611+0000","last_became_active":"2024-09-13T18:54:09.030639+0000","last_became_peered":"2024-09-13T18:54:09.030639+0000","last_unstale":"2024-09-13T18:54:10.512611+0000","last_undegraded":"2024-09-13T18:54:10.512611+0000","last_fullsized":"2024-09-13T18:54:10.512611+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:38:33.066000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"51'6","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511636+0000","last_change":"2024-09-13T18:54:08.632413+0000","last_active":"2024-09-13T18:54:10.511636+0000","last_peered":"2024-09-13T18:54:10.511636+0000","last_clean":"2024-09-13T18:54:10.511636+0000","last_became_active":"2024-09-13T18:54:08.632154+0000","last_became_peered":"2024-09-13T18:54:08.632154+0000","last_unstale":"2024-09-13T18:54:10.511636+0000","last_undegraded":"2024-09-13T18:54:10.511636+0000","last_fullsized":"2024-09-13T18:54:10.511636+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:05.014350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.016198+0000","last_change":"2024-09-13T18:54:09.031848+0000","last_active":"2024-09-13T18:54:10.016198+0000","last_peered":"2024-09-13T18:54:10.016198+0000","last_clean":"2024-09-13T18:54:10.016198+0000","last_became_active":"2024-09-13T18:54:09.031589+0000","last_became_peered":"2024-09-13T18:54:09.031589+0000","last_unstale":"2024-09-13T18:54:10.016198+0000","last_undegraded":"2024-09-13T18:54:10.016198+0000","last_fullsized":"2024-09-13T18:54:10.016198+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:50.905909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"51'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013554+0000","last_change":"2024-09-13T18:54:08.634808+0000","last_active":"2024-09-13T18:54:10.013554+0000","last_peered":"2024-09-13T18:54:10.013554+0000","last_clean":"2024-09-13T18:54:10.013554+0000","last_became_active":"2024-09-13T18:54:08.634448+0000","last_became_peered":"2024-09-13T18:54:08.634448+0000","last_unstale":"2024-09-13T18:54:10.013554+0000","last_undegraded":"2024-09-13T18:54:10.013554+0000","last_fullsized":"2024-09-13T18:54:10.013554+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:49.647759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510987+0000","last_change":"2024-09-13T18:54:09.038992+0000","last_active":"2024-09-13T18:54:10.510987+0000","last_peered":"2024-09-13T18:54:10.510987+0000","last_clean":"2024-09-13T18:54:10.510987+0000","last_became_active":"2024-09-13T18:54:09.036624+0000","last_became_peered":"2024-09-13T18:54:09.036624+0000","last_unstale":"2024-09-13T18:54:10.510987+0000","last_undegraded":"2024-09-13T18:54:10.510987+0000","last_fullsized":"2024-09-13T18:54:10.510987+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:30:48.528143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512212+0000","last_change":"2024-09-13T18:54:08.642806+0000","last_active":"2024-09-13T18:54:10.512212+0000","last_peered":"2024-09-13T18:54:10.512212+0000","last_clean":"2024-09-13T18:54:10.512212+0000","last_became_active":"2024-09-13T18:54:08.641339+0000","last_became_peered":"2024-09-13T18:54:08.641339+0000","last_unstale":"2024-09-13T18:54:10.512212+0000","last_undegraded":"2024-09-13T18:54:10.512212+0000","last_fullsized":"2024-09-13T18:54:10.512212+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:49.894463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.012954+0000","last_change":"2024-09-13T18:54:09.018040+0000","last_active":"2024-09-13T18:54:10.012954+0000","last_peered":"2024-09-13T18:54:10.012954+0000","last_clean":"2024-09-13T18:54:10.012954+0000","last_became_active":"2024-09-13T18:54:09.017747+0000","last_became_peered":"2024-09-13T18:54:09.017747+0000","last_unstale":"2024-09-13T18:54:10.012954+0000","last_undegraded":"2024-09-13T18:54:10.012954+0000","last_fullsized":"2024-09-13T18:54:10.012954+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:17:26.348636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013060+0000","last_change":"2024-09-13T18:54:08.633363+0000","last_active":"2024-09-13T18:54:10.013060+0000","last_peered":"2024-09-13T18:54:10.013060+0000","last_clean":"2024-09-13T18:54:10.013060+0000","last_became_active":"2024-09-13T18:54:08.633093+0000","last_became_peered":"2024-09-13T18:54:08.633093+0000","last_unstale":"2024-09-13T18:54:10.013060+0000","last_undegraded":"2024-09-13T18:54:10.013060+0000","last_fullsized":"2024-09-13T18:54:10.013060+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:20:21.015963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512167+0000","last_change":"2024-09-13T18:54:09.036997+0000","last_active":"2024-09-13T18:54:10.512167+0000","last_peered":"2024-09-13T18:54:10.512167+0000","last_clean":"2024-09-13T18:54:10.512167+0000","last_became_active":"2024-09-13T18:54:09.036719+0000","last_became_peered":"2024-09-13T18:54:09.036719+0000","last_unstale":"2024-09-13T18:54:10.512167+0000","last_undegraded":"2024-09-13T18:54:10.512167+0000","last_fullsized":"2024-09-13T18:54:10.512167+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:27:58.720221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"51'8","reported_seq":19,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511694+0000","last_change":"2024-09-13T18:54:08.641810+0000","last_active":"2024-09-13T18:54:10.511694+0000","last_peered":"2024-09-13T18:54:10.511694+0000","last_clean":"2024-09-13T18:54:10.511694+0000","last_became_active":"2024-09-13T18:54:08.641451+0000","last_became_peered":"2024-09-13T18:54:08.641451+0000","last_unstale":"2024-09-13T18:54:10.511694+0000","last_undegraded":"2024-09-13T18:54:10.511694+0000","last_fullsized":"2024-09-13T18:54:10.511694+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:20:16.425833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512299+0000","last_change":"2024-09-13T18:54:09.035525+0000","last_active":"2024-09-13T18:54:10.512299+0000","last_peered":"2024-09-13T18:54:10.512299+0000","last_clean":"2024-09-13T18:54:10.512299+0000","last_became_active":"2024-09-13T18:54:09.035270+0000","last_became_peered":"2024-09-13T18:54:09.035270+0000","last_unstale":"2024-09-13T18:54:10.512299+0000","last_undegraded":"2024-09-13T18:54:10.512299+0000","last_fullsized":"2024-09-13T18:54:10.512299+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:52.700687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"51'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512754+0000","last_change":"2024-09-13T18:54:08.632985+0000","last_active":"2024-09-13T18:54:10.512754+0000","last_peered":"2024-09-13T18:54:10.512754+0000","last_clean":"2024-09-13T18:54:10.512754+0000","last_became_active":"2024-09-13T18:54:08.632713+0000","last_became_peered":"2024-09-13T18:54:08.632713+0000","last_unstale":"2024-09-13T18:54:10.512754+0000","last_undegraded":"2024-09-13T18:54:10.512754+0000","last_fullsized":"2024-09-13T18:54:10.512754+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:34:32.208511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511928+0000","last_change":"2024-09-13T18:54:09.033978+0000","last_active":"2024-09-13T18:54:10.511928+0000","last_peered":"2024-09-13T18:54:10.511928+0000","last_clean":"2024-09-13T18:54:10.511928+0000","last_became_active":"2024-09-13T18:54:09.033706+0000","last_became_peered":"2024-09-13T18:54:09.033706+0000","last_unstale":"2024-09-13T18:54:10.511928+0000","last_undegraded":"2024-09-13T18:54:10.511928+0000","last_fullsized":"2024-09-13T18:54:10.511928+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:50.172893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"51'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.018040+0000","last_change":"2024-09-13T18:54:08.633785+0000","last_active":"2024-09-13T18:54:10.018040+0000","last_peered":"2024-09-13T18:54:10.018040+0000","last_clean":"2024-09-13T18:54:10.018040+0000","last_became_active":"2024-09-13T18:54:08.633479+0000","last_became_peered":"2024-09-13T18:54:08.633479+0000","last_unstale":"2024-09-13T18:54:10.018040+0000","last_undegraded":"2024-09-13T18:54:10.018040+0000","last_fullsized":"2024-09-13T18:54:10.018040+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:30.803360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.016406+0000","last_change":"2024-09-13T18:54:09.028606+0000","last_active":"2024-09-13T18:54:10.016406+0000","last_peered":"2024-09-13T18:54:10.016406+0000","last_clean":"2024-09-13T18:54:10.016406+0000","last_became_active":"2024-09-13T18:54:09.028354+0000","last_became_peered":"2024-09-13T18:54:09.028354+0000","last_unstale":"2024-09-13T18:54:10.016406+0000","last_undegraded":"2024-09-13T18:54:10.016406+0000","last_fullsized":"2024-09-13T18:54:10.016406+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:42:28.771252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"51'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513017+0000","last_change":"2024-09-13T18:54:08.638649+0000","last_active":"2024-09-13T18:54:10.513017+0000","last_peered":"2024-09-13T18:54:10.513017+0000","last_clean":"2024-09-13T18:54:10.513017+0000","last_became_active":"2024-09-13T18:54:08.638402+0000","last_became_peered":"2024-09-13T18:54:08.638402+0000","last_unstale":"2024-09-13T18:54:10.513017+0000","last_undegraded":"2024-09-13T18:54:10.513017+0000","last_fullsized":"2024-09-13T18:54:10.513017+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:36.231227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.016876+0000","last_change":"2024-09-13T18:54:09.034238+0000","last_active":"2024-09-13T18:54:10.016876+0000","last_peered":"2024-09-13T18:54:10.016876+0000","last_clean":"2024-09-13T18:54:10.016876+0000","last_became_active":"2024-09-13T18:54:09.033625+0000","last_became_peered":"2024-09-13T18:54:09.033625+0000","last_unstale":"2024-09-13T18:54:10.016876+0000","last_undegraded":"2024-09-13T18:54:10.016876+0000","last_fullsized":"2024-09-13T18:54:10.016876+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:59.323928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013499+0000","last_change":"2024-09-13T18:54:08.638582+0000","last_active":"2024-09-13T18:54:10.013499+0000","last_peered":"2024-09-13T18:54:10.013499+0000","last_clean":"2024-09-13T18:54:10.013499+0000","last_became_active":"2024-09-13T18:54:08.638301+0000","last_became_peered":"2024-09-13T18:54:08.638301+0000","last_unstale":"2024-09-13T18:54:10.013499+0000","last_undegraded":"2024-09-13T18:54:10.013499+0000","last_fullsized":"2024-09-13T18:54:10.013499+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:06:36.231227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.012943+0000","last_change":"2024-09-13T18:54:09.018585+0000","last_active":"2024-09-13T18:54:10.012943+0000","last_peered":"2024-09-13T18:54:10.012943+0000","last_clean":"2024-09-13T18:54:10.012943+0000","last_became_active":"2024-09-13T18:54:09.018330+0000","last_became_peered":"2024-09-13T18:54:09.018330+0000","last_unstale":"2024-09-13T18:54:10.012943+0000","last_undegraded":"2024-09-13T18:54:10.012943+0000","last_fullsized":"2024-09-13T18:54:10.012943+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:03:03.283695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"51'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013757+0000","last_change":"2024-09-13T18:54:08.635266+0000","last_active":"2024-09-13T18:54:10.013757+0000","last_peered":"2024-09-13T18:54:10.013757+0000","last_clean":"2024-09-13T18:54:10.013757+0000","last_became_active":"2024-09-13T18:54:08.634849+0000","last_became_peered":"2024-09-13T18:54:08.634849+0000","last_unstale":"2024-09-13T18:54:10.013757+0000","last_undegraded":"2024-09-13T18:54:10.013757+0000","last_fullsized":"2024-09-13T18:54:10.013757+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:49.161446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513218+0000","last_change":"2024-09-13T18:54:09.033331+0000","last_active":"2024-09-13T18:54:10.513218+0000","last_peered":"2024-09-13T18:54:10.513218+0000","last_clean":"2024-09-13T18:54:10.513218+0000","last_became_active":"2024-09-13T18:54:09.033036+0000","last_became_peered":"2024-09-13T18:54:09.033036+0000","last_unstale":"2024-09-13T18:54:10.513218+0000","last_undegraded":"2024-09-13T18:54:10.513218+0000","last_fullsized":"2024-09-13T18:54:10.513218+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:59:18.717450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512975+0000","last_change":"2024-09-13T18:54:08.642111+0000","last_active":"2024-09-13T18:54:10.512975+0000","last_peered":"2024-09-13T18:54:10.512975+0000","last_clean":"2024-09-13T18:54:10.512975+0000","last_became_active":"2024-09-13T18:54:08.641851+0000","last_became_peered":"2024-09-13T18:54:08.641851+0000","last_unstale":"2024-09-13T18:54:10.512975+0000","last_undegraded":"2024-09-13T18:54:10.512975+0000","last_fullsized":"2024-09-13T18:54:10.512975+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:06:58.916296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510841+0000","last_change":"2024-09-13T18:54:09.030933+0000","last_active":"2024-09-13T18:54:10.510841+0000","last_peered":"2024-09-13T18:54:10.510841+0000","last_clean":"2024-09-13T18:54:10.510841+0000","last_became_active":"2024-09-13T18:54:09.030668+0000","last_became_peered":"2024-09-13T18:54:09.030668+0000","last_unstale":"2024-09-13T18:54:10.510841+0000","last_undegraded":"2024-09-13T18:54:10.510841+0000","last_fullsized":"2024-09-13T18:54:10.510841+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:58.743317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510734+0000","last_change":"2024-09-13T18:54:08.641045+0000","last_active":"2024-09-13T18:54:10.510734+0000","last_peered":"2024-09-13T18:54:10.510734+0000","last_clean":"2024-09-13T18:54:10.510734+0000","last_became_active":"2024-09-13T18:54:08.640761+0000","last_became_peered":"2024-09-13T18:54:08.640761+0000","last_unstale":"2024-09-13T18:54:10.510734+0000","last_undegraded":"2024-09-13T18:54:10.510734+0000","last_fullsized":"2024-09-13T18:54:10.510734+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:52.033459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510385+0000","last_change":"2024-09-13T18:54:09.036139+0000","last_active":"2024-09-13T18:54:10.510385+0000","last_peered":"2024-09-13T18:54:10.510385+0000","last_clean":"2024-09-13T18:54:10.510385+0000","last_became_active":"2024-09-13T18:54:09.035869+0000","last_became_peered":"2024-09-13T18:54:09.035869+0000","last_unstale":"2024-09-13T18:54:10.510385+0000","last_undegraded":"2024-09-13T18:54:10.510385+0000","last_fullsized":"2024-09-13T18:54:10.510385+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:32:03.439867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511704+0000","last_change":"2024-09-13T18:54:08.642596+0000","last_active":"2024-09-13T18:54:10.511704+0000","last_peered":"2024-09-13T18:54:10.511704+0000","last_clean":"2024-09-13T18:54:10.511704+0000","last_became_active":"2024-09-13T18:54:08.641301+0000","last_became_peered":"2024-09-13T18:54:08.641301+0000","last_unstale":"2024-09-13T18:54:10.511704+0000","last_undegraded":"2024-09-13T18:54:10.511704+0000","last_fullsized":"2024-09-13T18:54:10.511704+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:37.853548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017550+0000","last_change":"2024-09-13T18:54:09.038480+0000","last_active":"2024-09-13T18:54:10.017550+0000","last_peered":"2024-09-13T18:54:10.017550+0000","last_clean":"2024-09-13T18:54:10.017550+0000","last_became_active":"2024-09-13T18:54:09.038209+0000","last_became_peered":"2024-09-13T18:54:09.038209+0000","last_unstale":"2024-09-13T18:54:10.017550+0000","last_undegraded":"2024-09-13T18:54:10.017550+0000","last_fullsized":"2024-09-13T18:54:10.017550+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:59.919515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013558+0000","last_change":"2024-09-13T18:54:08.632672+0000","last_active":"2024-09-13T18:54:10.013558+0000","last_peered":"2024-09-13T18:54:10.013558+0000","last_clean":"2024-09-13T18:54:10.013558+0000","last_became_active":"2024-09-13T18:54:08.632438+0000","last_became_peered":"2024-09-13T18:54:08.632438+0000","last_unstale":"2024-09-13T18:54:10.013558+0000","last_undegraded":"2024-09-13T18:54:10.013558+0000","last_fullsized":"2024-09-13T18:54:10.013558+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:13.609544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511169+0000","last_change":"2024-09-13T18:54:09.034818+0000","last_active":"2024-09-13T18:54:10.511169+0000","last_peered":"2024-09-13T18:54:10.511169+0000","last_clean":"2024-09-13T18:54:10.511169+0000","last_became_active":"2024-09-13T18:54:09.034547+0000","last_became_peered":"2024-09-13T18:54:09.034547+0000","last_unstale":"2024-09-13T18:54:10.511169+0000","last_undegraded":"2024-09-13T18:54:10.511169+0000","last_fullsized":"2024-09-13T18:54:10.511169+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:27:58.720221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513232+0000","last_change":"2024-09-13T18:54:08.644292+0000","last_active":"2024-09-13T18:54:10.513232+0000","last_peered":"2024-09-13T18:54:10.513232+0000","last_clean":"2024-09-13T18:54:10.513232+0000","last_became_active":"2024-09-13T18:54:08.644026+0000","last_became_peered":"2024-09-13T18:54:08.644026+0000","last_unstale":"2024-09-13T18:54:10.513232+0000","last_undegraded":"2024-09-13T18:54:10.513232+0000","last_fullsized":"2024-09-13T18:54:10.513232+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:57:58.908069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512830+0000","last_change":"2024-09-13T18:54:09.038442+0000","last_active":"2024-09-13T18:54:10.512830+0000","last_peered":"2024-09-13T18:54:10.512830+0000","last_clean":"2024-09-13T18:54:10.512830+0000","last_became_active":"2024-09-13T18:54:09.038196+0000","last_became_peered":"2024-09-13T18:54:09.038196+0000","last_unstale":"2024-09-13T18:54:10.512830+0000","last_undegraded":"2024-09-13T18:54:10.512830+0000","last_fullsized":"2024-09-13T18:54:10.512830+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:38:03.777920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513109+0000","last_change":"2024-09-13T18:54:08.636715+0000","last_active":"2024-09-13T18:54:10.513109+0000","last_peered":"2024-09-13T18:54:10.513109+0000","last_clean":"2024-09-13T18:54:10.513109+0000","last_became_active":"2024-09-13T18:54:08.636436+0000","last_became_peered":"2024-09-13T18:54:08.636436+0000","last_unstale":"2024-09-13T18:54:10.513109+0000","last_undegraded":"2024-09-13T18:54:10.513109+0000","last_fullsized":"2024-09-13T18:54:10.513109+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:52.033459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013171+0000","last_change":"2024-09-13T18:54:09.029802+0000","last_active":"2024-09-13T18:54:10.013171+0000","last_peered":"2024-09-13T18:54:10.013171+0000","last_clean":"2024-09-13T18:54:10.013171+0000","last_became_active":"2024-09-13T18:54:09.029417+0000","last_became_peered":"2024-09-13T18:54:09.029417+0000","last_unstale":"2024-09-13T18:54:10.013171+0000","last_undegraded":"2024-09-13T18:54:10.013171+0000","last_fullsized":"2024-09-13T18:54:10.013171+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:31:00.210734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512511+0000","last_change":"2024-09-13T18:54:08.643816+0000","last_active":"2024-09-13T18:54:10.512511+0000","last_peered":"2024-09-13T18:54:10.512511+0000","last_clean":"2024-09-13T18:54:10.512511+0000","last_became_active":"2024-09-13T18:54:08.643521+0000","last_became_peered":"2024-09-13T18:54:08.643521+0000","last_unstale":"2024-09-13T18:54:10.512511+0000","last_undegraded":"2024-09-13T18:54:10.512511+0000","last_fullsized":"2024-09-13T18:54:10.512511+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:32:02.428421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.013343+0000","last_change":"2024-09-13T18:54:09.022602+0000","last_active":"2024-09-13T18:54:10.013343+0000","last_peered":"2024-09-13T18:54:10.013343+0000","last_clean":"2024-09-13T18:54:10.013343+0000","last_became_active":"2024-09-13T18:54:09.022295+0000","last_became_peered":"2024-09-13T18:54:09.022295+0000","last_unstale":"2024-09-13T18:54:10.013343+0000","last_undegraded":"2024-09-13T18:54:10.013343+0000","last_fullsized":"2024-09-13T18:54:10.013343+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:05:46.663258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.017335+0000","last_change":"2024-09-13T18:54:08.630805+0000","last_active":"2024-09-13T18:54:10.017335+0000","last_peered":"2024-09-13T18:54:10.017335+0000","last_clean":"2024-09-13T18:54:10.017335+0000","last_became_active":"2024-09-13T18:54:08.630559+0000","last_became_peered":"2024-09-13T18:54:08.630559+0000","last_unstale":"2024-09-13T18:54:10.017335+0000","last_undegraded":"2024-09-13T18:54:10.017335+0000","last_fullsized":"2024-09-13T18:54:10.017335+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:42:10.101010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512829+0000","last_change":"2024-09-13T18:54:09.032997+0000","last_active":"2024-09-13T18:54:10.512829+0000","last_peered":"2024-09-13T18:54:10.512829+0000","last_clean":"2024-09-13T18:54:10.512829+0000","last_became_active":"2024-09-13T18:54:09.032715+0000","last_became_peered":"2024-09-13T18:54:09.032715+0000","last_unstale":"2024-09-13T18:54:10.512829+0000","last_undegraded":"2024-09-13T18:54:10.512829+0000","last_fullsized":"2024-09-13T18:54:10.512829+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:38:33.066000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.512962+0000","last_change":"2024-09-13T18:54:08.638488+0000","last_active":"2024-09-13T18:54:10.512962+0000","last_peered":"2024-09-13T18:54:10.512962+0000","last_clean":"2024-09-13T18:54:10.512962+0000","last_became_active":"2024-09-13T18:54:08.638221+0000","last_became_peered":"2024-09-13T18:54:08.638221+0000","last_unstale":"2024-09-13T18:54:10.512962+0000","last_undegraded":"2024-09-13T18:54:10.512962+0000","last_fullsized":"2024-09-13T18:54:10.512962+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:29.334736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.016358+0000","last_change":"2024-09-13T18:54:09.034617+0000","last_active":"2024-09-13T18:54:10.016358+0000","last_peered":"2024-09-13T18:54:10.016358+0000","last_clean":"2024-09-13T18:54:10.016358+0000","last_became_active":"2024-09-13T18:54:09.034331+0000","last_became_peered":"2024-09-13T18:54:09.034331+0000","last_unstale":"2024-09-13T18:54:10.016358+0000","last_undegraded":"2024-09-13T18:54:10.016358+0000","last_fullsized":"2024-09-13T18:54:10.016358+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:54:52.700687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.510617+0000","last_change":"2024-09-13T18:54:08.633373+0000","last_active":"2024-09-13T18:54:10.510617+0000","last_peered":"2024-09-13T18:54:10.510617+0000","last_clean":"2024-09-13T18:54:10.510617+0000","last_became_active":"2024-09-13T18:54:08.633068+0000","last_became_peered":"2024-09-13T18:54:08.633068+0000","last_unstale":"2024-09-13T18:54:10.510617+0000","last_undegraded":"2024-09-13T18:54:10.510617+0000","last_fullsized":"2024-09-13T18:54:10.510617+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:08:50.580470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.511761+0000","last_change":"2024-09-13T18:54:09.039179+0000","last_active":"2024-09-13T18:54:10.511761+0000","last_peered":"2024-09-13T18:54:10.511761+0000","last_clean":"2024-09-13T18:54:10.511761+0000","last_became_active":"2024-09-13T18:54:09.036603+0000","last_became_peered":"2024-09-13T18:54:09.036603+0000","last_unstale":"2024-09-13T18:54:10.511761+0000","last_undegraded":"2024-09-13T18:54:10.511761+0000","last_fullsized":"2024-09-13T18:54:10.511761+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:08.602889+0000","last_clean_scrub_stamp":"2024-09-13T18:54:08.602889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:14.924883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"51'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-13T18:54:10.513272+0000","last_change":"2024-09-13T18:54:08.641300+0000","last_active":"2024-09-13T18:54:10.513272+0000","last_peered":"2024-09-13T18:54:10.513272+0000","last_clean":"2024-09-13T18:54:10.513272+0000","last_became_active":"2024-09-13T18:54:08.639819+0000","last_became_peered":"2024-09-13T18:54:08.639819+0000","last_unstale":"2024-09-13T18:54:10.513272+0000","last_undegraded":"2024-09-13T18:54:10.513272+0000","last_fullsized":"2024-09-13T18:54:10.513272+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:54:07.591443+0000","last_clean_scrub_stamp":"2024-09-13T18:54:07.591443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:54:49.647759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":184,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-13T18:54:09.023777+0000","last_change":"2024-09-13T18:52:15.080080+0000","last_active":"2024-09-13T18:54:09.023777+0000","last_peered":"2024-09-13T18:54:09.023777+0000","last_clean":"2024-09-13T18:54:09.023777+0000","last_became_active":"2024-09-13T18:52:15.078100+0000","last_became_peered":"2024-09-13T18:52:15.078100+0000","last_unstale":"2024-09-13T18:54:09.023777+0000","last_undegraded":"2024-09-13T18:54:09.023777+0000","last_fullsized":"2024-09-13T18:54:09.023777+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_clean_scrub_stamp":"2024-09-13T18:52:13.196473+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:29:59.452757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28192,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343648,"statfs":{"total":96636764160,"available":96607895552,"internally_reserved":0,"allocated":1196032,"data_stored":837002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.64500000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.56100000000000005}]},{"osd":4,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.73999999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.081,"15min":1.081},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.46700000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.76500000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":618496,"data_stored":253516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.706,"5min":1.706,"15min":1.706},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.97199999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":1.204}]},{"osd":2,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.91700000000000004}]},{"osd":3,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":1.1000000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":1.171},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":1.0780000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.004},{"interface":"front","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":1.2589999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":3.0859999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.23}]}]},{"osd":5,"up_from":14,"seq":60129542172,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":614400,"data_stored":252914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.3300000000000001,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.66600000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.70399999999999996}]},{"osd":2,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.996}]},{"osd":3,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.81699999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.05,"15min":1.05},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.91400000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.081}]},{"osd":7,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"max":{"1min":3.4369999999999998,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":1.107}]}]},{"osd":4,"up_from":14,"seq":60129542171,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":622592,"data_stored":254408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.747}]},{"osd":1,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.71499999999999997}]},{"osd":2,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.878}]},{"osd":3,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.84799999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.022}]},{"osd":7,"last update":"Fri Sep 13 18:53:13 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.94499999999999995}]}]},{"osd":3,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":622592,"data_stored":253482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.871},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.61799999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.70799999999999996}]},{"osd":2,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":1.081}]},{"osd":4,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.038,"5min":1.038,"15min":1.038},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.27},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.3140000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.893,"5min":1.893,"15min":1.893},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.1559999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.1200000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:53:12 2024","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.996},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.391}]}]},{"osd":2,"up_from":14,"seq":60129542171,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35816,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94336024,"statfs":{"total":96636764160,"available":96600088576,"internally_reserved":0,"allocated":614400,"data_stored":248130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.61399999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.93200000000000005}]},{"osd":3,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.82399999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.85199999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.796,"5min":1.796,"15min":1.796},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":1.1379999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":1.238}]},{"osd":7,"last update":"Fri Sep 13 18:53:11 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.2090000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":614400,"data_stored":252914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":2.786,"5min":2.786,"15min":2.786},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.78200000000000003}]},{"osd":2,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.113,"5min":3.113,"15min":3.113},"last":0.62},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.93700000000000006}]},{"osd":3,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.65300000000000002}]},{"osd":4,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.052}]},{"osd":5,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.4780000000000002,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":0.90900000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.878}]},{"osd":7,"last update":"Fri Sep 13 18:53:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.825,"5min":1.825,"15min":1.825},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.173,"5min":1.173,"15min":1.173},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.4409999999999998,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.0029999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1196032,"data_stored":836946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.55100000000000005}]},{"osd":2,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.006}]},{"osd":3,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":1.036},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.73499999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.091}]},{"osd":5,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.077,"5min":1.077,"15min":1.077},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":1.131},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.98999999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:53:14 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":1.5329999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:54:13.440 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T18:54:13.440 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T18:54:13.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.559+0000 7f67d8817700 1 Processor -- start 2024-09-13T18:54:13.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.559+0000 7f67d8817700 1 -- start start 2024-09-13T18:54:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d8817700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f67d415ce90 0x7f67d40ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d8817700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f67d406dde0 0x7f67d406e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d8817700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d40cd730 con 0x7f67d40cdbc0 2024-09-13T18:54:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d8817700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d406e760 con 0x7f67d406dde0 2024-09-13T18:54:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d8817700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d40d0850 con 0x7f67d415ce90 2024-09-13T18:54:13.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d359e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f67d40cdbc0 0x7f67d40ce050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36698/0 (socket says 172.21.15.7:36698) 2024-09-13T18:54:13.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d359e700 1 -- 172.21.15.7:0/3086931373 learned_addr learned my addr 172.21.15.7:0/3086931373 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67ca59c700 1 --2- 172.21.15.7:0/3086931373 >> v2:172.21.15.171:3300/0 conn(0x7f67d406dde0 0x7f67d406e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d2d9d700 1 --2- 172.21.15.7:0/3086931373 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f67d415ce90 0x7f67d40ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 -- 172.21.15.7:0/3086931373 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 691035116 0 0) 0x7f67d40cd730 con 0x7f67d40cdbc0 2024-09-13T18:54:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 -- 172.21.15.7:0/3086931373 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67c00035d0 con 0x7f67d40cdbc0 2024-09-13T18:54:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 -- 172.21.15.7:0/3086931373 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 106529129 0 0) 0x7f67c00035d0 con 0x7f67d40cdbc0 2024-09-13T18:54:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 -- 172.21.15.7:0/3086931373 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f67d415ce90 msgr2=0x7f67d40ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 --2- 172.21.15.7:0/3086931373 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f67d415ce90 0x7f67d40ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 -- 172.21.15.7:0/3086931373 >> v2:172.21.15.171:3300/0 conn(0x7f67d406dde0 msgr2=0x7f67d406e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 --2- 172.21.15.7:0/3086931373 >> v2:172.21.15.171:3300/0 conn(0x7f67d406dde0 0x7f67d406e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 -- 172.21.15.7:0/3086931373 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d40adf30 con 0x7f67d40cdbc0 2024-09-13T18:54:13.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 -- 172.21.15.7:0/3086931373 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7f67cc0019f0 con 0x7f67d40cdbc0 2024-09-13T18:54:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 -- 172.21.15.7:0/3086931373 >> v1:172.21.15.7:6789/0 conn(0x7f67d40cdbc0 legacy=0x7f67d40ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 --2- 172.21.15.7:0/3086931373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c0003a20 0x7f67c0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 --2- 172.21.15.7:0/3086931373 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f67d415ce90 0x7f67c0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 --2- 172.21.15.7:0/3086931373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67c0004db0 0x7f67c0005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 -- 172.21.15.7:0/3086931373 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d406e760 con 0x7f67c0003a20 2024-09-13T18:54:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 -- 172.21.15.7:0/3086931373 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d40d0850 con 0x7f67c0004db0 2024-09-13T18:54:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 -- 172.21.15.7:0/3086931373 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c000ba30 con 0x7f67d415ce90 2024-09-13T18:54:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d2d9d700 1 --2- 172.21.15.7:0/3086931373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c0003a20 0x7f67c0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67ca59c700 1 --2- 172.21.15.7:0/3086931373 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f67d415ce90 0x7f67c0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d359e700 1 --2- 172.21.15.7:0/3086931373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67c0004db0 0x7f67c0005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d2d9d700 1 -- 172.21.15.7:0/3086931373 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f67d415ce90 msgr2=0x7f67c0004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d2d9d700 1 --2- 172.21.15.7:0/3086931373 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f67d415ce90 0x7f67c0004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d2d9d700 1 -- 172.21.15.7:0/3086931373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67c0004db0 msgr2=0x7f67c0005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:13.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d2d9d700 1 --2- 172.21.15.7:0/3086931373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67c0004db0 0x7f67c0005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d2d9d700 1 -- 172.21.15.7:0/3086931373 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c00035d0 con 0x7f67c0003a20 2024-09-13T18:54:13.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d2d9d700 1 --2- 172.21.15.7:0/3086931373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c0003a20 0x7f67c0003e40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f67c4009b70 tx=0x7f67c00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c75b27a745e3ce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:13.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 -- 172.21.15.7:0/3086931373 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c400e6c0 con 0x7f67c0003a20 2024-09-13T18:54:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 -- 172.21.15.7:0/3086931373 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c400e820 con 0x7f67c0003a20 2024-09-13T18:54:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.563+0000 7f67d259c700 1 -- 172.21.15.7:0/3086931373 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c400eb90 con 0x7f67c0003a20 2024-09-13T18:54:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.567+0000 7f67d8817700 1 -- 172.21.15.7:0/3086931373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c0003a20 msgr2=0x7f67c0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.567+0000 7f67d8817700 1 --2- 172.21.15.7:0/3086931373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67c0003a20 0x7f67c0003e40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f67c4009b70 tx=0x7f67c00058a0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.567+0000 7f67d359e700 1 -- 172.21.15.7:0/3086931373 reap_dead start 2024-09-13T18:54:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.567+0000 7f67d8817700 1 -- 172.21.15.7:0/3086931373 shutdown_connections 2024-09-13T18:54:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.567+0000 7f67d8817700 1 -- 172.21.15.7:0/3086931373 >> 172.21.15.7:0/3086931373 conn(0x7f67d40cf9a0 msgr2=0x7f67d40cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.567+0000 7f67d8817700 1 -- 172.21.15.7:0/3086931373 shutdown_connections 2024-09-13T18:54:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.567+0000 7f67d8817700 1 -- 172.21.15.7:0/3086931373 wait complete. 2024-09-13T18:54:13.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.567+0000 7f67d8817700 1 Processor -- start 2024-09-13T18:54:13.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.567+0000 7f67d8817700 1 -- start start 2024-09-13T18:54:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67d8817700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67d41e7470 0x7f67d4155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67d8817700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f67d4155000 0x7f67d4143e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67d8817700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67d41e7850 0x7f67d4154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67d8817700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d4153310 con 0x7f67d41e7470 2024-09-13T18:54:13.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67d8817700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d4153560 con 0x7f67d41e7850 2024-09-13T18:54:13.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67d8817700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d4153840 con 0x7f67d4155000 2024-09-13T18:54:13.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67d2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67d41e7470 0x7f67d4155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67d2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67d41e7470 0x7f67d4155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55460/0 (socket says 172.21.15.7:55460) 2024-09-13T18:54:13.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67d2d9d700 1 -- 172.21.15.7:0/1432801110 learned_addr learned my addr 172.21.15.7:0/1432801110 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:13.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67d359e700 1 --2- 172.21.15.7:0/1432801110 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67d41e7850 0x7f67d4154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67ca59c700 1 --2- 172.21.15.7:0/1432801110 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f67d4155000 0x7f67d4143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67d2d9d700 1 -- 172.21.15.7:0/1432801110 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f67d4155000 msgr2=0x7f67d4143e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67d2d9d700 1 --2- 172.21.15.7:0/1432801110 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f67d4155000 0x7f67d4143e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67d2d9d700 1 -- 172.21.15.7:0/1432801110 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67d41e7850 msgr2=0x7f67d4154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67d2d9d700 1 --2- 172.21.15.7:0/1432801110 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67d41e7850 0x7f67d4154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67d2d9d700 1 -- 172.21.15.7:0/1432801110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d4153b20 con 0x7f67d41e7470 2024-09-13T18:54:13.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67d2d9d700 1 --2- 172.21.15.7:0/1432801110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67d41e7470 0x7f67d4155e90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f67c4016000 tx=0x7f67c40047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:13.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67cbfff700 1 -- 172.21.15.7:0/1432801110 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c400e6c0 con 0x7f67d41e7470 2024-09-13T18:54:13.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67cbfff700 1 -- 172.21.15.7:0/1432801110 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c4003ad0 con 0x7f67d41e7470 2024-09-13T18:54:13.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67cbfff700 1 -- 172.21.15.7:0/1432801110 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c4003e60 con 0x7f67d41e7470 2024-09-13T18:54:13.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67d8817700 1 -- 172.21.15.7:0/1432801110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d4153e00 con 0x7f67d41e7470 2024-09-13T18:54:13.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.571+0000 7f67d8817700 1 -- 172.21.15.7:0/1432801110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d4144390 con 0x7f67d41e7470 2024-09-13T18:54:13.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.575+0000 7f67d8817700 1 -- 172.21.15.7:0/1432801110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d4144580 con 0x7f67d41e7470 2024-09-13T18:54:13.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.575+0000 7f67cbfff700 1 -- 172.21.15.7:0/1432801110 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f67c4013470 con 0x7f67d41e7470 2024-09-13T18:54:13.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.575+0000 7f67cbfff700 1 --2- 172.21.15.7:0/1432801110 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f67b4066760 0x7f67b4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.575+0000 7f67cbfff700 1 -- 172.21.15.7:0/1432801110 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67c4085a80 con 0x7f67d41e7470 2024-09-13T18:54:13.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.575+0000 7f67ca59c700 1 --2- 172.21.15.7:0/1432801110 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f67b4066760 0x7f67b4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.579+0000 7f67ca59c700 1 --2- 172.21.15.7:0/1432801110 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f67b4066760 0x7f67b4068c20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f67d40b3da0 tx=0x7f67bc009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:13.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.583+0000 7f67cbfff700 1 -- 172.21.15.7:0/1432801110 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f67c4056a50 con 0x7f67d41e7470 2024-09-13T18:54:13.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.815+0000 7f67d8817700 1 -- 172.21.15.7:0/1432801110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f67d4153310 con 0x7f67d41e7470 2024-09-13T18:54:13.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.823+0000 7f67cbfff700 1 -- 172.21.15.7:0/1432801110 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f67c40565e0 con 0x7f67d41e7470 2024-09-13T18:54:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.823+0000 7f67d8817700 1 -- 172.21.15.7:0/1432801110 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f67b4066760 msgr2=0x7f67b4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.823+0000 7f67d8817700 1 --2- 172.21.15.7:0/1432801110 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f67b4066760 0x7f67b4068c20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f67d40b3da0 tx=0x7f67bc009240 comp rx=0 tx=0).stop 2024-09-13T18:54:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.823+0000 7f67d8817700 1 -- 172.21.15.7:0/1432801110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67d41e7470 msgr2=0x7f67d4155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.823+0000 7f67d8817700 1 --2- 172.21.15.7:0/1432801110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67d41e7470 0x7f67d4155e90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f67c4016000 tx=0x7f67c40047e0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.823+0000 7f67d8817700 1 -- 172.21.15.7:0/1432801110 shutdown_connections 2024-09-13T18:54:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.823+0000 7f67d8817700 1 --2- 172.21.15.7:0/1432801110 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f67b4066760 0x7f67b4068c20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.823+0000 7f67d8817700 1 --2- 172.21.15.7:0/1432801110 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f67d41e7850 0x7f67d4154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.823+0000 7f67d8817700 1 --2- 172.21.15.7:0/1432801110 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f67d4155000 0x7f67d4143e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.823+0000 7f67d8817700 1 --2- 172.21.15.7:0/1432801110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67d41e7470 0x7f67d4155e90 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.823+0000 7f67d8817700 1 -- 172.21.15.7:0/1432801110 >> 172.21.15.7:0/1432801110 conn(0x7f67d40cf9a0 msgr2=0x7f67d40af3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:13.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.823+0000 7f67d8817700 1 -- 172.21.15.7:0/1432801110 shutdown_connections 2024-09-13T18:54:13.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.823+0000 7f67d8817700 1 -- 172.21.15.7:0/1432801110 wait complete. 2024-09-13T18:54:13.834 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-09-13T18:54:13.848 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:13.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda736c700 1 Processor -- start 2024-09-13T18:54:13.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda736c700 1 -- start start 2024-09-13T18:54:13.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda736c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd98054bb0 0x7fbd98053240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda736c700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fbd98053780 0x7fbd980e24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda736c700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd98053b90 con 0x7fbd980093d0 2024-09-13T18:54:13.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda736c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd980097b0 con 0x7fbd98054bb0 2024-09-13T18:54:13.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda736c700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd98055070 con 0x7fbd98053780 2024-09-13T18:54:13.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda5b69700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd98054bb0 0x7fbd98053240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda5b69700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd98054bb0 0x7fbd98053240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55468/0 (socket says 172.21.15.7:55468) 2024-09-13T18:54:13.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda5b69700 1 -- 172.21.15.7:0/1646871467 learned_addr learned my addr 172.21.15.7:0/1646871467 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:13.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda6b6b700 1 --2- 172.21.15.7:0/1646871467 >> v2:172.21.15.171:3300/0 conn(0x7fbd98053780 0x7fbd980e24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda5b69700 1 -- 172.21.15.7:0/1646871467 >> v2:172.21.15.171:3300/0 conn(0x7fbd98053780 msgr2=0x7fbd980e24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:13.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda5b69700 1 --2- 172.21.15.7:0/1646871467 >> v2:172.21.15.171:3300/0 conn(0x7fbd98053780 0x7fbd980e24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda5b69700 1 -- 172.21.15.7:0/1646871467 >> v1:172.21.15.171:6789/0 conn(0x7fbd980093d0 legacy=0x7fbd9800cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda5b69700 1 -- 172.21.15.7:0/1646871467 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd98055350 con 0x7fbd98054bb0 2024-09-13T18:54:13.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda5b69700 1 --2- 172.21.15.7:0/1646871467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd98054bb0 0x7fbd98053240 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fbd980e7870 tx=0x7fbd9c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6930f295744a53dc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:13.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda5368700 1 -- 172.21.15.7:0/1646871467 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c011830 con 0x7fbd98054bb0 2024-09-13T18:54:13.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda5368700 1 -- 172.21.15.7:0/1646871467 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c011990 con 0x7fbd98054bb0 2024-09-13T18:54:13.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda5368700 1 -- 172.21.15.7:0/1646871467 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c011d20 con 0x7fbd98054bb0 2024-09-13T18:54:13.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda736c700 1 -- 172.21.15.7:0/1646871467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd98054bb0 msgr2=0x7fbd98053240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda736c700 1 --2- 172.21.15.7:0/1646871467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd98054bb0 0x7fbd98053240 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fbd980e7870 tx=0x7fbd9c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda736c700 1 -- 172.21.15.7:0/1646871467 shutdown_connections 2024-09-13T18:54:13.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda736c700 1 --2- 172.21.15.7:0/1646871467 >> v2:172.21.15.171:3300/0 conn(0x7fbd98053780 0x7fbd980e24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda736c700 1 --2- 172.21.15.7:0/1646871467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd98054bb0 0x7fbd98053240 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda736c700 1 -- 172.21.15.7:0/1646871467 >> 172.21.15.7:0/1646871467 conn(0x7fbd9801fbf0 msgr2=0x7fbd98056860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:13.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda736c700 1 -- 172.21.15.7:0/1646871467 shutdown_connections 2024-09-13T18:54:13.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.959+0000 7fbda736c700 1 -- 172.21.15.7:0/1646871467 wait complete. 2024-09-13T18:54:13.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda736c700 1 Processor -- start 2024-09-13T18:54:13.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda736c700 1 -- start start 2024-09-13T18:54:13.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda736c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbd980093d0 0x7fbd9803c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda736c700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbd98053780 0x7fbd980381e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda736c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd98054bb0 0x7fbd980e6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda736c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd98038c80 con 0x7fbd98054bb0 2024-09-13T18:54:13.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda736c700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd980396b0 con 0x7fbd980093d0 2024-09-13T18:54:13.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda736c700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd98039990 con 0x7fbd98053780 2024-09-13T18:54:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda6b6b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd98054bb0 0x7fbd980e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda6b6b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd98054bb0 0x7fbd980e6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55474/0 (socket says 172.21.15.7:55474) 2024-09-13T18:54:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda6b6b700 1 -- 172.21.15.7:0/1938555188 learned_addr learned my addr 172.21.15.7:0/1938555188 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda5b69700 1 --2- 172.21.15.7:0/1938555188 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbd98053780 0x7fbd980381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda636a700 1 --2- 172.21.15.7:0/1938555188 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbd980093d0 0x7fbd9803c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda6b6b700 1 -- 172.21.15.7:0/1938555188 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbd98053780 msgr2=0x7fbd980381e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda6b6b700 1 --2- 172.21.15.7:0/1938555188 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbd98053780 0x7fbd980381e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda6b6b700 1 -- 172.21.15.7:0/1938555188 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbd980093d0 msgr2=0x7fbd9803c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda6b6b700 1 --2- 172.21.15.7:0/1938555188 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbd980093d0 0x7fbd9803c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda6b6b700 1 -- 172.21.15.7:0/1938555188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c009570 con 0x7fbd98054bb0 2024-09-13T18:54:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.963+0000 7fbda6b6b700 1 --2- 172.21.15.7:0/1938555188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd98054bb0 0x7fbd980e6760 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbda0067980 tx=0x7fbda00733d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.967+0000 7fbd96ffd700 1 -- 172.21.15.7:0/1938555188 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda008f070 con 0x7fbd98054bb0 2024-09-13T18:54:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.967+0000 7fbd96ffd700 1 -- 172.21.15.7:0/1938555188 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda0073ed0 con 0x7fbd98054bb0 2024-09-13T18:54:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.967+0000 7fbd96ffd700 1 -- 172.21.15.7:0/1938555188 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda0067be0 con 0x7fbd98054bb0 2024-09-13T18:54:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.967+0000 7fbda736c700 1 -- 172.21.15.7:0/1938555188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd98039c70 con 0x7fbd98054bb0 2024-09-13T18:54:13.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.967+0000 7fbda736c700 1 -- 172.21.15.7:0/1938555188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9803c5b0 con 0x7fbd98054bb0 2024-09-13T18:54:13.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.967+0000 7fbda736c700 1 -- 172.21.15.7:0/1938555188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9803a190 con 0x7fbd98054bb0 2024-09-13T18:54:13.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.967+0000 7fbd96ffd700 1 -- 172.21.15.7:0/1938555188 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbda0067d40 con 0x7fbd98054bb0 2024-09-13T18:54:13.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.967+0000 7fbd96ffd700 1 --2- 172.21.15.7:0/1938555188 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbd84066730 0x7fbd84068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:13.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.967+0000 7fbd96ffd700 1 -- 172.21.15.7:0/1938555188 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbda00fd940 con 0x7fbd98054bb0 2024-09-13T18:54:13.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.971+0000 7fbda636a700 1 --2- 172.21.15.7:0/1938555188 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbd84066730 0x7fbd84068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:13.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.971+0000 7fbda636a700 1 --2- 172.21.15.7:0/1938555188 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbd84066730 0x7fbd84068bf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbd98050a30 tx=0x7fbd90009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:13.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:13.975+0000 7fbd96ffd700 1 -- 172.21.15.7:0/1938555188 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbda00cacb0 con 0x7fbd98054bb0 2024-09-13T18:54:14.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.183+0000 7fbda736c700 1 -- 172.21.15.7:0/1938555188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd98053240 con 0x7fbd98054bb0 2024-09-13T18:54:14.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.187+0000 7fbd96ffd700 1 -- 172.21.15.7:0/1938555188 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fbda0099090 con 0x7fbd98054bb0 2024-09-13T18:54:14.195 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:14.195 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":30,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5249,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6836/1527871720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1527871720},{"type":"v1","addr":"172.21.15.171:6836","nonce":1527871720}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5252,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/4126503064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6835","nonce":4126503064},{"type":"v1","addr":"172.21.15.171:6837","nonce":4126503064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":5629,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1132382369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1132382369},{"type":"v1","addr":"172.21.15.7:6833","nonce":1132382369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:12.030791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5721":{"gid":5721,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3168373873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3168373873},{"type":"v1","addr":"172.21.15.7:6835","nonce":3168373873}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:14.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.187+0000 7fbda736c700 1 -- 172.21.15.7:0/1938555188 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbd84066730 msgr2=0x7fbd84068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.187+0000 7fbda736c700 1 --2- 172.21.15.7:0/1938555188 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbd84066730 0x7fbd84068bf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbd98050a30 tx=0x7fbd90009040 comp rx=0 tx=0).stop 2024-09-13T18:54:14.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.187+0000 7fbda736c700 1 -- 172.21.15.7:0/1938555188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd98054bb0 msgr2=0x7fbd980e6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.187+0000 7fbda736c700 1 --2- 172.21.15.7:0/1938555188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd98054bb0 0x7fbd980e6760 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fbda0067980 tx=0x7fbda00733d0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.187+0000 7fbda736c700 1 -- 172.21.15.7:0/1938555188 shutdown_connections 2024-09-13T18:54:14.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.187+0000 7fbda736c700 1 --2- 172.21.15.7:0/1938555188 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbd84066730 0x7fbd84068bf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.187+0000 7fbda736c700 1 --2- 172.21.15.7:0/1938555188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd98054bb0 0x7fbd980e6760 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.187+0000 7fbda736c700 1 --2- 172.21.15.7:0/1938555188 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbd98053780 0x7fbd980381e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.187+0000 7fbda736c700 1 --2- 172.21.15.7:0/1938555188 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbd980093d0 0x7fbd9803c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.187+0000 7fbda736c700 1 -- 172.21.15.7:0/1938555188 >> 172.21.15.7:0/1938555188 conn(0x7fbd9801fbf0 msgr2=0x7fbd98007c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:14.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.187+0000 7fbda736c700 1 -- 172.21.15.7:0/1938555188 shutdown_connections 2024-09-13T18:54:14.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.187+0000 7fbda736c700 1 -- 172.21.15.7:0/1938555188 wait complete. 2024-09-13T18:54:14.198 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2024-09-13T18:54:14.613 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T18:54:09.004150+0000', 'modified': '2024-09-13T18:54:12.030791+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5721}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5721': {'gid': 5721, 'name': 'c', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/3168373873', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 3168373873}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 3168373873}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T18:54:14.613 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T18:54:14.613 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-13T18:54:14.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e2ab92700 1 Processor -- start 2024-09-13T18:54:14.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e2ab92700 1 -- start start 2024-09-13T18:54:14.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e2ab92700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e2415cda0 0x7f7e240d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e2ab92700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7e240b5e00 0x7f7e240b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e2ab92700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e240d0880 con 0x7f7e240b6780 2024-09-13T18:54:14.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e2ab92700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e240cde50 con 0x7f7e240b5e00 2024-09-13T18:54:14.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e2ab92700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e240d16e0 con 0x7f7e2415cda0 2024-09-13T18:54:14.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e23fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7e240b5e00 0x7f7e240b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e23fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7e240b5e00 0x7f7e240b6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55488/0 (socket says 172.21.15.7:55488) 2024-09-13T18:54:14.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e23fff700 1 -- 172.21.15.7:0/3598251880 learned_addr learned my addr 172.21.15.7:0/3598251880 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:14.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e2912f700 1 --1- 172.21.15.7:0/3598251880 >> v1:172.21.15.7:6789/0 conn(0x7f7e240b6780 0x7f7e2406a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36702/0 (socket says 172.21.15.7:36702) 2024-09-13T18:54:14.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e2892e700 1 --2- 172.21.15.7:0/3598251880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e2415cda0 0x7f7e240d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e23fff700 1 -- 172.21.15.7:0/3598251880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e2415cda0 msgr2=0x7f7e240d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:14.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e23fff700 1 --2- 172.21.15.7:0/3598251880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e2415cda0 0x7f7e240d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e23fff700 1 -- 172.21.15.7:0/3598251880 >> v1:172.21.15.7:6789/0 conn(0x7f7e240b6780 legacy=0x7f7e2406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e237fe700 1 -- 172.21.15.7:0/3598251880 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714269040 0 0) 0x7f7e240d0880 con 0x7f7e240b6780 2024-09-13T18:54:14.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e23fff700 1 -- 172.21.15.7:0/3598251880 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e240d0ad0 con 0x7f7e240b5e00 2024-09-13T18:54:14.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e23fff700 1 --2- 172.21.15.7:0/3598251880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e240b5e00 0x7f7e240b6240 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009940 tx=0x7f7e18009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fbbecb0b697025 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:14.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e237fe700 1 -- 172.21.15.7:0/3598251880 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1800ba20 con 0x7f7e240b5e00 2024-09-13T18:54:14.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e237fe700 1 -- 172.21.15.7:0/3598251880 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e18009e90 con 0x7f7e240b5e00 2024-09-13T18:54:14.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e237fe700 1 -- 172.21.15.7:0/3598251880 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1800bda0 con 0x7f7e240b5e00 2024-09-13T18:54:14.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e2ab92700 1 -- 172.21.15.7:0/3598251880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e240b5e00 msgr2=0x7f7e240b6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e2ab92700 1 --2- 172.21.15.7:0/3598251880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e240b5e00 0x7f7e240b6240 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009940 tx=0x7f7e18009c50 comp rx=0 tx=0).stop 2024-09-13T18:54:14.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e2ab92700 1 -- 172.21.15.7:0/3598251880 shutdown_connections 2024-09-13T18:54:14.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e2ab92700 1 --2- 172.21.15.7:0/3598251880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e240b5e00 0x7f7e240b6240 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e2ab92700 1 --2- 172.21.15.7:0/3598251880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e2415cda0 0x7f7e240d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e2ab92700 1 -- 172.21.15.7:0/3598251880 >> 172.21.15.7:0/3598251880 conn(0x7f7e240d2cb0 msgr2=0x7f7e240d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:14.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e2ab92700 1 -- 172.21.15.7:0/3598251880 shutdown_connections 2024-09-13T18:54:14.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.739+0000 7f7e2ab92700 1 -- 172.21.15.7:0/3598251880 wait complete. 2024-09-13T18:54:14.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e2ab92700 1 Processor -- start 2024-09-13T18:54:14.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e2ab92700 1 -- start start 2024-09-13T18:54:14.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e2ab92700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e240b5e00 0x7f7e2415db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e2ab92700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e240b6780 0x7f7e2415f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e2ab92700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e2415cda0 0x7f7e241521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e2ab92700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e241f36c0 con 0x7f7e240b6780 2024-09-13T18:54:14.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e2ab92700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2415d320 con 0x7f7e2415cda0 2024-09-13T18:54:14.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e2ab92700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2415d600 con 0x7f7e240b5e00 2024-09-13T18:54:14.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e23fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e240b6780 0x7f7e2415f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e2892e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e240b5e00 0x7f7e2415db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e23fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e240b6780 0x7f7e2415f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55492/0 (socket says 172.21.15.7:55492) 2024-09-13T18:54:14.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e23fff700 1 -- 172.21.15.7:0/1797544281 learned_addr learned my addr 172.21.15.7:0/1797544281 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:14.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e2912f700 1 --2- 172.21.15.7:0/1797544281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e2415cda0 0x7f7e241521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e23fff700 1 -- 172.21.15.7:0/1797544281 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e240b5e00 msgr2=0x7f7e2415db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e23fff700 1 --2- 172.21.15.7:0/1797544281 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e240b5e00 0x7f7e2415db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e23fff700 1 -- 172.21.15.7:0/1797544281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e2415cda0 msgr2=0x7f7e241521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e23fff700 1 --2- 172.21.15.7:0/1797544281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e2415cda0 0x7f7e241521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e23fff700 1 -- 172.21.15.7:0/1797544281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e18009570 con 0x7f7e240b6780 2024-09-13T18:54:14.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.743+0000 7f7e23fff700 1 --2- 172.21.15.7:0/1797544281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e240b6780 0x7f7e2415f8d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7e18013000 tx=0x7f7e18004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:14.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.747+0000 7f7e217fa700 1 -- 172.21.15.7:0/1797544281 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1801a900 con 0x7f7e240b6780 2024-09-13T18:54:14.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.747+0000 7f7e217fa700 1 -- 172.21.15.7:0/1797544281 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1801f070 con 0x7f7e240b6780 2024-09-13T18:54:14.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.747+0000 7f7e217fa700 1 -- 172.21.15.7:0/1797544281 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1801ace0 con 0x7f7e240b6780 2024-09-13T18:54:14.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.747+0000 7f7e2ab92700 1 -- 172.21.15.7:0/1797544281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e24154920 con 0x7f7e240b6780 2024-09-13T18:54:14.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.747+0000 7f7e2ab92700 1 -- 172.21.15.7:0/1797544281 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2415fe10 con 0x7f7e240b6780 2024-09-13T18:54:14.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.747+0000 7f7e217fa700 1 -- 172.21.15.7:0/1797544281 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7e1801b430 con 0x7f7e240b6780 2024-09-13T18:54:14.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.747+0000 7f7e217fa700 1 --2- 172.21.15.7:0/1797544281 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7e10066730 0x7f7e10068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e217fa700 1 -- 172.21.15.7:0/1797544281 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e18021070 con 0x7f7e240b6780 2024-09-13T18:54:14.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e2892e700 1 --2- 172.21.15.7:0/1797544281 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7e10066730 0x7f7e10068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e2892e700 1 --2- 172.21.15.7:0/1797544281 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7e10066730 0x7f7e10068bf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7e14007a60 tx=0x7f7e14008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:14.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e2de43740 1 Processor -- start 2024-09-13T18:54:14.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e2de43740 1 -- start start 2024-09-13T18:54:14.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e2de43740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf42da0 0xf43180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e2de43740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf21d00 0xf0b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e2de43740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf436c0 0xf0bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e2de43740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0xe6ad80 con 0xf21d00 2024-09-13T18:54:14.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e2de43740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0xe68510 con 0xf436c0 2024-09-13T18:54:14.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e2de43740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0xe63e00 con 0xf42da0 2024-09-13T18:54:14.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e23fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf21d00 0xf0b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e23fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf21d00 0xf0b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55496/0 (socket says 172.21.15.7:55496) 2024-09-13T18:54:14.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e23fff700 1 -- 172.21.15.7:0/1445376208 learned_addr learned my addr 172.21.15.7:0/1445376208 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:14.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e2892e700 1 --2- 172.21.15.7:0/1445376208 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf42da0 0xf43180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e2912f700 1 --2- 172.21.15.7:0/1445376208 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf436c0 0xf0bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e23fff700 1 -- 172.21.15.7:0/1445376208 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf42da0 msgr2=0xf43180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:14.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e23fff700 1 --2- 172.21.15.7:0/1445376208 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf42da0 0xf43180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e23fff700 1 -- 172.21.15.7:0/1445376208 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf436c0 msgr2=0xf0bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:14.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e23fff700 1 --2- 172.21.15.7:0/1445376208 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf436c0 0xf0bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.751+0000 7f7e23fff700 1 -- 172.21.15.7:0/1445376208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xcd6960 con 0xf21d00 2024-09-13T18:54:14.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e23fff700 1 --2- 172.21.15.7:0/1445376208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf21d00 0xf0b890 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7e18056900 tx=0x7f7e1800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66da2424db8ed1cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:14.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e0a7fc700 1 -- 172.21.15.7:0/1445376208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e1806e070 con 0xf21d00 2024-09-13T18:54:14.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e0a7fc700 1 -- 172.21.15.7:0/1445376208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1801ae40 con 0xf21d00 2024-09-13T18:54:14.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e0a7fc700 1 -- 172.21.15.7:0/1445376208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18057070 con 0xf21d00 2024-09-13T18:54:14.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e2de43740 1 -- 172.21.15.7:0/1445376208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf21d00 msgr2=0xf0b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e2de43740 1 --2- 172.21.15.7:0/1445376208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf21d00 0xf0b890 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7e18056900 tx=0x7f7e1800bfd0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e2de43740 1 -- 172.21.15.7:0/1445376208 shutdown_connections 2024-09-13T18:54:14.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e2de43740 1 --2- 172.21.15.7:0/1445376208 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf436c0 0xf0bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e2de43740 1 --2- 172.21.15.7:0/1445376208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf21d00 0xf0b890 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e2de43740 1 --2- 172.21.15.7:0/1445376208 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf42da0 0xf43180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e2de43740 1 -- 172.21.15.7:0/1445376208 >> 172.21.15.7:0/1445376208 conn(0xc8b4d0 msgr2=0xf24c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:14.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e2de43740 1 -- 172.21.15.7:0/1445376208 shutdown_connections 2024-09-13T18:54:14.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e2de43740 1 -- 172.21.15.7:0/1445376208 wait complete. 2024-09-13T18:54:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e2de43740 1 -- start start 2024-09-13T18:54:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e2de43740 1 Processor -- start 2024-09-13T18:54:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e2de43740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf21d00 0xf1d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e2de43740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf42da0 0xf4cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e2de43740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf436c0 0xf4d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e2de43740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0xe21730 con 0xf42da0 2024-09-13T18:54:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e2de43740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0xe275b0 con 0xf436c0 2024-09-13T18:54:14.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e2de43740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0xd0faa0 con 0xf21d00 2024-09-13T18:54:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.755+0000 7f7e2de43740 10 client.0 _ll_register_callbacks cb 0xe5d610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:54:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e23fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf42da0 0xf4cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e23fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf42da0 0xf4cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55512/0 (socket says 172.21.15.7:55512) 2024-09-13T18:54:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e23fff700 1 -- 172.21.15.7:0/154746380 learned_addr learned my addr 172.21.15.7:0/154746380 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e2892e700 1 --2- 172.21.15.7:0/154746380 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf21d00 0xf1d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e2912f700 1 --2- 172.21.15.7:0/154746380 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf436c0 0xf4d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e23fff700 1 -- 172.21.15.7:0/154746380 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf21d00 msgr2=0xf1d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e23fff700 1 --2- 172.21.15.7:0/154746380 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf21d00 0xf1d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e23fff700 1 -- 172.21.15.7:0/154746380 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf436c0 msgr2=0xf4d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e23fff700 1 --2- 172.21.15.7:0/154746380 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf436c0 0xf4d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e23fff700 1 -- 172.21.15.7:0/154746380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2415fe10 con 0xf42da0 2024-09-13T18:54:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e037fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-13T18:54:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e23fff700 1 --2- 172.21.15.7:0/154746380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf42da0 0xf4cf90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7e18056fd0 tx=0x7f7e18009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6ad841978fce20 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:14.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e037fe700 1 -- 172.21.15.7:0/154746380 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18072070 con 0xf42da0 2024-09-13T18:54:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e037fe700 1 -- 172.21.15.7:0/154746380 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e18057070 con 0xf42da0 2024-09-13T18:54:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e2de43740 1 -- 172.21.15.7:0/154746380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0xc981e0 con 0xf42da0 2024-09-13T18:54:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e037fe700 1 -- 172.21.15.7:0/154746380 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e1806e040 con 0xf42da0 2024-09-13T18:54:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e2de43740 10 client.5865 fetch_fsmap learned FSMap version 30 2024-09-13T18:54:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e2de43740 1 -- 172.21.15.7:0/154746380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0xe2a220 con 0xf42da0 2024-09-13T18:54:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e037fe700 1 -- 172.21.15.7:0/154746380 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0xc981e0 con 0xf42da0 2024-09-13T18:54:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e037fe700 1 -- 172.21.15.7:0/154746380 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7e1806e1a0 con 0xf42da0 2024-09-13T18:54:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e2de43740 10 client.5865 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:54:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e2de43740 10 client.5865 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T18:54:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e2de43740 20 client.5865 populate_metadata read hostname 'smithi007' 2024-09-13T18:54:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e2de43740 1 --2- 172.21.15.7:0/154746380 >> [v2:172.21.15.7:6834/3168373873,v1:172.21.15.7:6835/3168373873] conn(0xef7820 0xef7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e2de43740 4 client.5865 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:54:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e2de43740 1 -- 172.21.15.7:0/154746380 --> [v2:172.21.15.7:6834/3168373873,v1:172.21.15.7:6835/3168373873] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0xce2460 con 0xef7820 2024-09-13T18:54:14.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e2892e700 1 --2- 172.21.15.7:0/154746380 >> [v2:172.21.15.7:6834/3168373873,v1:172.21.15.7:6835/3168373873] conn(0xef7820 0xef7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e2892e700 1 --2- 172.21.15.7:0/154746380 >> [v2:172.21.15.7:6834/3168373873,v1:172.21.15.7:6835/3168373873] conn(0xef7820 0xef7c00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76cc6daa7337b4e3 server_cookie=f467ec90d41c61c0 in_seq=0 out_seq=0 2024-09-13T18:54:14.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e037fe700 10 client.5865 ms_handle_connect on v2:172.21.15.7:6834/3168373873 2024-09-13T18:54:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e037fe700 1 -- 172.21.15.7:0/154746380 <== mds.0 v2:172.21.15.7:6834/3168373873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0xce2460 con 0xef7820 2024-09-13T18:54:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e037fe700 10 client.5865 handle_command_reply: tid=0 2024-09-13T18:54:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.759+0000 7f7e2de43740 1 client.5865 shutdown 2024-09-13T18:54:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.763+0000 7f7e2de43740 1 -- 172.21.15.7:0/154746380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf42da0 msgr2=0xf4cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.763+0000 7f7e2de43740 1 --2- 172.21.15.7:0/154746380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf42da0 0xf4cf90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7e18056fd0 tx=0x7f7e18009c50 comp rx=0 tx=0).stop 2024-09-13T18:54:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.763+0000 7f7e2de43740 1 -- 172.21.15.7:0/154746380 shutdown_connections 2024-09-13T18:54:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.763+0000 7f7e2de43740 1 --2- 172.21.15.7:0/154746380 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf436c0 0xf4d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.763+0000 7f7e2de43740 1 --2- 172.21.15.7:0/154746380 >> [v2:172.21.15.7:6834/3168373873,v1:172.21.15.7:6835/3168373873] conn(0xef7820 0xef7c00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.763+0000 7f7e2de43740 1 --2- 172.21.15.7:0/154746380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf42da0 0xf4cf90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.763+0000 7f7e2de43740 1 --2- 172.21.15.7:0/154746380 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf21d00 0xf1d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.763+0000 7f7e037fe700 0 client.5865 ms_handle_reset on v2:172.21.15.7:6834/3168373873 2024-09-13T18:54:14.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.763+0000 7f7e2de43740 1 -- 172.21.15.7:0/154746380 >> 172.21.15.7:0/154746380 conn(0xc8b4d0 msgr2=0xf1e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:14.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.763+0000 7f7e2de43740 1 -- 172.21.15.7:0/154746380 shutdown_connections 2024-09-13T18:54:14.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.763+0000 7f7e2de43740 1 -- 172.21.15.7:0/154746380 wait complete. 2024-09-13T18:54:14.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.763+0000 7f7e2de43740 20 client.5865 trim_cache size 0 max 16384 2024-09-13T18:54:14.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.779+0000 7f7e2de43740 1 Processor -- start 2024-09-13T18:54:14.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.779+0000 7f7e2de43740 1 -- start start 2024-09-13T18:54:14.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.779+0000 7f7e2de43740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf21d00 0xf4caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.779+0000 7f7e2de43740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf42da0 0xf4d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.779+0000 7f7e2de43740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf436c0 0xf4ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.779+0000 7f7e2de43740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0xe6ad80 con 0xf42da0 2024-09-13T18:54:14.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.779+0000 7f7e2de43740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0xe02470 con 0xf21d00 2024-09-13T18:54:14.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.779+0000 7f7e2de43740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0xeba580 con 0xf436c0 2024-09-13T18:54:14.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e23fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf42da0 0xf4d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e23fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf42da0 0xf4d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55526/0 (socket says 172.21.15.7:55526) 2024-09-13T18:54:14.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e23fff700 1 -- 172.21.15.7:0/1685308907 learned_addr learned my addr 172.21.15.7:0/1685308907 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:14.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2892e700 1 --2- 172.21.15.7:0/1685308907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf21d00 0xf4caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2912f700 1 --2- 172.21.15.7:0/1685308907 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf436c0 0xf4ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e23fff700 1 -- 172.21.15.7:0/1685308907 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf436c0 msgr2=0xf4ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e23fff700 1 --2- 172.21.15.7:0/1685308907 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf436c0 0xf4ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e23fff700 1 -- 172.21.15.7:0/1685308907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf21d00 msgr2=0xf4caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e23fff700 1 --2- 172.21.15.7:0/1685308907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf21d00 0xf4caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e23fff700 1 -- 172.21.15.7:0/1685308907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0xe10450 con 0xf42da0 2024-09-13T18:54:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e23fff700 1 --2- 172.21.15.7:0/1685308907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf42da0 0xf4d030 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800b3c0 tx=0x7f7e1807f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82fa7c820a20d579 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e0affd700 1 -- 172.21.15.7:0/1685308907 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18072040 con 0xf42da0 2024-09-13T18:54:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e0affd700 1 -- 172.21.15.7:0/1685308907 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e18081070 con 0xf42da0 2024-09-13T18:54:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e0affd700 1 -- 172.21.15.7:0/1685308907 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e18057070 con 0xf42da0 2024-09-13T18:54:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2de43740 1 -- 172.21.15.7:0/1685308907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf42da0 msgr2=0xf4d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2de43740 1 --2- 172.21.15.7:0/1685308907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf42da0 0xf4d030 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800b3c0 tx=0x7f7e1807f070 comp rx=0 tx=0).stop 2024-09-13T18:54:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2de43740 1 -- 172.21.15.7:0/1685308907 shutdown_connections 2024-09-13T18:54:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2de43740 1 --2- 172.21.15.7:0/1685308907 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf436c0 0xf4ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2de43740 1 --2- 172.21.15.7:0/1685308907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf42da0 0xf4d030 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2de43740 1 --2- 172.21.15.7:0/1685308907 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf21d00 0xf4caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2de43740 1 -- 172.21.15.7:0/1685308907 >> 172.21.15.7:0/1685308907 conn(0xc8b4d0 msgr2=0xf20950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:14.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2de43740 1 -- 172.21.15.7:0/1685308907 shutdown_connections 2024-09-13T18:54:14.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2de43740 1 -- 172.21.15.7:0/1685308907 wait complete. 2024-09-13T18:54:14.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2de43740 1 -- start start 2024-09-13T18:54:14.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2de43740 1 Processor -- start 2024-09-13T18:54:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2de43740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf21d00 0xfa39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2de43740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf42da0 0xfa4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2de43740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf436c0 0xfa4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2de43740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0xe92ec0 con 0xf21d00 2024-09-13T18:54:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2de43740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0xf62210 con 0xf42da0 2024-09-13T18:54:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2de43740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0xe26070 con 0xf436c0 2024-09-13T18:54:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2de43740 10 client.0 _ll_register_callbacks cb 0xeba710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:54:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.783+0000 7f7e2892e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf21d00 0xfa39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2892e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf21d00 0xfa39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55540/0 (socket says 172.21.15.7:55540) 2024-09-13T18:54:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2892e700 1 -- 172.21.15.7:0/2444399810 learned_addr learned my addr 172.21.15.7:0/2444399810 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e23fff700 1 --2- 172.21.15.7:0/2444399810 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf42da0 0xfa4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2912f700 1 --2- 172.21.15.7:0/2444399810 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf436c0 0xfa4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2892e700 1 -- 172.21.15.7:0/2444399810 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf436c0 msgr2=0xfa4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2892e700 1 --2- 172.21.15.7:0/2444399810 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf436c0 0xfa4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2892e700 1 -- 172.21.15.7:0/2444399810 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf42da0 msgr2=0xfa4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2892e700 1 --2- 172.21.15.7:0/2444399810 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf42da0 0xfa4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2892e700 1 -- 172.21.15.7:0/2444399810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0xe2a220 con 0xf21d00 2024-09-13T18:54:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e23fff700 1 --2- 172.21.15.7:0/2444399810 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf42da0 0xfa4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2892e700 1 --2- 172.21.15.7:0/2444399810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf21d00 0xfa39d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7e140106b0 tx=0x7f7e140047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ed11761e771435 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e037fe700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-13T18:54:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e037fe700 1 -- 172.21.15.7:0/2444399810 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e14029040 con 0xf21d00 2024-09-13T18:54:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2de43740 1 -- 172.21.15.7:0/2444399810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e1801ae40 con 0xf21d00 2024-09-13T18:54:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e037fe700 1 -- 172.21.15.7:0/2444399810 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e14005c50 con 0xf21d00 2024-09-13T18:54:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e037fe700 1 -- 172.21.15.7:0/2444399810 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e14005e10 con 0xf21d00 2024-09-13T18:54:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2de43740 10 client.5871 fetch_fsmap learned FSMap version 30 2024-09-13T18:54:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2de43740 1 -- 172.21.15.7:0/2444399810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0xe04720 con 0xf21d00 2024-09-13T18:54:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e037fe700 1 -- 172.21.15.7:0/2444399810 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e1801ae40 con 0xf21d00 2024-09-13T18:54:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e037fe700 1 -- 172.21.15.7:0/2444399810 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 30) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7e14016a40 con 0xf21d00 2024-09-13T18:54:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2de43740 10 client.5871 fetch_fsmap finished waiting for FSMap version 30 2024-09-13T18:54:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2de43740 10 client.5871 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T18:54:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2de43740 20 client.5871 populate_metadata read hostname 'smithi007' 2024-09-13T18:54:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2de43740 1 --2- 172.21.15.7:0/2444399810 >> [v2:172.21.15.7:6834/3168373873,v1:172.21.15.7:6835/3168373873] conn(0xf52250 0xf52630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2de43740 4 client.5871 mds_command: new command op to 5721 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:54:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2de43740 1 -- 172.21.15.7:0/2444399810 --> [v2:172.21.15.7:6834/3168373873,v1:172.21.15.7:6835/3168373873] -- command(tid 0: {"prefix": "status"}) v1 -- 0xd071e0 con 0xf52250 2024-09-13T18:54:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e23fff700 1 --2- 172.21.15.7:0/2444399810 >> [v2:172.21.15.7:6834/3168373873,v1:172.21.15.7:6835/3168373873] conn(0xf52250 0xf52630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e23fff700 1 --2- 172.21.15.7:0/2444399810 >> [v2:172.21.15.7:6834/3168373873,v1:172.21.15.7:6835/3168373873] conn(0xf52250 0xf52630 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8389c6220638fe4c server_cookie=c74ed19c720f09ab in_seq=0 out_seq=0 2024-09-13T18:54:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e037fe700 10 client.5871 ms_handle_connect on v2:172.21.15.7:6834/3168373873 2024-09-13T18:54:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e037fe700 1 -- 172.21.15.7:0/2444399810 <== mds.0 v2:172.21.15.7:6834/3168373873 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0xd071e0 con 0xf52250 2024-09-13T18:54:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e037fe700 10 client.5871 handle_command_reply: tid=0 2024-09-13T18:54:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2de43740 1 client.5871 shutdown 2024-09-13T18:54:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2de43740 1 -- 172.21.15.7:0/2444399810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf21d00 msgr2=0xfa39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2de43740 1 --2- 172.21.15.7:0/2444399810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf21d00 0xfa39d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7e140106b0 tx=0x7f7e140047e0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2de43740 1 -- 172.21.15.7:0/2444399810 shutdown_connections 2024-09-13T18:54:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2de43740 1 --2- 172.21.15.7:0/2444399810 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0xf436c0 0xfa4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2de43740 1 --2- 172.21.15.7:0/2444399810 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0xf42da0 0xfa4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2de43740 1 --2- 172.21.15.7:0/2444399810 >> [v2:172.21.15.7:6834/3168373873,v1:172.21.15.7:6835/3168373873] conn(0xf52250 0xf52630 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2de43740 1 --2- 172.21.15.7:0/2444399810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0xf21d00 0xfa39d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e2de43740 1 -- 172.21.15.7:0/2444399810 >> 172.21.15.7:0/2444399810 conn(0xc8b4d0 msgr2=0xef8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.787+0000 7f7e037fe700 0 client.5871 ms_handle_reset on v2:172.21.15.7:6834/3168373873 2024-09-13T18:54:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.791+0000 7f7e2de43740 1 -- 172.21.15.7:0/2444399810 shutdown_connections 2024-09-13T18:54:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.791+0000 7f7e2de43740 1 -- 172.21.15.7:0/2444399810 wait complete. 2024-09-13T18:54:14.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.791+0000 7f7e2de43740 20 client.5871 trim_cache size 0 max 16384 2024-09-13T18:54:14.802 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:54:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "33406d92-f401-438a-876e-a56de3962d8e", 2024-09-13T18:54:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-13T18:54:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5721, 2024-09-13T18:54:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-13T18:54:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-13T18:54:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-13T18:54:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 5.7868682050000002, 2024-09-13T18:54:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 30, 2024-09-13T18:54:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 52, 2024-09-13T18:54:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 52, 2024-09-13T18:54:14.802 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 10.252018883 2024-09-13T18:54:14.803 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:54:14.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.791+0000 7f7e2ab92700 1 -- 172.21.15.7:0/1797544281 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7e10066730 msgr2=0x7f7e10068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.791+0000 7f7e2ab92700 1 --2- 172.21.15.7:0/1797544281 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7e10066730 0x7f7e10068bf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7e14007a60 tx=0x7f7e14008040 comp rx=0 tx=0).stop 2024-09-13T18:54:14.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.791+0000 7f7e2ab92700 1 -- 172.21.15.7:0/1797544281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e240b6780 msgr2=0x7f7e2415f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.791+0000 7f7e2ab92700 1 --2- 172.21.15.7:0/1797544281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e240b6780 0x7f7e2415f8d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7e18013000 tx=0x7f7e18004770 comp rx=0 tx=0).stop 2024-09-13T18:54:14.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.791+0000 7f7e2ab92700 1 -- 172.21.15.7:0/1797544281 shutdown_connections 2024-09-13T18:54:14.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.791+0000 7f7e2ab92700 1 --2- 172.21.15.7:0/1797544281 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7e10066730 0x7f7e10068bf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.791+0000 7f7e2ab92700 1 --2- 172.21.15.7:0/1797544281 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e2415cda0 0x7f7e241521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.791+0000 7f7e2ab92700 1 --2- 172.21.15.7:0/1797544281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e240b6780 0x7f7e2415f8d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.791+0000 7f7e2ab92700 1 --2- 172.21.15.7:0/1797544281 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7e240b5e00 0x7f7e2415db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.791+0000 7f7e2ab92700 1 -- 172.21.15.7:0/1797544281 >> 172.21.15.7:0/1797544281 conn(0x7f7e240d2cb0 msgr2=0x7f7e240b73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:14.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.791+0000 7f7e2ab92700 1 -- 172.21.15.7:0/1797544281 shutdown_connections 2024-09-13T18:54:14.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.791+0000 7f7e2ab92700 1 -- 172.21.15.7:0/1797544281 wait complete. 2024-09-13T18:54:14.813 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T18:54:14.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.923+0000 7f353246e700 1 Processor -- start 2024-09-13T18:54:14.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.923+0000 7f353246e700 1 -- start start 2024-09-13T18:54:14.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f353246e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f352c06e970 0x7f352c06ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f353246e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f352c0cb5b0 con 0x7f352c06c3d0 2024-09-13T18:54:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f353246e700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f352c0cb890 con 0x7f352c159770 2024-09-13T18:54:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f353246e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352c06b3d0 con 0x7f352c06e970 2024-09-13T18:54:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f3531c6d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f352c06c3d0 0x7f352c070320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36712/0 (socket says 172.21.15.7:36712) 2024-09-13T18:54:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f3531c6d700 1 -- 172.21.15.7:0/2215465614 learned_addr learned my addr 172.21.15.7:0/2215465614 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:14.936 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f1f58042700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T18:54:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f3530c6b700 1 --2- 172.21.15.7:0/2215465614 >> v2:172.21.15.171:3300/0 conn(0x7f352c06e970 0x7f352c06ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f3523fff700 1 -- 172.21.15.7:0/2215465614 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714815680 0 0) 0x7f352c0cb5b0 con 0x7f352c06c3d0 2024-09-13T18:54:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f3523fff700 1 -- 172.21.15.7:0/2215465614 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35180035d0 con 0x7f352c06c3d0 2024-09-13T18:54:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f3523fff700 1 -- 172.21.15.7:0/2215465614 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951668979 0 0) 0x7f352c0cb890 con 0x7f352c159770 2024-09-13T18:54:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f3523fff700 1 -- 172.21.15.7:0/2215465614 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f352c0cb5b0 con 0x7f352c159770 2024-09-13T18:54:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f3530c6b700 1 -- 172.21.15.7:0/2215465614 >> v1:172.21.15.171:6789/0 conn(0x7f352c159770 legacy=0x7f352c06bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f3530c6b700 1 -- 172.21.15.7:0/2215465614 >> v1:172.21.15.7:6789/0 conn(0x7f352c06c3d0 legacy=0x7f352c070320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f3530c6b700 1 -- 172.21.15.7:0/2215465614 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c06a3b0 con 0x7f352c06e970 2024-09-13T18:54:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f3530c6b700 1 --2- 172.21.15.7:0/2215465614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f352c06e970 0x7f352c06ae90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f351c009850 tx=0x7f351c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4e0db4fd93c1bda server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f3523fff700 1 -- 172.21.15.7:0/2215465614 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c004230 con 0x7f352c06e970 2024-09-13T18:54:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f3523fff700 1 -- 172.21.15.7:0/2215465614 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351c004390 con 0x7f352c06e970 2024-09-13T18:54:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.927+0000 7f3523fff700 1 -- 172.21.15.7:0/2215465614 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c0036c0 con 0x7f352c06e970 2024-09-13T18:54:14.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f353246e700 1 -- 172.21.15.7:0/2215465614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f352c06e970 msgr2=0x7f352c06ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f353246e700 1 --2- 172.21.15.7:0/2215465614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f352c06e970 0x7f352c06ae90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f351c009850 tx=0x7f351c009b60 comp rx=0 tx=0).stop 2024-09-13T18:54:14.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f353246e700 1 -- 172.21.15.7:0/2215465614 shutdown_connections 2024-09-13T18:54:14.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f353246e700 1 --2- 172.21.15.7:0/2215465614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f352c06e970 0x7f352c06ae90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f353246e700 1 -- 172.21.15.7:0/2215465614 >> 172.21.15.7:0/2215465614 conn(0x7f352c06d900 msgr2=0x7f352c06dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:14.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f353246e700 1 -- 172.21.15.7:0/2215465614 shutdown_connections 2024-09-13T18:54:14.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f353246e700 1 -- 172.21.15.7:0/2215465614 wait complete. 2024-09-13T18:54:14.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f353246e700 1 Processor -- start 2024-09-13T18:54:14.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f353246e700 1 -- start start 2024-09-13T18:54:14.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f353246e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f352c06c3d0 0x7f352c1f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f353246e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f352c06e970 0x7f352c152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f353246e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f352c159770 0x7f352c1586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f353246e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c1f3240 con 0x7f352c06e970 2024-09-13T18:54:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f353246e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c1f3520 con 0x7f352c159770 2024-09-13T18:54:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f353246e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352c1f3800 con 0x7f352c06c3d0 2024-09-13T18:54:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f3530c6b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f352c06e970 0x7f352c152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f3530c6b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f352c06e970 0x7f352c152410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55552/0 (socket says 172.21.15.7:55552) 2024-09-13T18:54:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f3530c6b700 1 -- 172.21.15.7:0/3059655000 learned_addr learned my addr 172.21.15.7:0/3059655000 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f353146c700 1 --2- 172.21.15.7:0/3059655000 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f352c06c3d0 0x7f352c1f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.931+0000 7f3531c6d700 1 --2- 172.21.15.7:0/3059655000 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f352c159770 0x7f352c1586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.935+0000 7f3530c6b700 1 -- 172.21.15.7:0/3059655000 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f352c06c3d0 msgr2=0x7f352c1f2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.935+0000 7f3530c6b700 1 --2- 172.21.15.7:0/3059655000 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f352c06c3d0 0x7f352c1f2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.935+0000 7f3530c6b700 1 -- 172.21.15.7:0/3059655000 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f352c159770 msgr2=0x7f352c1586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.935+0000 7f3530c6b700 1 --2- 172.21.15.7:0/3059655000 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f352c159770 0x7f352c1586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.935+0000 7f3530c6b700 1 -- 172.21.15.7:0/3059655000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351c0094b0 con 0x7f352c06e970 2024-09-13T18:54:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.935+0000 7f3530c6b700 1 --2- 172.21.15.7:0/3059655000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f352c06e970 0x7f352c152410 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f351c0097a0 tx=0x7f351c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.935+0000 7f3521ffb700 1 -- 172.21.15.7:0/3059655000 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c006ea0 con 0x7f352c06e970 2024-09-13T18:54:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.935+0000 7f3521ffb700 1 -- 172.21.15.7:0/3059655000 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351c01f070 con 0x7f352c06e970 2024-09-13T18:54:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.935+0000 7f353246e700 1 -- 172.21.15.7:0/3059655000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352c156ad0 con 0x7f352c06e970 2024-09-13T18:54:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.935+0000 7f3521ffb700 1 -- 172.21.15.7:0/3059655000 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c003e30 con 0x7f352c06e970 2024-09-13T18:54:14.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.935+0000 7f353246e700 1 -- 172.21.15.7:0/3059655000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352c158c10 con 0x7f352c06e970 2024-09-13T18:54:14.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.935+0000 7f353246e700 1 -- 172.21.15.7:0/3059655000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352c156930 con 0x7f352c06e970 2024-09-13T18:54:14.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.935+0000 7f3521ffb700 1 -- 172.21.15.7:0/3059655000 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f351c021030 con 0x7f352c06e970 2024-09-13T18:54:14.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.935+0000 7f3521ffb700 1 --2- 172.21.15.7:0/3059655000 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3510066730 0x7f3510068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:14.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.939+0000 7f3521ffb700 1 -- 172.21.15.7:0/3059655000 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f351c0855b0 con 0x7f352c06e970 2024-09-13T18:54:14.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.939+0000 7f353146c700 1 --2- 172.21.15.7:0/3059655000 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3510066730 0x7f3510068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:14.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.939+0000 7f353146c700 1 --2- 172.21.15.7:0/3059655000 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3510066730 0x7f3510068bf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f352c06e410 tx=0x7f3528009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:14.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:14.947+0000 7f3521ffb700 1 -- 172.21.15.7:0/3059655000 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f352c156930 con 0x7f352c06e970 2024-09-13T18:54:15.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:15.155+0000 7f353246e700 1 -- 172.21.15.7:0/3059655000 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f352c156930 con 0x7f352c06e970 2024-09-13T18:54:15.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:15.155+0000 7f3521ffb700 1 -- 172.21.15.7:0/3059655000 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) v1 ==== 53+0+73 (secure 0 0 0) 0x7f352c156930 con 0x7f352c06e970 2024-09-13T18:54:15.163 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T18:54:15.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:15.155+0000 7f353246e700 1 -- 172.21.15.7:0/3059655000 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3510066730 msgr2=0x7f3510068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:15.155+0000 7f353246e700 1 --2- 172.21.15.7:0/3059655000 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3510066730 0x7f3510068bf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f352c06e410 tx=0x7f3528009040 comp rx=0 tx=0).stop 2024-09-13T18:54:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:15.155+0000 7f353246e700 1 -- 172.21.15.7:0/3059655000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f352c06e970 msgr2=0x7f352c152410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:15.155+0000 7f353246e700 1 --2- 172.21.15.7:0/3059655000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f352c06e970 0x7f352c152410 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f351c0097a0 tx=0x7f351c004770 comp rx=0 tx=0).stop 2024-09-13T18:54:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:15.155+0000 7f353246e700 1 -- 172.21.15.7:0/3059655000 shutdown_connections 2024-09-13T18:54:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:15.155+0000 7f353246e700 1 --2- 172.21.15.7:0/3059655000 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3510066730 0x7f3510068bf0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:15.155+0000 7f353246e700 1 --2- 172.21.15.7:0/3059655000 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f352c159770 0x7f352c1586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:15.155+0000 7f353246e700 1 --2- 172.21.15.7:0/3059655000 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f352c06e970 0x7f352c152410 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:15.155+0000 7f353246e700 1 --2- 172.21.15.7:0/3059655000 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f352c06c3d0 0x7f352c1f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:15.155+0000 7f353246e700 1 -- 172.21.15.7:0/3059655000 >> 172.21.15.7:0/3059655000 conn(0x7f352c06d900 msgr2=0x7f352c06dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:15.155+0000 7f353246e700 1 -- 172.21.15.7:0/3059655000 shutdown_connections 2024-09-13T18:54:15.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:15.155+0000 7f353246e700 1 -- 172.21.15.7:0/3059655000 wait complete. 2024-09-13T18:54:15.178 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T18:54:15.178 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T18:54:15.178 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-13T18:54:15.178 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T18:54:15.178 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T18:54:15.178 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T18:54:15.179 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T18:54:15.179 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T18:54:15.179 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T18:54:15.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:15.179 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-13T18:54:15.187 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T18:54:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T18:54:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T18:54:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:54:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-13T18:54:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:54:15.187 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:54:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-13T18:54:15.188 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:54:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-13T18:54:15.188 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T18:54:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-13T18:54:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-13T18:54:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42457sec preferred_lft 42457sec 2024-09-13T18:54:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-13T18:54:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:54:15.188 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T18:54:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-13T18:54:15.188 INFO:teuthology.orchestra.run.smithi007.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T18:54:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 02:42:8e:73:06:85 brd ff:ff:ff:ff:ff:ff 2024-09-13T18:54:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T18:54:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:54:15.189 INFO:teuthology.orchestra.run.smithi007.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T18:54:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 72:5b:90:5f:a5:2f brd ff:ff:ff:ff:ff:ff 2024-09-13T18:54:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T18:54:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:54:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::9896:a1ff:fe28:24ca/64 scope link 2024-09-13T18:54:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:54:15.189 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T18:54:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 72:5b:90:5f:a5:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T18:54:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::705b:90ff:fe5f:a52f/64 scope link 2024-09-13T18:54:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:54:15.190 INFO:teuthology.orchestra.run.smithi007.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T18:54:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether ea:d0:e0:a8:33:f2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T18:54:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::e8d0:e0ff:fea8:33f2/64 scope link 2024-09-13T18:54:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-13T18:54:15.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:15.190 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T18:54:15.203 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-13T18:54:15.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:15.205 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-13T18:54:15.255 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-13T18:54:15.255 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T18:54:15.255 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-13T18:54:15.255 INFO:teuthology.orchestra.run.smithi007.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T18:54:15.255 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-13T18:54:15.255 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T18:54:15.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:15.256 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2024-09-13T18:54:15.256 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-13T18:54:15.256 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-13T18:54:15.256 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-13T18:54:15.256 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-13T18:54:15.256 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-13T18:54:15.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:15.298 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-13T18:54:15.303 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-13T18:54:15.304 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T18:54:15.304 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T18:54:15.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:15.304 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-13T18:54:15.315 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-13T18:54:15.315 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:54:15.315 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T18:54:15.364 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T18:54:15.365 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T18:54:15.365 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T18:54:15.371 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-09-13T18:54:15.387 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:54:15.387 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:54:15.445 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:54:15.446 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:54:15.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:15.447 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-09-13T18:54:15.451 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T18:54:15.452 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-13T18:54:15.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-09-13T18:54:15.472 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:54:15.472 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:54:15.530 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:54:15.531 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:54:15.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:15.531 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-09-13T18:54:15.548 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:2024-09-13T18:54:15.539+0000 7fbc94040640 -1 init, newargv = 0x55cee0c8fc50 newargc=15 2024-09-13T18:54:15.548 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[37393]: starting ceph client 2024-09-13T18:54:15.561 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[37393]: starting fuse 2024-09-13T18:54:16.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2024-09-13T18:54:16.655 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T18:54:16.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T18:54:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T18:54:16.718 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T18:54:16.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:16.719 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2024-09-13T18:54:16.726 INFO:teuthology.orchestra.run.smithi007.stdout:52 2024-09-13T18:54:16.726 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2024-09-13T18:54:16.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:54:16.727 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2024-09-13T18:54:16.727 DEBUG:teuthology.orchestra.run.smithi007:> import re 2024-09-13T18:54:16.727 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-13T18:54:16.727 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2024-09-13T18:54:16.727 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:54:16.727 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2024-09-13T18:54:16.727 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T18:54:16.727 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2024-09-13T18:54:16.727 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T18:54:16.727 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:54:16.727 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2024-09-13T18:54:16.727 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2024-09-13T18:54:16.727 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2024-09-13T18:54:16.727 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2024-09-13T18:54:16.728 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:54:16.728 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2024-09-13T18:54:16.728 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T18:54:16.728 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T18:54:16.728 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T18:54:16.728 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2024-09-13T18:54:16.728 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2024-09-13T18:54:16.728 DEBUG:teuthology.orchestra.run.smithi007:> return f 2024-09-13T18:54:16.728 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T18:54:16.728 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:54:16.728 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.0")) 2024-09-13T18:54:16.728 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-13T18:54:16.812 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.0.37393.asok 2024-09-13T18:54:16.995 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.37393.asok 2024-09-13T18:54:16.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:16.995 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.37393.asok status 2024-09-13T18:54:17.137 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:54:17.137 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2024-09-13T18:54:17.137 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T18:54:17.137 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T18:54:17.137 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "0", 2024-09-13T18:54:17.138 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2024-09-13T18:54:17.138 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T18:54:17.138 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "37393", 2024-09-13T18:54:17.138 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2024-09-13T18:54:17.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:17.138 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2024-09-13T18:54:17.138 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2024-09-13T18:54:17.138 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5363, 2024-09-13T18:54:17.138 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2024-09-13T18:54:17.138 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2024-09-13T18:54:17.138 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2024-09-13T18:54:17.138 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 5363 2024-09-13T18:54:17.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:17.138 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-09-13T18:54:17.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-13T18:54:17.139 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-09-13T18:54:17.139 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1993122112 2024-09-13T18:54:17.139 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-13T18:54:17.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:17.139 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2024-09-13T18:54:17.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-13T18:54:17.139 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:0", 2024-09-13T18:54:17.139 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1993122112 2024-09-13T18:54:17.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-13T18:54:17.139 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.5363 172.21.15.7:0/1993122112", 2024-09-13T18:54:17.139 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "172.21.15.7:0/1993122112", 2024-09-13T18:54:17.139 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2024-09-13T18:54:17.139 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 30, 2024-09-13T18:54:17.140 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 52, 2024-09-13T18:54:17.140 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2024-09-13T18:54:17.140 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2024-09-13T18:54:17.140 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2024-09-13T18:54:17.140 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:54:17.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:17.148 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2024-09-13T18:54:17.148 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T18:54:17.154 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2024-09-13T18:54:17.154 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T18:54:17.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:17.154 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T18:54:17.167 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T18:54:17.170 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T18:54:17.200 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-09-13T18:54:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-09-13T18:54:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T18:54:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-09-13T18:54:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-09-13T18:54:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-09-13T18:54:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-09-13T18:54:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-09-13T18:54:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-09-13T18:54:17.201 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-09-13T18:54:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T18:54:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-09-13T18:54:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T18:54:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T18:54:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-09-13T18:54:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-09-13T18:54:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-09-13T18:54:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-09-13T18:54:17.202 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-09-13T18:54:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-09-13T18:54:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-09-13T18:54:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-09-13T18:54:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T18:54:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T18:54:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-09-13T18:54:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T18:54:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-09-13T18:54:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-09-13T18:54:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.203 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-09-13T18:54:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-09-13T18:54:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-09-13T18:54:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.171,[v2:172.21.15.171:3301,v1:172.21.15.171:6790]" 2024-09-13T18:54:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 33406d92-f401-438a-876e-a56de3962d8e 2024-09-13T18:54:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.204 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-09-13T18:54:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-09-13T18:54:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-09-13T18:54:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-09-13T18:54:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-09-13T18:54:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.204 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-09-13T18:54:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-09-13T18:54:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-09-13T18:54:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-09-13T18:54:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-09-13T18:54:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-09-13T18:54:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-09-13T18:54:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-09-13T18:54:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-09-13T18:54:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-09-13T18:54:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-09-13T18:54:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-09-13T18:54:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-09-13T18:54:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-09-13T18:54:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: bdev async discard = True 2024-09-13T18:54:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: bdev enable discard = True 2024-09-13T18:54:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore allocator = bitmap 2024-09-13T18:54:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-09-13T18:54:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-09-13T18:54:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 1/20 2024-09-13T18:54:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 1/20 2024-09-13T18:54:17.206 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:54:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-09-13T18:54:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 4/10 2024-09-13T18:54:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T18:54:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-09-13T18:54:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-09-13T18:54:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2024-09-13T18:54:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2024-09-13T18:54:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-09-13T18:54:17.207 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.207 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-09-13T18:54:17.208 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:54:17.208 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-09-13T18:54:17.208 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-13T18:54:17.208 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-13T18:54:17.208 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:54:17.208 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:54:17.208 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-09-13T18:54:17.208 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-09-13T18:54:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.209 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-09-13T18:54:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:54:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-13T18:54:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-09-13T18:54:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-13T18:54:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-09-13T18:54:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-09-13T18:54:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:54:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-09-13T18:54:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-09-13T18:54:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:54:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.209 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T18:54:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T18:54:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T18:54:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T18:54:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T18:54:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T18:54:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-09-13T18:54:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-09-13T18:54:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-09-13T18:54:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: mon op complaint time = 120 2024-09-13T18:54:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on pool no app = False 2024-09-13T18:54:17.210 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.211 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-09-13T18:54:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-09-13T18:54:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-09-13T18:54:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-09-13T18:54:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T18:54:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T18:54:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 600 2024-09-13T18:54:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-09-13T18:54:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:54:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = "" 2024-09-13T18:54:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-09-13T18:54:17.211 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-09-13T18:54:17.211 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-09-13T18:54:17.212 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-09-13T18:54:17.212 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-09-13T18:54:17.212 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-09-13T18:54:17.212 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-09-13T18:54:17.212 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-09-13T18:54:17.212 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-09-13T18:54:17.212 INFO:teuthology.orchestra.run.smithi007.stdout:[mds] 2024-09-13T18:54:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds = 20 2024-09-13T18:54:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds balancer = 20 2024-09-13T18:54:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:54:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug frag = True 2024-09-13T18:54:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug scatterstat = True 2024-09-13T18:54:17.212 INFO:teuthology.orchestra.run.smithi007.stdout: mds op complaint time = 180 2024-09-13T18:54:17.213 INFO:teuthology.orchestra.run.smithi007.stdout: mds verify scatter = True 2024-09-13T18:54:17.213 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-09-13T18:54:17.213 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-09-13T18:54:17.213 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-09-13T18:54:17.213 INFO:teuthology.orchestra.run.smithi007.stdout: osd_deep_scrub_large_omap_object_key_threshold = 1 2024-09-13T18:54:17.214 INFO:teuthology.orchestra.run.smithi171.stdout:[global] 2024-09-13T18:54:17.214 INFO:teuthology.orchestra.run.smithi171.stdout: chdir = "" 2024-09-13T18:54:17.215 INFO:teuthology.orchestra.run.smithi171.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T18:54:17.215 INFO:teuthology.orchestra.run.smithi171.stdout: auth supported = cephx 2024-09-13T18:54:17.215 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.215 INFO:teuthology.orchestra.run.smithi171.stdout: filestore xattr use omap = true 2024-09-13T18:54:17.215 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.215 INFO:teuthology.orchestra.run.smithi171.stdout: mon clock drift allowed = 1.000 2024-09-13T18:54:17.215 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.215 INFO:teuthology.orchestra.run.smithi171.stdout: osd crush chooseleaf type = 0 2024-09-13T18:54:17.215 INFO:teuthology.orchestra.run.smithi171.stdout: auth debug = true 2024-09-13T18:54:17.216 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.216 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on old message = true 2024-09-13T18:54:17.216 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on bug = true 2024-09-13T18:54:17.216 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.216 INFO:teuthology.orchestra.run.smithi171.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T18:54:17.216 INFO:teuthology.orchestra.run.smithi171.stdout: mon pg warn max object skew = 0 2024-09-13T18:54:17.216 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.216 INFO:teuthology.orchestra.run.smithi171.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T18:54:17.216 INFO:teuthology.orchestra.run.smithi171.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T18:54:17.216 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.216 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default size = 2 2024-09-13T18:54:17.216 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.216 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow primary affinity = true 2024-09-13T18:54:17.217 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow pg remap = true 2024-09-13T18:54:17.217 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on legacy crush tunables = false 2024-09-13T18:54:17.217 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on crush straw calc version zero = false 2024-09-13T18:54:17.217 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on no sortbitwise = false 2024-09-13T18:54:17.217 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on osd down out interval zero = false 2024-09-13T18:54:17.217 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on too few osds = false 2024-09-13T18:54:17.217 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T18:54:17.217 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T18:54:17.217 INFO:teuthology.orchestra.run.smithi171.stdout: mon_allow_pool_size_one = true 2024-09-13T18:54:17.217 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.217 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T18:54:17.217 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.217 INFO:teuthology.orchestra.run.smithi171.stdout: osd default data pool replay window = 5 2024-09-13T18:54:17.217 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.218 INFO:teuthology.orchestra.run.smithi171.stdout: mon allow pool delete = true 2024-09-13T18:54:17.218 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.218 INFO:teuthology.orchestra.run.smithi171.stdout: mon cluster log file level = debug 2024-09-13T18:54:17.218 INFO:teuthology.orchestra.run.smithi171.stdout: debug asserts on shutdown = true 2024-09-13T18:54:17.218 INFO:teuthology.orchestra.run.smithi171.stdout: mon health detail to clog = false 2024-09-13T18:54:17.218 INFO:teuthology.orchestra.run.smithi171.stdout: mon host = "172.21.15.7,172.21.15.171,[v2:172.21.15.171:3301,v1:172.21.15.171:6790]" 2024-09-13T18:54:17.218 INFO:teuthology.orchestra.run.smithi171.stdout: fsid = 33406d92-f401-438a-876e-a56de3962d8e 2024-09-13T18:54:17.218 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.218 INFO:teuthology.orchestra.run.smithi171.stdout:[osd] 2024-09-13T18:54:17.218 INFO:teuthology.orchestra.run.smithi171.stdout: osd journal size = 100 2024-09-13T18:54:17.218 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.218 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub load threshold = 5.0 2024-09-13T18:54:17.218 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub max interval = 600 2024-09-13T18:54:17.219 INFO:teuthology.orchestra.run.smithi171.stdout: osd mclock profile = high_recovery_ops 2024-09-13T18:54:17.219 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.219 INFO:teuthology.orchestra.run.smithi171.stdout: osd recover clone overlap = true 2024-09-13T18:54:17.219 INFO:teuthology.orchestra.run.smithi171.stdout: osd recovery max chunk = 1048576 2024-09-13T18:54:17.219 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.219 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug shutdown = true 2024-09-13T18:54:17.219 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug op order = true 2024-09-13T18:54:17.219 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug verify stray on activate = true 2024-09-13T18:54:17.219 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.219 INFO:teuthology.orchestra.run.smithi171.stdout: osd open classes on start = true 2024-09-13T18:54:17.219 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug pg log writeout = true 2024-09-13T18:54:17.219 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.219 INFO:teuthology.orchestra.run.smithi171.stdout: osd deep scrub update digest min age = 30 2024-09-13T18:54:17.220 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.220 INFO:teuthology.orchestra.run.smithi171.stdout: osd map max advance = 10 2024-09-13T18:54:17.220 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.220 INFO:teuthology.orchestra.run.smithi171.stdout: journal zero on create = true 2024-09-13T18:54:17.220 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.220 INFO:teuthology.orchestra.run.smithi171.stdout: filestore ondisk finisher threads = 3 2024-09-13T18:54:17.220 INFO:teuthology.orchestra.run.smithi171.stdout: filestore apply finisher threads = 3 2024-09-13T18:54:17.220 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.220 INFO:teuthology.orchestra.run.smithi171.stdout: bdev debug aio = true 2024-09-13T18:54:17.220 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug misdirected ops = true 2024-09-13T18:54:17.220 INFO:teuthology.orchestra.run.smithi171.stdout: bdev async discard = True 2024-09-13T18:54:17.220 INFO:teuthology.orchestra.run.smithi171.stdout: bdev enable discard = True 2024-09-13T18:54:17.220 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore allocator = bitmap 2024-09-13T18:54:17.220 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore block size = 96636764160 2024-09-13T18:54:17.221 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore fsck on mount = True 2024-09-13T18:54:17.221 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluefs = 1/20 2024-09-13T18:54:17.221 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluestore = 1/20 2024-09-13T18:54:17.221 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:54:17.221 INFO:teuthology.orchestra.run.smithi171.stdout: debug osd = 20 2024-09-13T18:54:17.221 INFO:teuthology.orchestra.run.smithi171.stdout: debug rocksdb = 4/10 2024-09-13T18:54:17.221 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T18:54:17.221 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd full ratio = 0.9 2024-09-13T18:54:17.221 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd nearfull ratio = 0.8 2024-09-13T18:54:17.221 INFO:teuthology.orchestra.run.smithi171.stdout: osd failsafe full ratio = 0.95 2024-09-13T18:54:17.221 INFO:teuthology.orchestra.run.smithi171.stdout: osd objectstore = bluestore 2024-09-13T18:54:17.221 INFO:teuthology.orchestra.run.smithi171.stdout: osd op complaint time = 180 2024-09-13T18:54:17.221 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.221 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr] 2024-09-13T18:54:17.222 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:54:17.222 INFO:teuthology.orchestra.run.smithi171.stdout: debug mgr = 20 2024-09-13T18:54:17.222 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-09-13T18:54:17.222 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-09-13T18:54:17.222 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:54:17.222 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:54:17.222 INFO:teuthology.orchestra.run.smithi171.stdout: mgr/telemetry/nag = false 2024-09-13T18:54:17.222 INFO:teuthology.orchestra.run.smithi171.stdout: debug client = 20 2024-09-13T18:54:17.222 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.222 INFO:teuthology.orchestra.run.smithi171.stdout:[mon] 2024-09-13T18:54:17.222 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:54:17.222 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-09-13T18:54:17.222 INFO:teuthology.orchestra.run.smithi171.stdout: debug paxos = 20 2024-09-13T18:54:17.222 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-09-13T18:54:17.223 INFO:teuthology.orchestra.run.smithi171.stdout: mon data avail warn = 5 2024-09-13T18:54:17.223 INFO:teuthology.orchestra.run.smithi171.stdout: mon mgr mkfs grace = 240 2024-09-13T18:54:17.223 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:54:17.223 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd reporter subtree level = osd 2024-09-13T18:54:17.223 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd prime pg temp = true 2024-09-13T18:54:17.223 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:54:17.223 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.223 INFO:teuthology.orchestra.run.smithi171.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T18:54:17.223 INFO:teuthology.orchestra.run.smithi171.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T18:54:17.223 INFO:teuthology.orchestra.run.smithi171.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T18:54:17.223 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.223 INFO:teuthology.orchestra.run.smithi171.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T18:54:17.223 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T18:54:17.223 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T18:54:17.223 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: # 1m isn't quite enough 2024-09-13T18:54:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: mon_down_mkfs_grace = 2m 2024-09-13T18:54:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_filestore_osds = false 2024-09-13T18:54:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: mon op complaint time = 120 2024-09-13T18:54:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on pool no app = False 2024-09-13T18:54:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:17.224 INFO:teuthology.orchestra.run.smithi171.stdout:[client] 2024-09-13T18:54:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: rgw cache enabled = true 2024-09-13T18:54:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable ops log = true 2024-09-13T18:54:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable usage log = true 2024-09-13T18:54:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T18:54:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T18:54:17.224 INFO:teuthology.orchestra.run.smithi171.stdout: client mount timeout = 600 2024-09-13T18:54:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: debug client = 20 2024-09-13T18:54:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:54:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: pid file = "" 2024-09-13T18:54:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: rados mon op timeout = 900 2024-09-13T18:54:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: rados osd op timeout = 900 2024-09-13T18:54:17.225 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.a] 2024-09-13T18:54:17.225 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.b] 2024-09-13T18:54:17.225 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.c] 2024-09-13T18:54:17.225 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.a] 2024-09-13T18:54:17.225 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.c] 2024-09-13T18:54:17.225 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.b] 2024-09-13T18:54:17.225 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.d] 2024-09-13T18:54:17.225 INFO:teuthology.orchestra.run.smithi171.stdout:[mds] 2024-09-13T18:54:17.225 INFO:teuthology.orchestra.run.smithi171.stdout: debug mds = 20 2024-09-13T18:54:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: debug mds balancer = 20 2024-09-13T18:54:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:54:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: mds debug frag = True 2024-09-13T18:54:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: mds debug scatterstat = True 2024-09-13T18:54:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: mds op complaint time = 180 2024-09-13T18:54:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: mds verify scatter = True 2024-09-13T18:54:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: osd op complaint time = 180 2024-09-13T18:54:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: rados mon op timeout = 900 2024-09-13T18:54:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: rados osd op timeout = 900 2024-09-13T18:54:17.226 INFO:teuthology.orchestra.run.smithi171.stdout: osd_deep_scrub_large_omap_object_key_threshold = 1 2024-09-13T18:54:17.227 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T18:54:17.227 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T18:54:17.227 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T18:54:17.227 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:54:17.227 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-13T18:54:17.227 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-13T18:54:17.228 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T18:54:17.228 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:54:17.228 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T18:54:17.228 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T18:54:17.228 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T18:54:17.228 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:54:17.228 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-13T18:54:17.228 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-13T18:54:17.229 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T18:54:17.229 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:54:17.242 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T18:54:17.242 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T18:54:17.246 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T18:54:17.247 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-13T18:54:17.276 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T18:54:17.277 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T18:54:17.278 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T18:54:17.278 DEBUG:teuthology.orchestra.run.smithi171:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-13T18:54:17.279 INFO:tasks.ceph.mds.b:Started 2024-09-13T18:54:17.280 DEBUG:teuthology.parallel:result is None 2024-09-13T18:54:17.281 INFO:tasks.ceph.mds.d:Started 2024-09-13T18:54:17.281 DEBUG:teuthology.parallel:result is None 2024-09-13T18:54:17.291 INFO:tasks.ceph.mds.a:Started 2024-09-13T18:54:17.291 DEBUG:teuthology.parallel:result is None 2024-09-13T18:54:17.292 INFO:tasks.ceph.mds.c:Started 2024-09-13T18:54:17.293 DEBUG:teuthology.parallel:result is None 2024-09-13T18:54:17.293 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:17.355 INFO:tasks.ceph.mds.b.smithi171.stdout:starting mds.b at 2024-09-13T18:54:17.361 INFO:tasks.ceph.mds.d.smithi171.stdout:starting mds.d at 2024-09-13T18:54:17.371 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-09-13T18:54:17.386 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-13T18:54:17.391 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-13T18:54:17.387+0000 7f28f0ff8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T18:54:17.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.407+0000 7f376b476700 1 Processor -- start 2024-09-13T18:54:17.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.407+0000 7f376b476700 1 -- start start 2024-09-13T18:54:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.407+0000 7f376b476700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f376415afb0 0x7f376406c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.407+0000 7f376b476700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f376406d5a0 0x7f376415c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.407+0000 7f376b476700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3764071f00 con 0x7f376415b390 2024-09-13T18:54:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.407+0000 7f376b476700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f376415e300 con 0x7f376406d5a0 2024-09-13T18:54:17.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.407+0000 7f376b476700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f376406cc90 con 0x7f376415afb0 2024-09-13T18:54:17.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.407+0000 7f3769c73700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f376406d5a0 0x7f376415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:17.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.407+0000 7f3769c73700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f376406d5a0 0x7f376415c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55572/0 (socket says 172.21.15.7:55572) 2024-09-13T18:54:17.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.407+0000 7f3769c73700 1 -- 172.21.15.7:0/1497373909 learned_addr learned my addr 172.21.15.7:0/1497373909 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:17.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.407+0000 7f376a474700 1 --2- 172.21.15.7:0/1497373909 >> v2:172.21.15.171:3300/0 conn(0x7f376415afb0 0x7f376406c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:17.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.411+0000 7f3769c73700 1 -- 172.21.15.7:0/1497373909 >> v2:172.21.15.171:3300/0 conn(0x7f376415afb0 msgr2=0x7f376406c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:17.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.411+0000 7f3769c73700 1 --2- 172.21.15.7:0/1497373909 >> v2:172.21.15.171:3300/0 conn(0x7f376415afb0 0x7f376406c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:17.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.411+0000 7f3769c73700 1 -- 172.21.15.7:0/1497373909 >> v1:172.21.15.7:6789/0 conn(0x7f376415b390 legacy=0x7f376415b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:17.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.411+0000 7f3769c73700 1 -- 172.21.15.7:0/1497373909 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3764071980 con 0x7f376406d5a0 2024-09-13T18:54:17.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.411+0000 7f3769c73700 1 --2- 172.21.15.7:0/1497373909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f376406d5a0 0x7f376415c100 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f376406f880 tx=0x7f3760009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a3d8417909e60 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:17.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.411+0000 7f3769472700 1 -- 172.21.15.7:0/1497373909 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760004230 con 0x7f376406d5a0 2024-09-13T18:54:17.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.411+0000 7f3769472700 1 -- 172.21.15.7:0/1497373909 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3760004390 con 0x7f376406d5a0 2024-09-13T18:54:17.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.411+0000 7f3769472700 1 -- 172.21.15.7:0/1497373909 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37600036c0 con 0x7f376406d5a0 2024-09-13T18:54:17.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.411+0000 7f376b476700 1 -- 172.21.15.7:0/1497373909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f376406d5a0 msgr2=0x7f376415c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.411+0000 7f376b476700 1 --2- 172.21.15.7:0/1497373909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f376406d5a0 0x7f376415c100 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f376406f880 tx=0x7f3760009e50 comp rx=0 tx=0).stop 2024-09-13T18:54:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.411+0000 7f376b476700 1 -- 172.21.15.7:0/1497373909 shutdown_connections 2024-09-13T18:54:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.411+0000 7f376b476700 1 --2- 172.21.15.7:0/1497373909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f376406d5a0 0x7f376415c100 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.411+0000 7f376b476700 1 --2- 172.21.15.7:0/1497373909 >> v2:172.21.15.171:3300/0 conn(0x7f376415afb0 0x7f376406c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.411+0000 7f376b476700 1 -- 172.21.15.7:0/1497373909 >> 172.21.15.7:0/1497373909 conn(0x7f376406a360 msgr2=0x7f376406a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.411+0000 7f376b476700 1 -- 172.21.15.7:0/1497373909 shutdown_connections 2024-09-13T18:54:17.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.411+0000 7f376b476700 1 -- 172.21.15.7:0/1497373909 wait complete. 2024-09-13T18:54:17.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.415+0000 7f376b476700 1 Processor -- start 2024-09-13T18:54:17.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.415+0000 7f376b476700 1 -- start start 2024-09-13T18:54:17.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.415+0000 7f376b476700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f376406d5a0 0x7f3764146c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:17.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.415+0000 7f376b476700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f376415afb0 0x7f37640a8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:17.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.415+0000 7f376b476700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f376415b390 0x7f37640a8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:17.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.415+0000 7f376b476700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37641475a0 con 0x7f376415afb0 2024-09-13T18:54:17.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.415+0000 7f376b476700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3764158eb0 con 0x7f376406d5a0 2024-09-13T18:54:17.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.415+0000 7f376b476700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37640ad370 con 0x7f376415b390 2024-09-13T18:54:17.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.415+0000 7f376ac75700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f376415b390 0x7f37640a8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:17.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.415+0000 7f376ac75700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f376415b390 0x7f37640a8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:47436/0 (socket says 172.21.15.7:47436) 2024-09-13T18:54:17.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.415+0000 7f376ac75700 1 -- 172.21.15.7:0/3580164720 learned_addr learned my addr 172.21.15.7:0/3580164720 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:17.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.415+0000 7f376ac75700 1 -- 172.21.15.7:0/3580164720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f376406d5a0 msgr2=0x7f3764146c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:17.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.415+0000 7f376ac75700 1 --2- 172.21.15.7:0/3580164720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f376406d5a0 0x7f3764146c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:17.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.415+0000 7f376ac75700 1 -- 172.21.15.7:0/3580164720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f376415afb0 msgr2=0x7f37640a8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:17.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.419+0000 7f3769c73700 1 --2- 172.21.15.7:0/3580164720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f376415afb0 0x7f37640a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:17.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.419+0000 7f376ac75700 1 --2- 172.21.15.7:0/3580164720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f376415afb0 0x7f37640a8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:17.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.419+0000 7f376ac75700 1 -- 172.21.15.7:0/3580164720 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3760009570 con 0x7f376415b390 2024-09-13T18:54:17.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.419+0000 7f376ac75700 1 --2- 172.21.15.7:0/3580164720 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f376415b390 0x7f37640a8990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f375c00b970 tx=0x7f375c00bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:17.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.419+0000 7f3756ffd700 1 -- 172.21.15.7:0/3580164720 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375c00c9b0 con 0x7f376415b390 2024-09-13T18:54:17.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.419+0000 7f3756ffd700 1 -- 172.21.15.7:0/3580164720 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375c00cb10 con 0x7f376415b390 2024-09-13T18:54:17.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.419+0000 7f3756ffd700 1 -- 172.21.15.7:0/3580164720 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375c014440 con 0x7f376415b390 2024-09-13T18:54:17.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.419+0000 7f376b476700 1 -- 172.21.15.7:0/3580164720 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37640ad650 con 0x7f376415b390 2024-09-13T18:54:17.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.419+0000 7f376b476700 1 -- 172.21.15.7:0/3580164720 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37641477f0 con 0x7f376415b390 2024-09-13T18:54:17.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.419+0000 7f376b476700 1 -- 172.21.15.7:0/3580164720 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f376415c100 con 0x7f376415b390 2024-09-13T18:54:17.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.419+0000 7f3756ffd700 1 -- 172.21.15.7:0/3580164720 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f375c01d3b0 con 0x7f376415b390 2024-09-13T18:54:17.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.419+0000 7f3756ffd700 1 --2- 172.21.15.7:0/3580164720 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3748066730 0x7f3748068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:17.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.419+0000 7f3756ffd700 1 -- 172.21.15.7:0/3580164720 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f375c083e40 con 0x7f376415b390 2024-09-13T18:54:17.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.419+0000 7f376a474700 1 --2- 172.21.15.7:0/3580164720 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3748066730 0x7f3748068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:17.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.423+0000 7f376a474700 1 --2- 172.21.15.7:0/3580164720 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3748066730 0x7f3748068bf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f375800d440 tx=0x7f3758008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.427+0000 7f3756ffd700 1 -- 172.21.15.7:0/3580164720 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f375c051150 con 0x7f376415b390 2024-09-13T18:54:17.480 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-13T18:54:17.480 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-13T18:54:17.481 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-13T18:54:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.635+0000 7f376b476700 1 -- 172.21.15.7:0/3580164720 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37641479e0 con 0x7f376415b390 2024-09-13T18:54:17.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.635+0000 7f3756ffd700 1 -- 172.21.15.7:0/3580164720 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f375c054770 con 0x7f376415b390 2024-09-13T18:54:17.641 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:17.642 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1829974483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1829974483},{"type":"v1","addr":"172.21.15.171:6835","nonce":1829974483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1948505526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1948505526},{"type":"v1","addr":"172.21.15.171:6837","nonce":1948505526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5629,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/1132382369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1132382369},{"type":"v1","addr":"172.21.15.7:6833","nonce":1132382369}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:17.397634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"c","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6835/3768750832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3768750832},{"type":"v1","addr":"172.21.15.7:6835","nonce":3768750832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.639+0000 7f376b476700 1 -- 172.21.15.7:0/3580164720 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3748066730 msgr2=0x7f3748068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.639+0000 7f376b476700 1 --2- 172.21.15.7:0/3580164720 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3748066730 0x7f3748068bf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f375800d440 tx=0x7f3758008040 comp rx=0 tx=0).stop 2024-09-13T18:54:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.639+0000 7f376b476700 1 -- 172.21.15.7:0/3580164720 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f376415b390 msgr2=0x7f37640a8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.639+0000 7f376b476700 1 --2- 172.21.15.7:0/3580164720 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f376415b390 0x7f37640a8990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f375c00b970 tx=0x7f375c00bd30 comp rx=0 tx=0).stop 2024-09-13T18:54:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.639+0000 7f376b476700 1 -- 172.21.15.7:0/3580164720 shutdown_connections 2024-09-13T18:54:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.639+0000 7f376b476700 1 --2- 172.21.15.7:0/3580164720 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3748066730 0x7f3748068bf0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.639+0000 7f376b476700 1 --2- 172.21.15.7:0/3580164720 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f376415b390 0x7f37640a8990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.639+0000 7f376b476700 1 --2- 172.21.15.7:0/3580164720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f376415afb0 0x7f37640a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.639+0000 7f376b476700 1 --2- 172.21.15.7:0/3580164720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f376406d5a0 0x7f3764146c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.639+0000 7f376b476700 1 -- 172.21.15.7:0/3580164720 >> 172.21.15.7:0/3580164720 conn(0x7f376406a360 msgr2=0x7f3764070140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.639+0000 7f376b476700 1 -- 172.21.15.7:0/3580164720 shutdown_connections 2024-09-13T18:54:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:17.639+0000 7f376b476700 1 -- 172.21.15.7:0/3580164720 wait complete. 2024-09-13T18:54:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 32 2024-09-13T18:54:18.059 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T18:54:09.004150+0000', 'modified': '2024-09-13T18:54:17.397634+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 55, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5910}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5910': {'gid': 5910, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.7:6835/3768750832', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 3768750832}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 3768750832}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T18:54:18.060 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5910:up:replay 2024-09-13T18:54:19.061 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.179+0000 7f0acebd3700 1 Processor -- start 2024-09-13T18:54:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.179+0000 7f0acebd3700 1 -- start start 2024-09-13T18:54:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0acebd3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ac8070e50 0x7f0ac80ada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0acebd3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac80b1aa0 con 0x7f0ac80cabf0 2024-09-13T18:54:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0acebd3700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac806be40 con 0x7f0ac81599b0 2024-09-13T18:54:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0acebd3700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac806c3c0 con 0x7f0ac8070e50 2024-09-13T18:54:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0acd3d0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ac8070e50 0x7f0ac80ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0acd3d0700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ac8070e50 0x7f0ac80ada80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55586/0 (socket says 172.21.15.7:55586) 2024-09-13T18:54:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0ace3d2700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0ac80cabf0 0x7f0ac80b1f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36750/0 (socket says 172.21.15.7:36750) 2024-09-13T18:54:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0acd3d0700 1 -- 172.21.15.7:0/1937735633 learned_addr learned my addr 172.21.15.7:0/1937735633 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:19.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0acd3d0700 1 -- 172.21.15.7:0/1937735633 >> v1:172.21.15.171:6789/0 conn(0x7f0ac81599b0 legacy=0x7f0ac8070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:19.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0acd3d0700 1 -- 172.21.15.7:0/1937735633 >> v1:172.21.15.7:6789/0 conn(0x7f0ac80cabf0 legacy=0x7f0ac80b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:19.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0acd3d0700 1 -- 172.21.15.7:0/1937735633 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac80cb140 con 0x7f0ac8070e50 2024-09-13T18:54:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0acd3d0700 1 --2- 172.21.15.7:0/1937735633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac8070e50 0x7f0ac80ada80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009910 tx=0x7f0ab8009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8d0155347441d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0accbcf700 1 -- 172.21.15.7:0/1937735633 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab800ba60 con 0x7f0ac8070e50 2024-09-13T18:54:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0accbcf700 1 -- 172.21.15.7:0/1937735633 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab800bbc0 con 0x7f0ac8070e50 2024-09-13T18:54:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0accbcf700 1 -- 172.21.15.7:0/1937735633 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8009e60 con 0x7f0ac8070e50 2024-09-13T18:54:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0acebd3700 1 -- 172.21.15.7:0/1937735633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac8070e50 msgr2=0x7f0ac80ada80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0acebd3700 1 --2- 172.21.15.7:0/1937735633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac8070e50 0x7f0ac80ada80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009910 tx=0x7f0ab8009c20 comp rx=0 tx=0).stop 2024-09-13T18:54:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0acebd3700 1 -- 172.21.15.7:0/1937735633 shutdown_connections 2024-09-13T18:54:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0acebd3700 1 --2- 172.21.15.7:0/1937735633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac8070e50 0x7f0ac80ada80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0acebd3700 1 -- 172.21.15.7:0/1937735633 >> 172.21.15.7:0/1937735633 conn(0x7f0ac80b0570 msgr2=0x7f0ac80b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:19.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0acebd3700 1 -- 172.21.15.7:0/1937735633 shutdown_connections 2024-09-13T18:54:19.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.183+0000 7f0acebd3700 1 -- 172.21.15.7:0/1937735633 wait complete. 2024-09-13T18:54:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.187+0000 7f0acebd3700 1 Processor -- start 2024-09-13T18:54:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.187+0000 7f0acebd3700 1 -- start start 2024-09-13T18:54:19.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.187+0000 7f0acebd3700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0ac8070e50 0x7f0ac80a9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:19.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0acebd3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ac80cabf0 0x7f0ac80a8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:19.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0acebd3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac81599b0 0x7f0ac814e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:19.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0acebd3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac80a93a0 con 0x7f0ac81599b0 2024-09-13T18:54:19.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0acebd3700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac81eeae0 con 0x7f0ac80cabf0 2024-09-13T18:54:19.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0acebd3700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac81eedc0 con 0x7f0ac8070e50 2024-09-13T18:54:19.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0ace3d2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac81599b0 0x7f0ac814e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:19.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0acd3d0700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ac80cabf0 0x7f0ac80a8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:19.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0ace3d2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac81599b0 0x7f0ac814e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55592/0 (socket says 172.21.15.7:55592) 2024-09-13T18:54:19.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0ace3d2700 1 -- 172.21.15.7:0/2230130611 learned_addr learned my addr 172.21.15.7:0/2230130611 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:19.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0acdbd1700 1 --2- 172.21.15.7:0/2230130611 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0ac8070e50 0x7f0ac80a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:19.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0ace3d2700 1 -- 172.21.15.7:0/2230130611 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0ac8070e50 msgr2=0x7f0ac80a9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:19.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0ace3d2700 1 --2- 172.21.15.7:0/2230130611 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0ac8070e50 0x7f0ac80a9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0ace3d2700 1 -- 172.21.15.7:0/2230130611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ac80cabf0 msgr2=0x7f0ac80a8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0ace3d2700 1 --2- 172.21.15.7:0/2230130611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ac80cabf0 0x7f0ac80a8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0ace3d2700 1 -- 172.21.15.7:0/2230130611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab8009570 con 0x7f0ac81599b0 2024-09-13T18:54:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0ace3d2700 1 --2- 172.21.15.7:0/2230130611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac81599b0 0x7f0ac814e180 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000cbd0 tx=0x7f0ac000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0abe7fc700 1 -- 172.21.15.7:0/2230130611 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac000aaa0 con 0x7f0ac81599b0 2024-09-13T18:54:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0abe7fc700 1 -- 172.21.15.7:0/2230130611 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac000ac00 con 0x7f0ac81599b0 2024-09-13T18:54:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0acebd3700 1 -- 172.21.15.7:0/2230130611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac81ef0a0 con 0x7f0ac81599b0 2024-09-13T18:54:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0abe7fc700 1 -- 172.21.15.7:0/2230130611 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0007070 con 0x7f0ac81599b0 2024-09-13T18:54:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0acebd3700 1 -- 172.21.15.7:0/2230130611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac81ef660 con 0x7f0ac81599b0 2024-09-13T18:54:19.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.191+0000 7f0acebd3700 1 -- 172.21.15.7:0/2230130611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac81ef850 con 0x7f0ac81599b0 2024-09-13T18:54:19.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.195+0000 7f0abe7fc700 1 -- 172.21.15.7:0/2230130611 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0ac00071d0 con 0x7f0ac81599b0 2024-09-13T18:54:19.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.195+0000 7f0abe7fc700 1 --2- 172.21.15.7:0/2230130611 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0aac066730 0x7f0aac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:19.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.195+0000 7f0abe7fc700 1 -- 172.21.15.7:0/2230130611 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0ac0083ec0 con 0x7f0ac81599b0 2024-09-13T18:54:19.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.195+0000 7f0acdbd1700 1 --2- 172.21.15.7:0/2230130611 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0aac066730 0x7f0aac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:19.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.195+0000 7f0acdbd1700 1 --2- 172.21.15.7:0/2230130611 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0aac066730 0x7f0aac068bf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4000f60 tx=0x7f0ac4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:19.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.203+0000 7f0abe7fc700 1 -- 172.21.15.7:0/2230130611 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0ac00511e0 con 0x7f0ac81599b0 2024-09-13T18:54:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.419+0000 7f0acebd3700 1 -- 172.21.15.7:0/2230130611 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ac80b1290 con 0x7f0ac81599b0 2024-09-13T18:54:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.419+0000 7f0abe7fc700 1 -- 172.21.15.7:0/2230130611 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f0ac0017020 con 0x7f0ac81599b0 2024-09-13T18:54:19.426 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:19.426 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1829974483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1829974483},{"type":"v1","addr":"172.21.15.171:6835","nonce":1829974483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1948505526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1948505526},{"type":"v1","addr":"172.21.15.171:6837","nonce":1948505526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3617964311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3617964311},{"type":"v1","addr":"172.21.15.7:6833","nonce":3617964311}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:19.403589+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"c","rank":0,"incarnation":32,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.7:6835/3768750832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3768750832},{"type":"v1","addr":"172.21.15.7:6835","nonce":3768750832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:19.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.423+0000 7f0acebd3700 1 -- 172.21.15.7:0/2230130611 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0aac066730 msgr2=0x7f0aac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:19.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.423+0000 7f0acebd3700 1 --2- 172.21.15.7:0/2230130611 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0aac066730 0x7f0aac068bf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4000f60 tx=0x7f0ac4009040 comp rx=0 tx=0).stop 2024-09-13T18:54:19.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.423+0000 7f0acebd3700 1 -- 172.21.15.7:0/2230130611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac81599b0 msgr2=0x7f0ac814e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:19.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.423+0000 7f0acebd3700 1 --2- 172.21.15.7:0/2230130611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac81599b0 0x7f0ac814e180 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000cbd0 tx=0x7f0ac000cee0 comp rx=0 tx=0).stop 2024-09-13T18:54:19.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.423+0000 7f0acebd3700 1 -- 172.21.15.7:0/2230130611 shutdown_connections 2024-09-13T18:54:19.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.423+0000 7f0acebd3700 1 --2- 172.21.15.7:0/2230130611 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0aac066730 0x7f0aac068bf0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:19.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.423+0000 7f0acebd3700 1 --2- 172.21.15.7:0/2230130611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ac81599b0 0x7f0ac814e180 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:19.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.423+0000 7f0acebd3700 1 --2- 172.21.15.7:0/2230130611 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ac80cabf0 0x7f0ac80a8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:19.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.423+0000 7f0acebd3700 1 --2- 172.21.15.7:0/2230130611 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0ac8070e50 0x7f0ac80a9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:19.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.423+0000 7f0acebd3700 1 -- 172.21.15.7:0/2230130611 >> 172.21.15.7:0/2230130611 conn(0x7f0ac80b0570 msgr2=0x7f0ac80aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:19.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.423+0000 7f0acebd3700 1 -- 172.21.15.7:0/2230130611 shutdown_connections 2024-09-13T18:54:19.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:19.423+0000 7f0acebd3700 1 -- 172.21.15.7:0/2230130611 wait complete. 2024-09-13T18:54:19.430 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2024-09-13T18:54:19.843 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T18:54:09.004150+0000', 'modified': '2024-09-13T18:54:19.403589+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 55, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5910}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5910': {'gid': 5910, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.7:6835/3768750832', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 3768750832}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 3768750832}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T18:54:19.843 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5910:up:rejoin 2024-09-13T18:54:20.844 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:20.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd68427700 1 Processor -- start 2024-09-13T18:54:20.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd68427700 1 -- start start 2024-09-13T18:54:20.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd68427700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efd60070080 0x7efd600cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:20.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd68427700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd600cce40 con 0x7efd600b2d00 2024-09-13T18:54:20.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd68427700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd600cd120 con 0x7efd600cc980 2024-09-13T18:54:20.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd68427700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd60071ce0 con 0x7efd60070080 2024-09-13T18:54:20.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd661c3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efd60070080 0x7efd600cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:20.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd661c3700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efd60070080 0x7efd600cdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55608/0 (socket says 172.21.15.7:55608) 2024-09-13T18:54:20.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd661c3700 1 -- 172.21.15.7:0/366054044 learned_addr learned my addr 172.21.15.7:0/366054044 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:20.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd661c3700 1 -- 172.21.15.7:0/366054044 >> v1:172.21.15.171:6789/0 conn(0x7efd600cc980 legacy=0x7efd60070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:20.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd661c3700 1 -- 172.21.15.7:0/366054044 >> v1:172.21.15.7:6789/0 conn(0x7efd600b2d00 legacy=0x7efd6006bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:20.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd661c3700 1 -- 172.21.15.7:0/366054044 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd600cd400 con 0x7efd60070080 2024-09-13T18:54:20.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd661c3700 1 --2- 172.21.15.7:0/366054044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd60070080 0x7efd600cdee0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009910 tx=0x7efd5c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a414591bd789142 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:20.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd651c1700 1 -- 172.21.15.7:0/366054044 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c00ba60 con 0x7efd60070080 2024-09-13T18:54:20.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd651c1700 1 -- 172.21.15.7:0/366054044 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5c00bbc0 con 0x7efd60070080 2024-09-13T18:54:20.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd651c1700 1 -- 172.21.15.7:0/366054044 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c009e60 con 0x7efd60070080 2024-09-13T18:54:20.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd68427700 1 -- 172.21.15.7:0/366054044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd60070080 msgr2=0x7efd600cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:20.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd68427700 1 --2- 172.21.15.7:0/366054044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd60070080 0x7efd600cdee0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009910 tx=0x7efd5c009c20 comp rx=0 tx=0).stop 2024-09-13T18:54:20.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd68427700 1 -- 172.21.15.7:0/366054044 shutdown_connections 2024-09-13T18:54:20.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd68427700 1 --2- 172.21.15.7:0/366054044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd60070080 0x7efd600cdee0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:20.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd68427700 1 -- 172.21.15.7:0/366054044 >> 172.21.15.7:0/366054044 conn(0x7efd600b0830 msgr2=0x7efd600b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:20.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd68427700 1 -- 172.21.15.7:0/366054044 shutdown_connections 2024-09-13T18:54:20.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.967+0000 7efd68427700 1 -- 172.21.15.7:0/366054044 wait complete. 2024-09-13T18:54:20.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.971+0000 7efd68427700 1 Processor -- start 2024-09-13T18:54:20.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.971+0000 7efd68427700 1 -- start start 2024-09-13T18:54:20.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.971+0000 7efd68427700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7efd60070080 0x7efd60143de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:20.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.971+0000 7efd68427700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd600b2d00 0x7efd600a8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:20.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.971+0000 7efd68427700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd600cc980 0x7efd600add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:20.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.971+0000 7efd68427700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd601eb750 con 0x7efd600b2d00 2024-09-13T18:54:20.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.971+0000 7efd68427700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd601eae20 con 0x7efd600cc980 2024-09-13T18:54:20.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.971+0000 7efd68427700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd601eb100 con 0x7efd60070080 2024-09-13T18:54:20.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.971+0000 7efd659c2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd600b2d00 0x7efd600a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:20.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.971+0000 7efd661c3700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7efd60070080 0x7efd60143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:20.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.971+0000 7efd661c3700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7efd60070080 0x7efd60143de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:47448/0 (socket says 172.21.15.7:47448) 2024-09-13T18:54:20.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.971+0000 7efd659c2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd600b2d00 0x7efd600a8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55614/0 (socket says 172.21.15.7:55614) 2024-09-13T18:54:20.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.971+0000 7efd661c3700 1 -- 172.21.15.7:0/2746503509 learned_addr learned my addr 172.21.15.7:0/2746503509 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:20.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.971+0000 7efd669c4700 1 --2- 172.21.15.7:0/2746503509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd600cc980 0x7efd600add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:20.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.971+0000 7efd659c2700 1 -- 172.21.15.7:0/2746503509 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7efd60070080 msgr2=0x7efd60143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:20.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.971+0000 7efd659c2700 1 --2- 172.21.15.7:0/2746503509 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7efd60070080 0x7efd60143de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:20.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.975+0000 7efd659c2700 1 -- 172.21.15.7:0/2746503509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd600cc980 msgr2=0x7efd600add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.975+0000 7efd659c2700 1 --2- 172.21.15.7:0/2746503509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd600cc980 0x7efd600add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.975+0000 7efd659c2700 1 -- 172.21.15.7:0/2746503509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5c009570 con 0x7efd600b2d00 2024-09-13T18:54:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.975+0000 7efd659c2700 1 --2- 172.21.15.7:0/2746503509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd600b2d00 0x7efd600a8450 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efd5800cb10 tx=0x7efd5800ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.975+0000 7efd56ffd700 1 -- 172.21.15.7:0/2746503509 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd58003c20 con 0x7efd600b2d00 2024-09-13T18:54:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.975+0000 7efd56ffd700 1 -- 172.21.15.7:0/2746503509 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd58003d80 con 0x7efd600b2d00 2024-09-13T18:54:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.975+0000 7efd68427700 1 -- 172.21.15.7:0/2746503509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd601eb3e0 con 0x7efd600b2d00 2024-09-13T18:54:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.975+0000 7efd56ffd700 1 -- 172.21.15.7:0/2746503509 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5800aa20 con 0x7efd600b2d00 2024-09-13T18:54:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.975+0000 7efd68427700 1 -- 172.21.15.7:0/2746503509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd600ae2a0 con 0x7efd600b2d00 2024-09-13T18:54:20.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.975+0000 7efd68427700 1 -- 172.21.15.7:0/2746503509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd600cdee0 con 0x7efd600b2d00 2024-09-13T18:54:20.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.979+0000 7efd56ffd700 1 -- 172.21.15.7:0/2746503509 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7efd5800ab80 con 0x7efd600b2d00 2024-09-13T18:54:20.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.979+0000 7efd56ffd700 1 --2- 172.21.15.7:0/2746503509 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7efd4c066730 0x7efd4c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:20.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.979+0000 7efd56ffd700 1 -- 172.21.15.7:0/2746503509 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7efd580840f0 con 0x7efd600b2d00 2024-09-13T18:54:20.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.979+0000 7efd661c3700 1 --2- 172.21.15.7:0/2746503509 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7efd4c066730 0x7efd4c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:20.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.979+0000 7efd661c3700 1 --2- 172.21.15.7:0/2746503509 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7efd4c066730 0x7efd4c068bf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009150 tx=0x7efd5c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:20.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:20.991+0000 7efd56ffd700 1 -- 172.21.15.7:0/2746503509 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efd58051380 con 0x7efd600b2d00 2024-09-13T18:54:21.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:21.203+0000 7efd68427700 1 -- 172.21.15.7:0/2746503509 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd601e6f30 con 0x7efd600b2d00 2024-09-13T18:54:21.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:21.203+0000 7efd56ffd700 1 -- 172.21.15.7:0/2746503509 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4741 (secure 0 0 0) 0x7efd580549a0 con 0x7efd600b2d00 2024-09-13T18:54:21.208 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:21.208 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1829974483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1829974483},{"type":"v1","addr":"172.21.15.171:6835","nonce":1829974483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1948505526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1948505526},{"type":"v1","addr":"172.21.15.171:6837","nonce":1948505526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3617964311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3617964311},{"type":"v1","addr":"172.21.15.7:6833","nonce":3617964311}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:20.412183+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"c","rank":0,"incarnation":32,"state":"up:clientreplay","state_seq":4,"addr":"172.21.15.7:6835/3768750832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3768750832},{"type":"v1","addr":"172.21.15.7:6835","nonce":3768750832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:21.207+0000 7efd68427700 1 -- 172.21.15.7:0/2746503509 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7efd4c066730 msgr2=0x7efd4c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:21.207+0000 7efd68427700 1 --2- 172.21.15.7:0/2746503509 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7efd4c066730 0x7efd4c068bf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009150 tx=0x7efd5c008030 comp rx=0 tx=0).stop 2024-09-13T18:54:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:21.207+0000 7efd68427700 1 -- 172.21.15.7:0/2746503509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd600b2d00 msgr2=0x7efd600a8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:21.207+0000 7efd68427700 1 --2- 172.21.15.7:0/2746503509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd600b2d00 0x7efd600a8450 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7efd5800cb10 tx=0x7efd5800ce20 comp rx=0 tx=0).stop 2024-09-13T18:54:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:21.207+0000 7efd68427700 1 -- 172.21.15.7:0/2746503509 shutdown_connections 2024-09-13T18:54:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:21.207+0000 7efd68427700 1 --2- 172.21.15.7:0/2746503509 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7efd4c066730 0x7efd4c068bf0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:21.207+0000 7efd68427700 1 --2- 172.21.15.7:0/2746503509 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efd600cc980 0x7efd600add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:21.207+0000 7efd68427700 1 --2- 172.21.15.7:0/2746503509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd600b2d00 0x7efd600a8450 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:21.207+0000 7efd68427700 1 --2- 172.21.15.7:0/2746503509 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7efd60070080 0x7efd60143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:21.207+0000 7efd68427700 1 -- 172.21.15.7:0/2746503509 >> 172.21.15.7:0/2746503509 conn(0x7efd600b0830 msgr2=0x7efd60148080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:21.207+0000 7efd68427700 1 -- 172.21.15.7:0/2746503509 shutdown_connections 2024-09-13T18:54:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:21.207+0000 7efd68427700 1 -- 172.21.15.7:0/2746503509 wait complete. 2024-09-13T18:54:21.212 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 35 2024-09-13T18:54:21.625 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T18:54:09.004150+0000', 'modified': '2024-09-13T18:54:20.412183+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 55, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5910}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5910': {'gid': 5910, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:clientreplay', 'state_seq': 4, 'addr': '172.21.15.7:6835/3768750832', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 3768750832}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 3768750832}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T18:54:21.625 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5910:up:clientreplay 2024-09-13T18:54:22.407 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-09-13T18:54:22.403+0000 7fc20ae24700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T18:54:22.626 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:22.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.747+0000 7f0412829700 1 Processor -- start 2024-09-13T18:54:22.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.747+0000 7f0412829700 1 -- start start 2024-09-13T18:54:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0412829700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f040c0cccf0 0x7f040c0cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0412829700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f040c06ad70 0x7f040c06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0412829700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f040c06ca10 con 0x7f040c15d320 2024-09-13T18:54:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0412829700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f040c0cd610 con 0x7f040c0cccf0 2024-09-13T18:54:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0412829700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040c06b7c0 con 0x7f040c06ad70 2024-09-13T18:54:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0411827700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f040c0cccf0 0x7f040c0cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0411827700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f040c0cccf0 0x7f040c0cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36868/0 (socket says 172.21.15.7:36868) 2024-09-13T18:54:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0411827700 1 -- 172.21.15.7:0/3352168829 learned_addr learned my addr 172.21.15.7:0/3352168829 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:22.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0411026700 1 --2- 172.21.15.7:0/3352168829 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f040c06ad70 0x7f040c06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0411827700 1 -- 172.21.15.7:0/3352168829 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f040c06ad70 msgr2=0x7f040c06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0411827700 1 --2- 172.21.15.7:0/3352168829 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f040c06ad70 0x7f040c06b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0411827700 1 -- 172.21.15.7:0/3352168829 >> v1:172.21.15.7:6789/0 conn(0x7f040c15d320 legacy=0x7f040c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0411827700 1 -- 172.21.15.7:0/3352168829 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c06baa0 con 0x7f040c0cccf0 2024-09-13T18:54:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0411827700 1 --2- 172.21.15.7:0/3352168829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f040c0cccf0 0x7f040c0cd0d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f040800bbb0 tx=0x7f040800bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153263fbe9bb323d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0410825700 1 -- 172.21.15.7:0/3352168829 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040800dd30 con 0x7f040c0cccf0 2024-09-13T18:54:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0410825700 1 -- 172.21.15.7:0/3352168829 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040800de90 con 0x7f040c0cccf0 2024-09-13T18:54:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0410825700 1 -- 172.21.15.7:0/3352168829 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0408010690 con 0x7f040c0cccf0 2024-09-13T18:54:22.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0412829700 1 -- 172.21.15.7:0/3352168829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f040c0cccf0 msgr2=0x7f040c0cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:22.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0412829700 1 --2- 172.21.15.7:0/3352168829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f040c0cccf0 0x7f040c0cd0d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f040800bbb0 tx=0x7f040800bec0 comp rx=0 tx=0).stop 2024-09-13T18:54:22.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0412829700 1 -- 172.21.15.7:0/3352168829 shutdown_connections 2024-09-13T18:54:22.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0412829700 1 --2- 172.21.15.7:0/3352168829 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f040c06ad70 0x7f040c06b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:22.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0412829700 1 --2- 172.21.15.7:0/3352168829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f040c0cccf0 0x7f040c0cd0d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:22.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0412829700 1 -- 172.21.15.7:0/3352168829 >> 172.21.15.7:0/3352168829 conn(0x7f040c0cf120 msgr2=0x7f040c0cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:22.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0412829700 1 -- 172.21.15.7:0/3352168829 shutdown_connections 2024-09-13T18:54:22.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.751+0000 7f0412829700 1 -- 172.21.15.7:0/3352168829 wait complete. 2024-09-13T18:54:22.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.755+0000 7f0412829700 1 Processor -- start 2024-09-13T18:54:22.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.755+0000 7f0412829700 1 -- start start 2024-09-13T18:54:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.755+0000 7f0412829700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f040c06ad70 0x7f040c155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.755+0000 7f0412829700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f040c0cccf0 0x7f040c071410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.755+0000 7f0412829700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f040c15d320 0x7f040c071950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.755+0000 7f0412829700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c155d60 con 0x7f040c0cccf0 2024-09-13T18:54:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.755+0000 7f0412829700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c0c4990 con 0x7f040c06ad70 2024-09-13T18:54:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.755+0000 7f0412829700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040c0c4c70 con 0x7f040c15d320 2024-09-13T18:54:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.755+0000 7f0411026700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f040c0cccf0 0x7f040c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.755+0000 7f0411026700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f040c0cccf0 0x7f040c071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36876/0 (socket says 172.21.15.7:36876) 2024-09-13T18:54:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.755+0000 7f0411026700 1 -- 172.21.15.7:0/1764965877 learned_addr learned my addr 172.21.15.7:0/1764965877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.755+0000 7f0411827700 1 --2- 172.21.15.7:0/1764965877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f040c06ad70 0x7f040c155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.755+0000 7f0412028700 1 --2- 172.21.15.7:0/1764965877 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f040c15d320 0x7f040c071950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:22.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.755+0000 7f0411026700 1 -- 172.21.15.7:0/1764965877 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f040c15d320 msgr2=0x7f040c071950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.755+0000 7f0411026700 1 --2- 172.21.15.7:0/1764965877 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f040c15d320 0x7f040c071950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.755+0000 7f0411026700 1 -- 172.21.15.7:0/1764965877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f040c06ad70 msgr2=0x7f040c155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.755+0000 7f0411026700 1 --2- 172.21.15.7:0/1764965877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f040c06ad70 0x7f040c155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.755+0000 7f0411026700 1 -- 172.21.15.7:0/1764965877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040800b810 con 0x7f040c0cccf0 2024-09-13T18:54:22.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.759+0000 7f0411026700 1 --2- 172.21.15.7:0/1764965877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f040c0cccf0 0x7f040c071410 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009fd0 tx=0x7f03fc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.759+0000 7f04027fc700 1 -- 172.21.15.7:0/1764965877 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc0095a0 con 0x7f040c0cccf0 2024-09-13T18:54:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.759+0000 7f04027fc700 1 -- 172.21.15.7:0/1764965877 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03fc013680 con 0x7f040c0cccf0 2024-09-13T18:54:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.759+0000 7f0412829700 1 -- 172.21.15.7:0/1764965877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f040c0c4f50 con 0x7f040c0cccf0 2024-09-13T18:54:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.759+0000 7f04027fc700 1 -- 172.21.15.7:0/1764965877 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc013a10 con 0x7f040c0cccf0 2024-09-13T18:54:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.759+0000 7f0412829700 1 -- 172.21.15.7:0/1764965877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f040c0c5510 con 0x7f040c0cccf0 2024-09-13T18:54:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.759+0000 7f0412829700 1 -- 172.21.15.7:0/1764965877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f040c067830 con 0x7f040c0cccf0 2024-09-13T18:54:22.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.759+0000 7f04027fc700 1 -- 172.21.15.7:0/1764965877 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f03fc013b70 con 0x7f040c0cccf0 2024-09-13T18:54:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.759+0000 7f04027fc700 1 --2- 172.21.15.7:0/1764965877 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f03f0066760 0x7f03f0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.763+0000 7f0411827700 1 --2- 172.21.15.7:0/1764965877 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f03f0066760 0x7f03f0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:22.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.763+0000 7f04027fc700 1 -- 172.21.15.7:0/1764965877 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f03fc083f50 con 0x7f040c0cccf0 2024-09-13T18:54:22.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.763+0000 7f0411827700 1 --2- 172.21.15.7:0/1764965877 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f03f0066760 0x7f03f0068c20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f040800bfd0 tx=0x7f040801b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:22.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.767+0000 7f04027fc700 1 -- 172.21.15.7:0/1764965877 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f03fc0512f0 con 0x7f040c0cccf0 2024-09-13T18:54:22.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.979+0000 7f0412829700 1 -- 172.21.15.7:0/1764965877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f040c06b1b0 con 0x7f040c0cccf0 2024-09-13T18:54:22.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.979+0000 7f04027fc700 1 -- 172.21.15.7:0/1764965877 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f03fc054910 con 0x7f040c0cccf0 2024-09-13T18:54:22.985 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:22.985 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1829974483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1829974483},{"type":"v1","addr":"172.21.15.171:6835","nonce":1829974483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1948505526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1948505526},{"type":"v1","addr":"172.21.15.171:6837","nonce":1948505526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3617964311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3617964311},{"type":"v1","addr":"172.21.15.7:6833","nonce":3617964311}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:21.419627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/3768750832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3768750832},{"type":"v1","addr":"172.21.15.7:6835","nonce":3768750832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.983+0000 7f0412829700 1 -- 172.21.15.7:0/1764965877 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f03f0066760 msgr2=0x7f03f0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.983+0000 7f0412829700 1 --2- 172.21.15.7:0/1764965877 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f03f0066760 0x7f03f0068c20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f040800bfd0 tx=0x7f040801b040 comp rx=0 tx=0).stop 2024-09-13T18:54:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.983+0000 7f0412829700 1 -- 172.21.15.7:0/1764965877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f040c0cccf0 msgr2=0x7f040c071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.983+0000 7f0412829700 1 --2- 172.21.15.7:0/1764965877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f040c0cccf0 0x7f040c071410 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009fd0 tx=0x7f03fc00eb20 comp rx=0 tx=0).stop 2024-09-13T18:54:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.983+0000 7f0412829700 1 -- 172.21.15.7:0/1764965877 shutdown_connections 2024-09-13T18:54:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.983+0000 7f0412829700 1 --2- 172.21.15.7:0/1764965877 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f03f0066760 0x7f03f0068c20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.983+0000 7f0412829700 1 --2- 172.21.15.7:0/1764965877 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f040c15d320 0x7f040c071950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.983+0000 7f0412829700 1 --2- 172.21.15.7:0/1764965877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f040c0cccf0 0x7f040c071410 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.983+0000 7f0412829700 1 --2- 172.21.15.7:0/1764965877 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f040c06ad70 0x7f040c155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.983+0000 7f0412829700 1 -- 172.21.15.7:0/1764965877 >> 172.21.15.7:0/1764965877 conn(0x7f040c0cf120 msgr2=0x7f040c0ca0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.983+0000 7f0412829700 1 -- 172.21.15.7:0/1764965877 shutdown_connections 2024-09-13T18:54:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:22.983+0000 7f0412829700 1 -- 172.21.15.7:0/1764965877 wait complete. 2024-09-13T18:54:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-09-13T18:54:23.404 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T18:54:09.004150+0000', 'modified': '2024-09-13T18:54:21.419627+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 55, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5910}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5910': {'gid': 5910, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.7:6835/3768750832', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6834', 'nonce': 3768750832}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 3768750832}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T18:54:23.404 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T18:54:23.404 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-13T18:54:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.527+0000 7f96bf321700 1 Processor -- start 2024-09-13T18:54:23.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.527+0000 7f96bf321700 1 -- start start 2024-09-13T18:54:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.527+0000 7f96bf321700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f96b806d9f0 0x7f96b806de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.527+0000 7f96bf321700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f96b80b2de0 0x7f96b80b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.527+0000 7f96bf321700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b806d560 con 0x7f96b8159940 2024-09-13T18:54:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.527+0000 7f96bf321700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b80b37d0 con 0x7f96b80b2de0 2024-09-13T18:54:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.527+0000 7f96bf321700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b806c740 con 0x7f96b806d9f0 2024-09-13T18:54:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.527+0000 7f96be31f700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f96b8159940 0x7f96b8066110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53014/0 (socket says 172.21.15.7:53014) 2024-09-13T18:54:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.527+0000 7f96be31f700 1 -- 172.21.15.7:0/895859187 learned_addr learned my addr 172.21.15.7:0/895859187 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96beb20700 1 --2- 172.21.15.7:0/895859187 >> v2:172.21.15.171:3300/0 conn(0x7f96b80b2de0 0x7f96b80b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bdb1e700 1 --2- 172.21.15.7:0/895859187 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f96b806d9f0 0x7f96b806de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 -- 172.21.15.7:0/895859187 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974935564 0 0) 0x7f96b806d560 con 0x7f96b8159940 2024-09-13T18:54:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 -- 172.21.15.7:0/895859187 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96a40035d0 con 0x7f96b8159940 2024-09-13T18:54:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 -- 172.21.15.7:0/895859187 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 901408030 0 0) 0x7f96a40035d0 con 0x7f96b8159940 2024-09-13T18:54:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 -- 172.21.15.7:0/895859187 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f96b806d9f0 msgr2=0x7f96b806de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 --2- 172.21.15.7:0/895859187 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f96b806d9f0 0x7f96b806de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 -- 172.21.15.7:0/895859187 >> v2:172.21.15.171:3300/0 conn(0x7f96b80b2de0 msgr2=0x7f96b80b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 --2- 172.21.15.7:0/895859187 >> v2:172.21.15.171:3300/0 conn(0x7f96b80b2de0 0x7f96b80b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 -- 172.21.15.7:0/895859187 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b806c950 con 0x7f96b8159940 2024-09-13T18:54:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 -- 172.21.15.7:0/895859187 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7f96b4001670 con 0x7f96b8159940 2024-09-13T18:54:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 -- 172.21.15.7:0/895859187 >> v1:172.21.15.7:6789/0 conn(0x7f96b8159940 legacy=0x7f96b8066110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 --2- 172.21.15.7:0/895859187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a4003a20 0x7f96a4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 --2- 172.21.15.7:0/895859187 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96a4004870 0x7f96a4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 --2- 172.21.15.7:0/895859187 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f96b806d9f0 0x7f96a4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 -- 172.21.15.7:0/895859187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b80b37d0 con 0x7f96a4003a20 2024-09-13T18:54:23.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 -- 172.21.15.7:0/895859187 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b806c740 con 0x7f96a4004870 2024-09-13T18:54:23.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 -- 172.21.15.7:0/895859187 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a400ba30 con 0x7f96b806d9f0 2024-09-13T18:54:23.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96be31f700 1 --2- 172.21.15.7:0/895859187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a4003a20 0x7f96a4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bdb1e700 1 --2- 172.21.15.7:0/895859187 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96a4004870 0x7f96a4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96beb20700 1 --2- 172.21.15.7:0/895859187 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f96b806d9f0 0x7f96a4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96be31f700 1 -- 172.21.15.7:0/895859187 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f96b806d9f0 msgr2=0x7f96a4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:23.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96be31f700 1 --2- 172.21.15.7:0/895859187 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f96b806d9f0 0x7f96a4005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96be31f700 1 -- 172.21.15.7:0/895859187 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96a4004870 msgr2=0x7f96a4004cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:23.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96be31f700 1 --2- 172.21.15.7:0/895859187 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96a4004870 0x7f96a4004cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96be31f700 1 -- 172.21.15.7:0/895859187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a40035d0 con 0x7f96a4003a20 2024-09-13T18:54:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bdb1e700 1 --2- 172.21.15.7:0/895859187 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96a4004870 0x7f96a4004cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96beb20700 1 --2- 172.21.15.7:0/895859187 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f96b806d9f0 0x7f96a4005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96be31f700 1 --2- 172.21.15.7:0/895859187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a4003a20 0x7f96a4003e40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f96b400c2f0 tx=0x7f96a40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459e43689c3f43b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 -- 172.21.15.7:0/895859187 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4007c80 con 0x7f96a4003a20 2024-09-13T18:54:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 -- 172.21.15.7:0/895859187 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b400ce90 con 0x7f96a4003a20 2024-09-13T18:54:23.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bd31d700 1 -- 172.21.15.7:0/895859187 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4005bc0 con 0x7f96a4003a20 2024-09-13T18:54:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bf321700 1 -- 172.21.15.7:0/895859187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a4003a20 msgr2=0x7f96a4003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bf321700 1 --2- 172.21.15.7:0/895859187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a4003a20 0x7f96a4003e40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f96b400c2f0 tx=0x7f96a40058a0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96beb20700 1 -- 172.21.15.7:0/895859187 reap_dead start 2024-09-13T18:54:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bf321700 1 -- 172.21.15.7:0/895859187 shutdown_connections 2024-09-13T18:54:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bf321700 1 -- 172.21.15.7:0/895859187 >> 172.21.15.7:0/895859187 conn(0x7f96b80cf300 msgr2=0x7f96b80cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bf321700 1 -- 172.21.15.7:0/895859187 shutdown_connections 2024-09-13T18:54:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.531+0000 7f96bf321700 1 -- 172.21.15.7:0/895859187 wait complete. 2024-09-13T18:54:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96bf321700 1 Processor -- start 2024-09-13T18:54:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96bf321700 1 -- start start 2024-09-13T18:54:23.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96bf321700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a4003a20 0x7f96b814bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96bf321700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f96b80ae4a0 0x7f96b806a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96bf321700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b80ae920 0x7f96b806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96bf321700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b80aa800 con 0x7f96a4003a20 2024-09-13T18:54:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96bf321700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b80ac3d0 con 0x7f96b80ae920 2024-09-13T18:54:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96bf321700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b80ac620 con 0x7f96b80ae4a0 2024-09-13T18:54:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96be31f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a4003a20 0x7f96b814bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96be31f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a4003a20 0x7f96b814bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36888/0 (socket says 172.21.15.7:36888) 2024-09-13T18:54:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96be31f700 1 -- 172.21.15.7:0/264795214 learned_addr learned my addr 172.21.15.7:0/264795214 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96bdb1e700 1 --2- 172.21.15.7:0/264795214 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f96b80ae4a0 0x7f96b806a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96beb20700 1 --2- 172.21.15.7:0/264795214 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b80ae920 0x7f96b806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96be31f700 1 -- 172.21.15.7:0/264795214 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f96b80ae4a0 msgr2=0x7f96b806a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96be31f700 1 --2- 172.21.15.7:0/264795214 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f96b80ae4a0 0x7f96b806a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96be31f700 1 -- 172.21.15.7:0/264795214 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b80ae920 msgr2=0x7f96b806a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:23.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96be31f700 1 --2- 172.21.15.7:0/264795214 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b80ae920 0x7f96b806a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96be31f700 1 -- 172.21.15.7:0/264795214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b80ac900 con 0x7f96a4003a20 2024-09-13T18:54:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96be31f700 1 --2- 172.21.15.7:0/264795214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a4003a20 0x7f96b814bf40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f96a4004750 tx=0x7f96b4000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96aeffd700 1 -- 172.21.15.7:0/264795214 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b40070d0 con 0x7f96a4003a20 2024-09-13T18:54:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96aeffd700 1 -- 172.21.15.7:0/264795214 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b4007250 con 0x7f96a4003a20 2024-09-13T18:54:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96bf321700 1 -- 172.21.15.7:0/264795214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b80acbe0 con 0x7f96a4003a20 2024-09-13T18:54:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96aeffd700 1 -- 172.21.15.7:0/264795214 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4007600 con 0x7f96a4003a20 2024-09-13T18:54:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.535+0000 7f96bf321700 1 -- 172.21.15.7:0/264795214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b81f0830 con 0x7f96a4003a20 2024-09-13T18:54:23.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.539+0000 7f96aeffd700 1 -- 172.21.15.7:0/264795214 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f96b4019030 con 0x7f96a4003a20 2024-09-13T18:54:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.539+0000 7f96aeffd700 1 --2- 172.21.15.7:0/264795214 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f969c066730 0x7f969c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.539+0000 7f96aeffd700 1 -- 172.21.15.7:0/264795214 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f96b40854e0 con 0x7f96a4003a20 2024-09-13T18:54:23.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.539+0000 7f96bdb1e700 1 --2- 172.21.15.7:0/264795214 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f969c066730 0x7f969c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96bdb1e700 1 --2- 172.21.15.7:0/264795214 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f969c066730 0x7f969c068bf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f96b80cd6f0 tx=0x7f96a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96c3834740 1 Processor -- start 2024-09-13T18:54:23.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96c3834740 1 -- start start 2024-09-13T18:54:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96c3834740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2247230 0x2247610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96c3834740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2225e90 0x220f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96c3834740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2247b50 0x220fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96c3834740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x216ed80 con 0x2247230 2024-09-13T18:54:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96c3834740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x216c510 con 0x2247b50 2024-09-13T18:54:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96c3834740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x2167e00 con 0x2225e90 2024-09-13T18:54:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96be31f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2247230 0x2247610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96be31f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2247230 0x2247610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36904/0 (socket says 172.21.15.7:36904) 2024-09-13T18:54:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96be31f700 1 -- 172.21.15.7:0/2900310272 learned_addr learned my addr 172.21.15.7:0/2900310272 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96bdb1e700 1 --2- 172.21.15.7:0/2900310272 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2225e90 0x220f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96beb20700 1 --2- 172.21.15.7:0/2900310272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2247b50 0x220fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96be31f700 1 -- 172.21.15.7:0/2900310272 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2225e90 msgr2=0x220f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96be31f700 1 --2- 172.21.15.7:0/2900310272 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2225e90 0x220f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96be31f700 1 -- 172.21.15.7:0/2900310272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2247b50 msgr2=0x220fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96be31f700 1 --2- 172.21.15.7:0/2900310272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2247b50 0x220fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96be31f700 1 -- 172.21.15.7:0/2900310272 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1fda960 con 0x2247230 2024-09-13T18:54:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96bdb1e700 1 --2- 172.21.15.7:0/2900310272 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2225e90 0x220f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:23.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96beb20700 1 --2- 172.21.15.7:0/2900310272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2247b50 0x220fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:23.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f96be31f700 1 --2- 172.21.15.7:0/2900310272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2247230 0x2247610 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f96b4015040 tx=0x7f96b4055ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2e4bcc4c2f6266 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f969bfff700 1 -- 172.21.15.7:0/2900310272 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b405a2f0 con 0x2247230 2024-09-13T18:54:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f969bfff700 1 -- 172.21.15.7:0/2900310272 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b405e8b0 con 0x2247230 2024-09-13T18:54:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.543+0000 7f969bfff700 1 -- 172.21.15.7:0/2900310272 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4054910 con 0x2247230 2024-09-13T18:54:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96c3834740 1 -- 172.21.15.7:0/2900310272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2247230 msgr2=0x2247610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96c3834740 1 --2- 172.21.15.7:0/2900310272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2247230 0x2247610 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f96b4015040 tx=0x7f96b4055ed0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96c3834740 1 -- 172.21.15.7:0/2900310272 shutdown_connections 2024-09-13T18:54:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96c3834740 1 --2- 172.21.15.7:0/2900310272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2247b50 0x220fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96c3834740 1 --2- 172.21.15.7:0/2900310272 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2225e90 0x220f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96c3834740 1 --2- 172.21.15.7:0/2900310272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2247230 0x2247610 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96c3834740 1 -- 172.21.15.7:0/2900310272 >> 172.21.15.7:0/2900310272 conn(0x1f8f4d0 msgr2=0x2228e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:23.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96c3834740 1 -- 172.21.15.7:0/2900310272 shutdown_connections 2024-09-13T18:54:23.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96c3834740 1 -- 172.21.15.7:0/2900310272 wait complete. 2024-09-13T18:54:23.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96c3834740 1 -- start start 2024-09-13T18:54:23.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96c3834740 1 Processor -- start 2024-09-13T18:54:23.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96c3834740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2225e90 0x2246a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96c3834740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2247230 0x2225540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96c3834740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2247b50 0x21ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96c3834740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x2125730 con 0x2225e90 2024-09-13T18:54:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96c3834740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x212b5b0 con 0x2247b50 2024-09-13T18:54:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96c3834740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x2013aa0 con 0x2247230 2024-09-13T18:54:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96c3834740 10 client.0 _ll_register_callbacks cb 0x2161610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:54:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96be31f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2225e90 0x2246a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96be31f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2225e90 0x2246a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36910/0 (socket says 172.21.15.7:36910) 2024-09-13T18:54:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96be31f700 1 -- 172.21.15.7:0/2065127534 learned_addr learned my addr 172.21.15.7:0/2065127534 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96beb20700 1 --2- 172.21.15.7:0/2065127534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2247b50 0x21ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.547+0000 7f96bdb1e700 1 --2- 172.21.15.7:0/2065127534 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2247230 0x2225540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96be31f700 1 -- 172.21.15.7:0/2065127534 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2247230 msgr2=0x2225540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96be31f700 1 --2- 172.21.15.7:0/2065127534 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2247230 0x2225540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96be31f700 1 -- 172.21.15.7:0/2065127534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2247b50 msgr2=0x21ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96be31f700 1 --2- 172.21.15.7:0/2065127534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2247b50 0x21ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96be31f700 1 -- 172.21.15.7:0/2065127534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b81f0830 con 0x2225e90 2024-09-13T18:54:23.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96be31f700 1 --2- 172.21.15.7:0/2065127534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2225e90 0x2246a00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f96b4056270 tx=0x7f96b4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f82bbad01ec025 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:23.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96997fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-13T18:54:23.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96997fa700 1 -- 172.21.15.7:0/2065127534 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4070070 con 0x2225e90 2024-09-13T18:54:23.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96997fa700 1 -- 172.21.15.7:0/2065127534 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f96b4054910 con 0x2225e90 2024-09-13T18:54:23.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96c3834740 1 -- 172.21.15.7:0/2065127534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1f9c1e0 con 0x2225e90 2024-09-13T18:54:23.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96997fa700 1 -- 172.21.15.7:0/2065127534 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96b4007760 con 0x2225e90 2024-09-13T18:54:23.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96c3834740 10 client.5952 fetch_fsmap learned FSMap version 36 2024-09-13T18:54:23.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96c3834740 1 -- 172.21.15.7:0/2065127534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x212e220 con 0x2225e90 2024-09-13T18:54:23.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96997fa700 1 -- 172.21.15.7:0/2065127534 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1f9c1e0 con 0x2225e90 2024-09-13T18:54:23.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96997fa700 1 -- 172.21.15.7:0/2065127534 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 36) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f96b40567d0 con 0x2225e90 2024-09-13T18:54:23.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96c3834740 10 client.5952 fetch_fsmap finished waiting for FSMap version 36 2024-09-13T18:54:23.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96c3834740 10 client.5952 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T18:54:23.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96c3834740 20 client.5952 populate_metadata read hostname 'smithi007' 2024-09-13T18:54:23.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96c3834740 1 --2- 172.21.15.7:0/2065127534 >> [v2:172.21.15.7:6834/3768750832,v1:172.21.15.7:6835/3768750832] conn(0x224d5a0 0x224d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96c3834740 4 client.5952 mds_command: new command op to 5910 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T18:54:23.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96c3834740 1 -- 172.21.15.7:0/2065127534 --> [v2:172.21.15.7:6834/3768750832,v1:172.21.15.7:6835/3768750832] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1fe6460 con 0x224d5a0 2024-09-13T18:54:23.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96bdb1e700 1 --2- 172.21.15.7:0/2065127534 >> [v2:172.21.15.7:6834/3768750832,v1:172.21.15.7:6835/3768750832] conn(0x224d5a0 0x224d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96997fa700 10 client.5952 ms_handle_connect on v2:172.21.15.7:6834/3768750832 2024-09-13T18:54:23.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96bdb1e700 1 --2- 172.21.15.7:0/2065127534 >> [v2:172.21.15.7:6834/3768750832,v1:172.21.15.7:6835/3768750832] conn(0x224d5a0 0x224d980 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad79173a9be6658a server_cookie=a687a1278ab0d2d in_seq=0 out_seq=0 2024-09-13T18:54:23.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96997fa700 1 -- 172.21.15.7:0/2065127534 <== mds.0 v2:172.21.15.7:6834/3768750832 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x1fe6460 con 0x224d5a0 2024-09-13T18:54:23.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96997fa700 10 client.5952 handle_command_reply: tid=0 2024-09-13T18:54:23.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96c3834740 1 client.5952 shutdown 2024-09-13T18:54:23.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96c3834740 1 -- 172.21.15.7:0/2065127534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2225e90 msgr2=0x2246a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96c3834740 1 --2- 172.21.15.7:0/2065127534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2225e90 0x2246a00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f96b4056270 tx=0x7f96b4005f90 comp rx=0 tx=0).stop 2024-09-13T18:54:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96c3834740 1 -- 172.21.15.7:0/2065127534 shutdown_connections 2024-09-13T18:54:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96c3834740 1 --2- 172.21.15.7:0/2065127534 >> [v2:172.21.15.7:6834/3768750832,v1:172.21.15.7:6835/3768750832] conn(0x224d5a0 0x224d980 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96c3834740 1 --2- 172.21.15.7:0/2065127534 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2247b50 0x21ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96997fa700 0 client.5952 ms_handle_reset on v2:172.21.15.7:6834/3768750832 2024-09-13T18:54:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96c3834740 1 --2- 172.21.15.7:0/2065127534 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2247230 0x2225540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96c3834740 1 --2- 172.21.15.7:0/2065127534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2225e90 0x2246a00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.551+0000 7f96c3834740 1 -- 172.21.15.7:0/2065127534 >> 172.21.15.7:0/2065127534 conn(0x1f8f4d0 msgr2=0x21fb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.555+0000 7f96c3834740 1 -- 172.21.15.7:0/2065127534 shutdown_connections 2024-09-13T18:54:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.555+0000 7f96c3834740 1 -- 172.21.15.7:0/2065127534 wait complete. 2024-09-13T18:54:23.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.555+0000 7f96c3834740 20 client.5952 trim_cache size 0 max 16384 2024-09-13T18:54:23.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.575+0000 7f96c3834740 1 Processor -- start 2024-09-13T18:54:23.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.575+0000 7f96c3834740 1 -- start start 2024-09-13T18:54:23.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96c3834740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1f8f4d0 0x2241110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96c3834740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2225e90 0x2241650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96c3834740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2247230 0x2241b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96c3834740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x2196ec0 con 0x2225e90 2024-09-13T18:54:23.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96c3834740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x2167e00 con 0x1f8f4d0 2024-09-13T18:54:23.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96c3834740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x2125730 con 0x2247230 2024-09-13T18:54:23.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96bdb1e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2225e90 0x2241650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96bdb1e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2225e90 0x2241650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36912/0 (socket says 172.21.15.7:36912) 2024-09-13T18:54:23.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96bdb1e700 1 -- 172.21.15.7:0/2421182791 learned_addr learned my addr 172.21.15.7:0/2421182791 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:23.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96beb20700 1 --2- 172.21.15.7:0/2421182791 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2247230 0x2241b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96be31f700 1 --2- 172.21.15.7:0/2421182791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1f8f4d0 0x2241110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96bdb1e700 1 -- 172.21.15.7:0/2421182791 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2247230 msgr2=0x2241b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:23.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96bdb1e700 1 --2- 172.21.15.7:0/2421182791 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2247230 0x2241b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96bdb1e700 1 -- 172.21.15.7:0/2421182791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1f8f4d0 msgr2=0x2241110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:23.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96bdb1e700 1 --2- 172.21.15.7:0/2421182791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1f8f4d0 0x2241110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96bdb1e700 1 -- 172.21.15.7:0/2421182791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2201220 con 0x2225e90 2024-09-13T18:54:23.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96bdb1e700 1 --2- 172.21.15.7:0/2421182791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2225e90 0x2241650 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f96a8012e20 tx=0x7f96a8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd82915498355ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:23.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96acff9700 1 -- 172.21.15.7:0/2421182791 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8029040 con 0x2225e90 2024-09-13T18:54:23.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96acff9700 1 -- 172.21.15.7:0/2421182791 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a802cbe0 con 0x2225e90 2024-09-13T18:54:23.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96acff9700 1 -- 172.21.15.7:0/2421182791 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a802c330 con 0x2225e90 2024-09-13T18:54:23.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96c3834740 1 -- 172.21.15.7:0/2421182791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2225e90 msgr2=0x2241650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:23.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96c3834740 1 --2- 172.21.15.7:0/2421182791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2225e90 0x2241650 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f96a8012e20 tx=0x7f96a8015000 comp rx=0 tx=0).stop 2024-09-13T18:54:23.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96c3834740 1 -- 172.21.15.7:0/2421182791 shutdown_connections 2024-09-13T18:54:23.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96c3834740 1 --2- 172.21.15.7:0/2421182791 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x2247230 0x2241b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96c3834740 1 --2- 172.21.15.7:0/2421182791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2225e90 0x2241650 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96c3834740 1 --2- 172.21.15.7:0/2421182791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x1f8f4d0 0x2241110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96c3834740 1 -- 172.21.15.7:0/2421182791 >> 172.21.15.7:0/2421182791 conn(0x2225820 msgr2=0x21ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:23.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96c3834740 1 -- 172.21.15.7:0/2421182791 shutdown_connections 2024-09-13T18:54:23.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.579+0000 7f96c3834740 1 -- 172.21.15.7:0/2421182791 wait complete. 2024-09-13T18:54:23.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96c3834740 1 -- start start 2024-09-13T18:54:23.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96c3834740 1 Processor -- start 2024-09-13T18:54:23.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96c3834740 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1f8f4d0 0x22abc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96c3834740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2225e90 0x2227a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96c3834740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2247230 0x2227fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96c3834740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x216ed80 con 0x2247230 2024-09-13T18:54:23.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96c3834740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x2268a60 con 0x2225e90 2024-09-13T18:54:23.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96c3834740 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x2257260 con 0x1f8f4d0 2024-09-13T18:54:23.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96c3834740 10 client.0 _ll_register_callbacks cb 0x2119ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T18:54:23.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96beb20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2247230 0x2227fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96beb20700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2247230 0x2227fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36918/0 (socket says 172.21.15.7:36918) 2024-09-13T18:54:23.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96beb20700 1 -- 172.21.15.7:0/2809231346 learned_addr learned my addr 172.21.15.7:0/2809231346 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:23.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96be31f700 1 --2- 172.21.15.7:0/2809231346 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1f8f4d0 0x22abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96bdb1e700 1 --2- 172.21.15.7:0/2809231346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2225e90 0x2227a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96beb20700 1 -- 172.21.15.7:0/2809231346 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1f8f4d0 msgr2=0x22abc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:23.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96beb20700 1 --2- 172.21.15.7:0/2809231346 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1f8f4d0 0x22abc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96beb20700 1 -- 172.21.15.7:0/2809231346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2225e90 msgr2=0x2227a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:23.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96beb20700 1 --2- 172.21.15.7:0/2809231346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2225e90 0x2227a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96beb20700 1 -- 172.21.15.7:0/2809231346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a8012920 con 0x2247230 2024-09-13T18:54:23.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96be31f700 1 --2- 172.21.15.7:0/2809231346 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1f8f4d0 0x22abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:23.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96beb20700 1 --2- 172.21.15.7:0/2809231346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2247230 0x2227fd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f96b0004570 tx=0x7f96b00045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600cf796e6733fc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:23.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96997fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-13T18:54:23.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96997fa700 1 -- 172.21.15.7:0/2809231346 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b001d070 con 0x2247230 2024-09-13T18:54:23.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.583+0000 7f96997fa700 1 -- 172.21.15.7:0/2809231346 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f96b0023b30 con 0x2247230 2024-09-13T18:54:23.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 1 -- 172.21.15.7:0/2809231346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96b405e8b0 con 0x2247230 2024-09-13T18:54:23.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96997fa700 1 -- 172.21.15.7:0/2809231346 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96b000e040 con 0x2247230 2024-09-13T18:54:23.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 10 client.5958 fetch_fsmap learned FSMap version 36 2024-09-13T18:54:23.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 1 -- 172.21.15.7:0/2809231346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) v3 -- 0x2114450 con 0x2247230 2024-09-13T18:54:23.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96997fa700 1 -- 172.21.15.7:0/2809231346 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b405e8b0 con 0x2247230 2024-09-13T18:54:23.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96997fa700 1 -- 172.21.15.7:0/2809231346 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 36) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f96b0023cf0 con 0x2247230 2024-09-13T18:54:23.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 10 client.5958 fetch_fsmap finished waiting for FSMap version 36 2024-09-13T18:54:23.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 10 client.5958 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-13T18:54:23.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 20 client.5958 populate_metadata read hostname 'smithi007' 2024-09-13T18:54:23.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 1 --2- 172.21.15.7:0/2809231346 >> [v2:172.21.15.7:6834/3768750832,v1:172.21.15.7:6835/3768750832] conn(0x2247b50 0x2267590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 4 client.5958 mds_command: new command op to 5910 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T18:54:23.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 1 -- 172.21.15.7:0/2809231346 --> [v2:172.21.15.7:6834/3768750832,v1:172.21.15.7:6835/3768750832] -- command(tid 0: {"prefix": "status"}) v1 -- 0x200b1e0 con 0x2247b50 2024-09-13T18:54:23.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96be31f700 1 --2- 172.21.15.7:0/2809231346 >> [v2:172.21.15.7:6834/3768750832,v1:172.21.15.7:6835/3768750832] conn(0x2247b50 0x2267590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96997fa700 10 client.5958 ms_handle_connect on v2:172.21.15.7:6834/3768750832 2024-09-13T18:54:23.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96be31f700 1 --2- 172.21.15.7:0/2809231346 >> [v2:172.21.15.7:6834/3768750832,v1:172.21.15.7:6835/3768750832] conn(0x2247b50 0x2267590 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e702a64821dea7e8 server_cookie=10b9e372751fa968 in_seq=0 out_seq=0 2024-09-13T18:54:23.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96997fa700 1 -- 172.21.15.7:0/2809231346 <== mds.0 v2:172.21.15.7:6834/3768750832 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x200b1e0 con 0x2247b50 2024-09-13T18:54:23.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96997fa700 10 client.5958 handle_command_reply: tid=0 2024-09-13T18:54:23.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 1 client.5958 shutdown 2024-09-13T18:54:23.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 1 -- 172.21.15.7:0/2809231346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2247230 msgr2=0x2227fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:23.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 1 --2- 172.21.15.7:0/2809231346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2247230 0x2227fd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f96b0004570 tx=0x7f96b00045a0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 1 -- 172.21.15.7:0/2809231346 shutdown_connections 2024-09-13T18:54:23.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 1 --2- 172.21.15.7:0/2809231346 >> [v2:172.21.15.7:6834/3768750832,v1:172.21.15.7:6835/3768750832] conn(0x2247b50 0x2267590 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 1 --2- 172.21.15.7:0/2809231346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x2247230 0x2227fd0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 1 --2- 172.21.15.7:0/2809231346 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x2225e90 0x2227a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 1 --2- 172.21.15.7:0/2809231346 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x1f8f4d0 0x22abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96997fa700 0 client.5958 ms_handle_reset on v2:172.21.15.7:6834/3768750832 2024-09-13T18:54:23.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 1 -- 172.21.15.7:0/2809231346 >> 172.21.15.7:0/2809231346 conn(0x2225820 msgr2=0x22567a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:23.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 1 -- 172.21.15.7:0/2809231346 shutdown_connections 2024-09-13T18:54:23.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 1 -- 172.21.15.7:0/2809231346 wait complete. 2024-09-13T18:54:23.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96c3834740 20 client.5958 trim_cache size 0 max 16384 2024-09-13T18:54:23.606 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:54:23.606 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "33406d92-f401-438a-876e-a56de3962d8e", 2024-09-13T18:54:23.606 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-13T18:54:23.606 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 5910, 2024-09-13T18:54:23.606 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-13T18:54:23.606 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-13T18:54:23.607 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-13T18:54:23.607 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.1864793960000002, 2024-09-13T18:54:23.607 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 36, 2024-09-13T18:54:23.607 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 56, 2024-09-13T18:54:23.607 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 56, 2024-09-13T18:54:23.608 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 6.2041695389999996 2024-09-13T18:54:23.608 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:54:23.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96bf321700 1 -- 172.21.15.7:0/264795214 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f969c066730 msgr2=0x7f969c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:23.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96bf321700 1 --2- 172.21.15.7:0/264795214 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f969c066730 0x7f969c068bf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f96b80cd6f0 tx=0x7f96a8008040 comp rx=0 tx=0).stop 2024-09-13T18:54:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96bf321700 1 -- 172.21.15.7:0/264795214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a4003a20 msgr2=0x7f96b814bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96bf321700 1 --2- 172.21.15.7:0/264795214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a4003a20 0x7f96b814bf40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f96a4004750 tx=0x7f96b4000fe0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96bf321700 1 -- 172.21.15.7:0/264795214 shutdown_connections 2024-09-13T18:54:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96bf321700 1 --2- 172.21.15.7:0/264795214 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f969c066730 0x7f969c068bf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96bf321700 1 --2- 172.21.15.7:0/264795214 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96b80ae920 0x7f96b806a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96bf321700 1 --2- 172.21.15.7:0/264795214 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f96b80ae4a0 0x7f96b806a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96bf321700 1 --2- 172.21.15.7:0/264795214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a4003a20 0x7f96b814bf40 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96bf321700 1 -- 172.21.15.7:0/264795214 >> 172.21.15.7:0/264795214 conn(0x7f96b80cf300 msgr2=0x7f96b8158150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96bf321700 1 -- 172.21.15.7:0/264795214 shutdown_connections 2024-09-13T18:54:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.587+0000 7f96bf321700 1 -- 172.21.15.7:0/264795214 wait complete. 2024-09-13T18:54:23.613 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:23.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b55bcf700 1 Processor -- start 2024-09-13T18:54:23.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b55bcf700 1 -- start start 2024-09-13T18:54:23.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b55bcf700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f3b501599b0 0x7f3b500ada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b55bcf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3b500cabf0 0x7f3b50070740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b55bcf700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3b50070c80 0x7f3b5015d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b55bcf700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b5006c3c0 con 0x7f3b500cabf0 2024-09-13T18:54:23.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b55bcf700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b50071110 con 0x7f3b501599b0 2024-09-13T18:54:23.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b55bcf700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b500b1aa0 con 0x7f3b50070c80 2024-09-13T18:54:23.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b4ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3b500cabf0 0x7f3b50070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b4ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3b500cabf0 0x7f3b50070740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36920/0 (socket says 172.21.15.7:36920) 2024-09-13T18:54:23.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b4ffff700 1 -- 172.21.15.7:0/2367342551 learned_addr learned my addr 172.21.15.7:0/2367342551 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:23.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b553ce700 1 --2- 172.21.15.7:0/2367342551 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3b50070c80 0x7f3b5015d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b54bcd700 1 --2- 172.21.15.7:0/2367342551 >> v2:172.21.15.171:3300/0 conn(0x7f3b501599b0 0x7f3b500ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b4ffff700 1 -- 172.21.15.7:0/2367342551 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3b50070c80 msgr2=0x7f3b5015d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:23.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b4ffff700 1 --2- 172.21.15.7:0/2367342551 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3b50070c80 0x7f3b5015d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b4ffff700 1 -- 172.21.15.7:0/2367342551 >> v2:172.21.15.171:3300/0 conn(0x7f3b501599b0 msgr2=0x7f3b500ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:23.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b4ffff700 1 --2- 172.21.15.7:0/2367342551 >> v2:172.21.15.171:3300/0 conn(0x7f3b501599b0 0x7f3b500ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b4ffff700 1 -- 172.21.15.7:0/2367342551 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5006be40 con 0x7f3b500cabf0 2024-09-13T18:54:23.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b4ffff700 1 --2- 172.21.15.7:0/2367342551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b500cabf0 0x7f3b50070740 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3b5006b300 tx=0x7f3b3c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b467f7635ccba1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:23.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b4f7fe700 1 -- 172.21.15.7:0/2367342551 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c011830 con 0x7f3b500cabf0 2024-09-13T18:54:23.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b4f7fe700 1 -- 172.21.15.7:0/2367342551 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3c011990 con 0x7f3b500cabf0 2024-09-13T18:54:23.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b4f7fe700 1 -- 172.21.15.7:0/2367342551 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c011d20 con 0x7f3b500cabf0 2024-09-13T18:54:23.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b55bcf700 1 -- 172.21.15.7:0/2367342551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b500cabf0 msgr2=0x7f3b50070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:23.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b55bcf700 1 --2- 172.21.15.7:0/2367342551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b500cabf0 0x7f3b50070740 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3b5006b300 tx=0x7f3b3c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b55bcf700 1 -- 172.21.15.7:0/2367342551 shutdown_connections 2024-09-13T18:54:23.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b55bcf700 1 --2- 172.21.15.7:0/2367342551 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3b50070c80 0x7f3b5015d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b55bcf700 1 --2- 172.21.15.7:0/2367342551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b500cabf0 0x7f3b50070740 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b55bcf700 1 --2- 172.21.15.7:0/2367342551 >> v2:172.21.15.171:3300/0 conn(0x7f3b501599b0 0x7f3b500ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b55bcf700 1 -- 172.21.15.7:0/2367342551 >> 172.21.15.7:0/2367342551 conn(0x7f3b500b0570 msgr2=0x7f3b500b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:23.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b55bcf700 1 -- 172.21.15.7:0/2367342551 shutdown_connections 2024-09-13T18:54:23.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.735+0000 7f3b55bcf700 1 -- 172.21.15.7:0/2367342551 wait complete. 2024-09-13T18:54:23.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.739+0000 7f3b55bcf700 1 Processor -- start 2024-09-13T18:54:23.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.739+0000 7f3b55bcf700 1 -- start start 2024-09-13T18:54:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.739+0000 7f3b55bcf700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3b50070c80 0x7f3b50166510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.739+0000 7f3b55bcf700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b500cabf0 0x7f3b50151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.739+0000 7f3b55bcf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b501599b0 0x7f3b50150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.739+0000 7f3b55bcf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b50151680 con 0x7f3b501599b0 2024-09-13T18:54:23.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.739+0000 7f3b55bcf700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b50150f60 con 0x7f3b500cabf0 2024-09-13T18:54:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.739+0000 7f3b55bcf700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5015e620 con 0x7f3b50070c80 2024-09-13T18:54:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.739+0000 7f3b553ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b501599b0 0x7f3b50150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.739+0000 7f3b4ffff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b500cabf0 0x7f3b50151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.739+0000 7f3b54bcd700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3b50070c80 0x7f3b50166510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.739+0000 7f3b553ce700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b501599b0 0x7f3b50150a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36930/0 (socket says 172.21.15.7:36930) 2024-09-13T18:54:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.739+0000 7f3b553ce700 1 -- 172.21.15.7:0/893122067 learned_addr learned my addr 172.21.15.7:0/893122067 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.743+0000 7f3b54bcd700 1 -- 172.21.15.7:0/893122067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b500cabf0 msgr2=0x7f3b50151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.743+0000 7f3b54bcd700 1 --2- 172.21.15.7:0/893122067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b500cabf0 0x7f3b50151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.743+0000 7f3b54bcd700 1 -- 172.21.15.7:0/893122067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b501599b0 msgr2=0x7f3b50150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.743+0000 7f3b54bcd700 1 --2- 172.21.15.7:0/893122067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b501599b0 0x7f3b50150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.743+0000 7f3b54bcd700 1 -- 172.21.15.7:0/893122067 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c009570 con 0x7f3b50070c80 2024-09-13T18:54:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.743+0000 7f3b553ce700 1 --2- 172.21.15.7:0/893122067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b501599b0 0x7f3b50150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.743+0000 7f3b54bcd700 1 --2- 172.21.15.7:0/893122067 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3b50070c80 0x7f3b50166510 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400e870 tx=0x7f3b4400eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:23.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.743+0000 7f3b4d7fa700 1 -- 172.21.15.7:0/893122067 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b44013680 con 0x7f3b50070c80 2024-09-13T18:54:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.743+0000 7f3b4d7fa700 1 -- 172.21.15.7:0/893122067 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b440137e0 con 0x7f3b50070c80 2024-09-13T18:54:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.743+0000 7f3b4d7fa700 1 -- 172.21.15.7:0/893122067 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b44013b70 con 0x7f3b50070c80 2024-09-13T18:54:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.743+0000 7f3b55bcf700 1 -- 172.21.15.7:0/893122067 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5015e900 con 0x7f3b50070c80 2024-09-13T18:54:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.743+0000 7f3b55bcf700 1 -- 172.21.15.7:0/893122067 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5015eec0 con 0x7f3b50070c80 2024-09-13T18:54:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.743+0000 7f3b55bcf700 1 -- 172.21.15.7:0/893122067 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b50070740 con 0x7f3b50070c80 2024-09-13T18:54:23.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.743+0000 7f3b4d7fa700 1 -- 172.21.15.7:0/893122067 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3b44013cd0 con 0x7f3b50070c80 2024-09-13T18:54:23.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.747+0000 7f3b4d7fa700 1 --2- 172.21.15.7:0/893122067 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3b34066730 0x7f3b34068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:23.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.747+0000 7f3b4d7fa700 1 -- 172.21.15.7:0/893122067 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3b440842c0 con 0x7f3b50070c80 2024-09-13T18:54:23.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.747+0000 7f3b4ffff700 1 --2- 172.21.15.7:0/893122067 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3b34066730 0x7f3b34068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:23.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.747+0000 7f3b4ffff700 1 --2- 172.21.15.7:0/893122067 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3b34066730 0x7f3b34068bf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c000c00 tx=0x7f3b3c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:23.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.755+0000 7f3b4d7fa700 1 -- 172.21.15.7:0/893122067 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3b440516b0 con 0x7f3b50070c80 2024-09-13T18:54:23.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.963+0000 7f3b55bcf700 1 -- 172.21.15.7:0/893122067 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b500b1290 con 0x7f3b50070c80 2024-09-13T18:54:23.968 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:23.969 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1829974483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1829974483},{"type":"v1","addr":"172.21.15.171:6835","nonce":1829974483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1948505526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1948505526},{"type":"v1","addr":"172.21.15.171:6837","nonce":1948505526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3617964311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3617964311},{"type":"v1","addr":"172.21.15.7:6833","nonce":3617964311}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:21.419627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/3768750832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3768750832},{"type":"v1","addr":"172.21.15.7:6835","nonce":3768750832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.963+0000 7f3b4d7fa700 1 -- 172.21.15.7:0/893122067 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f3b44017090 con 0x7f3b50070c80 2024-09-13T18:54:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.967+0000 7f3b55bcf700 1 -- 172.21.15.7:0/893122067 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3b34066730 msgr2=0x7f3b34068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.967+0000 7f3b55bcf700 1 --2- 172.21.15.7:0/893122067 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3b34066730 0x7f3b34068bf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c000c00 tx=0x7f3b3c008030 comp rx=0 tx=0).stop 2024-09-13T18:54:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.967+0000 7f3b55bcf700 1 -- 172.21.15.7:0/893122067 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3b50070c80 msgr2=0x7f3b50166510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.967+0000 7f3b55bcf700 1 --2- 172.21.15.7:0/893122067 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3b50070c80 0x7f3b50166510 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400e870 tx=0x7f3b4400eb80 comp rx=0 tx=0).stop 2024-09-13T18:54:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.967+0000 7f3b55bcf700 1 -- 172.21.15.7:0/893122067 shutdown_connections 2024-09-13T18:54:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.967+0000 7f3b55bcf700 1 --2- 172.21.15.7:0/893122067 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3b34066730 0x7f3b34068bf0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.967+0000 7f3b55bcf700 1 --2- 172.21.15.7:0/893122067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b501599b0 0x7f3b50150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.967+0000 7f3b55bcf700 1 --2- 172.21.15.7:0/893122067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b500cabf0 0x7f3b50151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.967+0000 7f3b55bcf700 1 --2- 172.21.15.7:0/893122067 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3b50070c80 0x7f3b50166510 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.967+0000 7f3b55bcf700 1 -- 172.21.15.7:0/893122067 >> 172.21.15.7:0/893122067 conn(0x7f3b500b0570 msgr2=0x7f3b5015d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.967+0000 7f3b55bcf700 1 -- 172.21.15.7:0/893122067 shutdown_connections 2024-09-13T18:54:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:23.967+0000 7f3b55bcf700 1 -- 172.21.15.7:0/893122067 wait complete. 2024-09-13T18:54:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-09-13T18:54:23.983 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T18:54:24.100 INFO:teuthology.orchestra.run.smithi007.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":41,"msgr_send_messages":33,"msgr_recv_bytes":112044,"msgr_send_bytes":11543,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.009849594,"msgr_running_send_time":0.001834864,"msgr_running_recv_time":0.010353565,"msgr_running_fast_dispatch_time":0.000687357,"msgr_send_messages_queue_lat":{"avgcount":33,"sum":0.004562231,"avgtime":0.000138249},"msgr_handle_ack_lat":{"avgcount":10,"sum":0.000004276,"avgtime":0.000000427},"msgr_recv_encrypted_bytes":104464,"msgr_send_encrypted_bytes":5888},"AsyncMessenger::Worker-1":{"msgr_recv_messages":17,"msgr_send_messages":21,"msgr_recv_bytes":10129,"msgr_send_bytes":95240,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.004516864,"msgr_running_send_time":0.000978655,"msgr_running_recv_time":0.002775052,"msgr_running_fast_dispatch_time":0.000501745,"msgr_send_messages_queue_lat":{"avgcount":19,"sum":0.003412686,"avgtime":0.000179615},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":88976},"AsyncMessenger::Worker-2":{"msgr_recv_messages":21,"msgr_send_messages":23,"msgr_recv_bytes":13455,"msgr_send_bytes":26377,"msgr_created_connections":7,"msgr_active_connections":6,"msgr_running_total_time":0.004904050,"msgr_running_send_time":0.000773106,"msgr_running_recv_time":0.003124703,"msgr_running_fast_dispatch_time":0.000632627,"msgr_send_messages_queue_lat":{"avgcount":22,"sum":0.006890593,"avgtime":0.000313208},"msgr_handle_ack_lat":{"avgcount":2,"sum":0.000000328,"avgtime":0.000000164},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":48,"sum":0.005612181,"avgtime":0.000116920}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000190271,"avgtime":0.000047567}},"mds":{"request":3,"reply":1,"reply_latency":{"avgcount":1,"sum":0.000473250,"avgtime":0.000473250},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":10,"inodes_top":7,"inodes_bottom":3,"inodes_pin_tail":0,"inodes_pinned":10,"inodes_expired":0,"inodes_with_caps":1,"caps":1,"subtrees":2,"traverse":3,"traverse_hit":3,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":0,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":0},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":2,"evex":0,"evtrm":0,"ev":4,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4196569,"rdpos":4195196,"jlat":{"avgcount":2,"sum":0.006832302,"avgtime":0.003416151},"replayed":2},"mds_mem":{"ino":13,"ino+":13,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":10,"dn+":10,"dn-":0,"cap":1,"cap+":1,"cap-":0,"rss":39516,"heap":223608},"mds_server":{"dispatch_client_request":2,"dispatch_server_request":0,"handle_client_request":3,"handle_client_session":0,"handle_peer_request":0,"req_create_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_getattr_latency":{"avgcount":1,"sum":0.000473250,"avgtime":0.000473250},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":772,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":229159,"buffer_anon_items":64,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5544,"osdmap_items":90,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":51904,"mds_co_items":189,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":46,"op_send_bytes":1469,"op_resend":0,"op_reply":46,"op_latency":{"avgcount":46,"sum":0.063998505,"avgtime":0.001391271},"op_inflight":0,"oplen_avg":{"avgcount":46,"sum":92},"op":46,"op_r":36,"op_w":10,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":2,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":56,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":1,"omap_rd":48,"omap_del":0},"oft":{"omap_total_objs":0,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":76,"get_sum":126547,"get_or_fail_fail":0,"get_or_fail_success":76,"take":0,"take_sum":0,"put":76,"put_sum":126547,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":46,"take_sum":2539,"put":46,"put_sum":2539,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":46,"take_sum":46,"put":46,"put_sum":46,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x560106079c20":{"val":0,"max":3758096384,"get_started":0,"get":2,"get_sum":931,"get_or_fail_fail":0,"get_or_fail_success":2,"take":0,"take_sum":0,"put":2,"put_sum":931,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T18:54:24.101 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 10, "avgtime": 4.27e-07, "sum": 4.276e-06 }, "msgr_recv_bytes": 112044, "msgr_recv_encrypted_bytes": 104464, "msgr_recv_messages": 41, "msgr_running_fast_dispatch_time": 0.000687357, "msgr_running_recv_time": 0.010353565, "msgr_running_send_time": 0.001834864, "msgr_running_total_time": 0.009849594, "msgr_send_bytes": 11543, "msgr_send_encrypted_bytes": 5888, "msgr_send_messages": 33, "msgr_send_messages_queue_lat": { "avgcount": 33, "avgtime": 0.000138249, "sum": 0.004562231 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 10129, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 17, "msgr_running_fast_dispatch_time": 0.000501745, "msgr_running_recv_time": 0.002775052, "msgr_running_send_time": 0.000978655, "msgr_running_total_time": 0.004516864, "msgr_send_bytes": 95240, "msgr_send_encrypted_bytes": 88976, "msgr_send_messages": 21, "msgr_send_messages_queue_lat": { "avgcount": 19, "avgtime": 0.000179615, "sum": 0.003412686 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 6, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 2, "avgtime": 1.64e-07, "sum": 3.28e-07 }, "msgr_recv_bytes": 13455, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 21, "msgr_running_fast_dispatch_time": 0.000632627, "msgr_running_recv_time": 0.003124703, "msgr_running_send_time": 0.000773106, "msgr_running_total_time": 0.00490405, "msgr_send_bytes": 26377, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 23, "msgr_send_messages_queue_lat": { "avgcount": 22, "avgtime": 0.000313208, "sum": 0.006890593 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 48, "avgtime": 0.00011692, "sum": 0.005612181 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 4.7567e-05, "sum": 0.000190271 }, "queue_len": 0 }, "mds": { "caps": 1, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 10, "inodes_bottom": 3, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 10, "inodes_top": 7, "inodes_with_caps": 1, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 0, "q": 0, "reply": 1, "reply_latency": { "avgcount": 1, "avgtime": 0.00047325, "sum": 0.00047325 }, "request": 3, "root_rbytes": 0, "root_rfiles": 0, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 3, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 3, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 4, "evadd": 2, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 2, "avgtime": 0.003416151, "sum": 0.006832302 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4196569 }, "mds_mem": { "cap": 1, "cap+": 1, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 10, "dn+": 10, "dn-": 0, "heap": 223608, "ino": 13, "ino+": 13, "ino-": 0, "rss": 39516 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 2, "dispatch_server_request": 0, "handle_client_request": 3, "handle_client_session": 0, "handle_peer_request": 0, "req_create_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.00047325, "sum": 0.00047325 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 772, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 229159, "buffer_anon_items": 64, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 51904, "mds_co_items": 189, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5544, "osdmap_items": 90, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 56, "map_full": 0, "map_inc": 1, "omap_del": 0, "omap_rd": 48, "omap_wr": 1, "op": 46, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 46, "avgtime": 0.001391271, "sum": 0.063998505 }, "op_pg": 0, "op_r": 36, "op_reply": 46, "op_resend": 0, "op_rmw": 0, "op_send": 46, "op_send_bytes": 1469, "op_w": 10, "oplen_avg": { "avgcount": 46, "sum": 92 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 2, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 0, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 76, "get_or_fail_fail": 0, "get_or_fail_success": 76, "get_started": 0, "get_sum": 126547, "max": 104857600, "put": 76, "put_sum": 126547, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 46, "put_sum": 2539, "take": 46, "take_sum": 2539, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 46, "put_sum": 46, "take": 46, "take_sum": 46, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x560106079c20": { "get": 2, "get_or_fail_fail": 0, "get_or_fail_success": 2, "get_started": 0, "get_sum": 931, "max": 3758096384, "put": 2, "put_sum": 931, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T18:54:24.102 INFO:tasks.cephfs.test_openfiletable:omap_total_updates_0:0 2024-09-13T18:54:24.102 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:54:24.102 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:54:24.107 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T18:54:24.107 DEBUG:teuthology.orchestra.run.smithi007:> import time 2024-09-13T18:54:24.107 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-13T18:54:24.107 DEBUG:teuthology.orchestra.run.smithi007:> with open("/home/ubuntu/cephtest/mnt.0/omap_counter_test_file", '"'"'w'"'"') as f: 2024-09-13T18:54:24.108 DEBUG:teuthology.orchestra.run.smithi007:> f.write("content") 2024-09-13T18:54:24.108 DEBUG:teuthology.orchestra.run.smithi007:> f.flush() 2024-09-13T18:54:24.108 DEBUG:teuthology.orchestra.run.smithi007:> while True: 2024-09-13T18:54:24.108 DEBUG:teuthology.orchestra.run.smithi007:> time.sleep(1) 2024-09-13T18:54:24.108 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-13T18:54:24.110 DEBUG:teuthology.orchestra.run.smithi007:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/omap_counter_test_file 2024-09-13T18:54:24.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:54:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0/omap_counter_test_file': No such file or directory 2024-09-13T18:54:24.192 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-13T18:54:24.193 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-13T18:54:24.193 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-13T18:54:25.117 DEBUG:teuthology.orchestra.run.smithi007:> stat --printf=%s /home/ubuntu/cephtest/mnt.0/omap_counter_test_file 2024-09-13T18:54:25.125 INFO:teuthology.orchestra.run.smithi007.stdout:7 2024-09-13T18:54:25.126 INFO:tasks.cephfs.mount:File omap_counter_test_file became visible with size 7 from 0 after 1s 2024-09-13T18:54:25.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.283+0000 7f6a7678d700 1 Processor -- start 2024-09-13T18:54:25.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.283+0000 7f6a7678d700 1 -- start start 2024-09-13T18:54:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a7678d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6a700d3770 0x7f6a700ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a7678d700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f6a700b67d0 0x7f6a700b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a7678d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a700b5aa0 con 0x7f6a700cf660 2024-09-13T18:54:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a7678d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a700d3bc0 con 0x7f6a700d3770 2024-09-13T18:54:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a7678d700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a700ce7f0 con 0x7f6a700b67d0 2024-09-13T18:54:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a74f8a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6a700d3770 0x7f6a700ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a7578b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6a700cf660 0x7f6a700cfa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53024/0 (socket says 172.21.15.7:53024) 2024-09-13T18:54:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a7578b700 1 -- 172.21.15.7:0/2960537327 learned_addr learned my addr 172.21.15.7:0/2960537327 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a75f8c700 1 --2- 172.21.15.7:0/2960537327 >> v2:172.21.15.171:3300/0 conn(0x7f6a700b67d0 0x7f6a700b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a74f8a700 1 -- 172.21.15.7:0/2960537327 >> v2:172.21.15.171:3300/0 conn(0x7f6a700b67d0 msgr2=0x7f6a700b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a74f8a700 1 --2- 172.21.15.7:0/2960537327 >> v2:172.21.15.171:3300/0 conn(0x7f6a700b67d0 0x7f6a700b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a74f8a700 1 -- 172.21.15.7:0/2960537327 >> v1:172.21.15.7:6789/0 conn(0x7f6a700cf660 legacy=0x7f6a700cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a74f8a700 1 -- 172.21.15.7:0/2960537327 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a700cea00 con 0x7f6a700d3770 2024-09-13T18:54:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a74f8a700 1 --2- 172.21.15.7:0/2960537327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a700d3770 0x7f6a700ce2b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009850 tx=0x7f6a6c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311a0a9904f410e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a67fff700 1 -- 172.21.15.7:0/2960537327 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c011830 con 0x7f6a700d3770 2024-09-13T18:54:25.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a67fff700 1 -- 172.21.15.7:0/2960537327 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c011990 con 0x7f6a700d3770 2024-09-13T18:54:25.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a67fff700 1 -- 172.21.15.7:0/2960537327 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c011d20 con 0x7f6a700d3770 2024-09-13T18:54:25.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a7678d700 1 -- 172.21.15.7:0/2960537327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a700d3770 msgr2=0x7f6a700ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:25.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a7678d700 1 --2- 172.21.15.7:0/2960537327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a700d3770 0x7f6a700ce2b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009850 tx=0x7f6a6c009b60 comp rx=0 tx=0).stop 2024-09-13T18:54:25.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a7678d700 1 -- 172.21.15.7:0/2960537327 shutdown_connections 2024-09-13T18:54:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a7678d700 1 --2- 172.21.15.7:0/2960537327 >> v2:172.21.15.171:3300/0 conn(0x7f6a700b67d0 0x7f6a700b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a7678d700 1 --2- 172.21.15.7:0/2960537327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a700d3770 0x7f6a700ce2b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a7678d700 1 -- 172.21.15.7:0/2960537327 >> 172.21.15.7:0/2960537327 conn(0x7f6a700d2c50 msgr2=0x7f6a700d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a7678d700 1 -- 172.21.15.7:0/2960537327 shutdown_connections 2024-09-13T18:54:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.287+0000 7f6a7678d700 1 -- 172.21.15.7:0/2960537327 wait complete. 2024-09-13T18:54:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.291+0000 7f6a7678d700 1 Processor -- start 2024-09-13T18:54:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.291+0000 7f6a7678d700 1 -- start start 2024-09-13T18:54:25.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.291+0000 7f6a7678d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6a700b67d0 0x7f6a700cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.291+0000 7f6a7678d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a700cf660 0x7f6a7015f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.291+0000 7f6a7678d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a700d3770 0x7f6a7015b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.291+0000 7f6a7678d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a70157930 con 0x7f6a700cf660 2024-09-13T18:54:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.291+0000 7f6a7678d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a700cb6e0 con 0x7f6a700d3770 2024-09-13T18:54:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.291+0000 7f6a7678d700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a700ca100 con 0x7f6a700b67d0 2024-09-13T18:54:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.291+0000 7f6a74f8a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a700cf660 0x7f6a7015f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.291+0000 7f6a74f8a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a700cf660 0x7f6a7015f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36952/0 (socket says 172.21.15.7:36952) 2024-09-13T18:54:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.291+0000 7f6a74f8a700 1 -- 172.21.15.7:0/3326495536 learned_addr learned my addr 172.21.15.7:0/3326495536 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.291+0000 7f6a7578b700 1 --2- 172.21.15.7:0/3326495536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6a700b67d0 0x7f6a700cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.291+0000 7f6a75f8c700 1 --2- 172.21.15.7:0/3326495536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a700d3770 0x7f6a7015b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.291+0000 7f6a74f8a700 1 -- 172.21.15.7:0/3326495536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6a700b67d0 msgr2=0x7f6a700cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.291+0000 7f6a74f8a700 1 --2- 172.21.15.7:0/3326495536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6a700b67d0 0x7f6a700cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:25.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.291+0000 7f6a74f8a700 1 -- 172.21.15.7:0/3326495536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a700d3770 msgr2=0x7f6a7015b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.291+0000 7f6a74f8a700 1 --2- 172.21.15.7:0/3326495536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a700d3770 0x7f6a7015b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.291+0000 7f6a74f8a700 1 -- 172.21.15.7:0/3326495536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c0094b0 con 0x7f6a700cf660 2024-09-13T18:54:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.295+0000 7f6a74f8a700 1 --2- 172.21.15.7:0/3326495536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a700cf660 0x7f6a7015f410 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c0090b0 tx=0x7f6a6c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.295+0000 7f6a65ffb700 1 -- 172.21.15.7:0/3326495536 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c004550 con 0x7f6a700cf660 2024-09-13T18:54:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.295+0000 7f6a65ffb700 1 -- 172.21.15.7:0/3326495536 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c01e070 con 0x7f6a700cf660 2024-09-13T18:54:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.295+0000 7f6a7678d700 1 -- 172.21.15.7:0/3326495536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a700ca3e0 con 0x7f6a700cf660 2024-09-13T18:54:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.295+0000 7f6a65ffb700 1 -- 172.21.15.7:0/3326495536 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c003900 con 0x7f6a700cf660 2024-09-13T18:54:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.295+0000 7f6a7678d700 1 -- 172.21.15.7:0/3326495536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a700ca9a0 con 0x7f6a700cf660 2024-09-13T18:54:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.295+0000 7f6a7678d700 1 -- 172.21.15.7:0/3326495536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a700ce2b0 con 0x7f6a700cf660 2024-09-13T18:54:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.295+0000 7f6a65ffb700 1 -- 172.21.15.7:0/3326495536 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f6a6c003a60 con 0x7f6a700cf660 2024-09-13T18:54:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.295+0000 7f6a65ffb700 1 --2- 172.21.15.7:0/3326495536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6a54066730 0x7f6a54068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.295+0000 7f6a65ffb700 1 -- 172.21.15.7:0/3326495536 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6a6c085080 con 0x7f6a700cf660 2024-09-13T18:54:25.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.299+0000 7f6a7578b700 1 --2- 172.21.15.7:0/3326495536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6a54066730 0x7f6a54068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:25.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.299+0000 7f6a7578b700 1 --2- 172.21.15.7:0/3326495536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6a54066730 0x7f6a54068bf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6a60005930 tx=0x7f6a60009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:25.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.303+0000 7f6a65ffb700 1 -- 172.21.15.7:0/3326495536 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6a6c0523a0 con 0x7f6a700cf660 2024-09-13T18:54:25.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.507+0000 7f6a7678d700 1 -- 172.21.15.7:0/3326495536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a700cf440 con 0x7f6a700cf660 2024-09-13T18:54:25.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.511+0000 7f6a65ffb700 1 -- 172.21.15.7:0/3326495536 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f6a6c0559c0 con 0x7f6a700cf660 2024-09-13T18:54:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:25.515 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1829974483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1829974483},{"type":"v1","addr":"172.21.15.171:6835","nonce":1829974483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1948505526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1948505526},{"type":"v1","addr":"172.21.15.171:6837","nonce":1948505526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3617964311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3617964311},{"type":"v1","addr":"172.21.15.7:6833","nonce":3617964311}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:21.419627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/3768750832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3768750832},{"type":"v1","addr":"172.21.15.7:6835","nonce":3768750832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:25.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.511+0000 7f6a7678d700 1 -- 172.21.15.7:0/3326495536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6a54066730 msgr2=0x7f6a54068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:25.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.511+0000 7f6a7678d700 1 --2- 172.21.15.7:0/3326495536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6a54066730 0x7f6a54068bf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6a60005930 tx=0x7f6a60009040 comp rx=0 tx=0).stop 2024-09-13T18:54:25.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.511+0000 7f6a7678d700 1 -- 172.21.15.7:0/3326495536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a700cf660 msgr2=0x7f6a7015f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:25.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.511+0000 7f6a7678d700 1 --2- 172.21.15.7:0/3326495536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a700cf660 0x7f6a7015f410 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c0090b0 tx=0x7f6a6c004770 comp rx=0 tx=0).stop 2024-09-13T18:54:25.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.511+0000 7f6a7678d700 1 -- 172.21.15.7:0/3326495536 shutdown_connections 2024-09-13T18:54:25.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.511+0000 7f6a7678d700 1 --2- 172.21.15.7:0/3326495536 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6a54066730 0x7f6a54068bf0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:25.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.511+0000 7f6a7678d700 1 --2- 172.21.15.7:0/3326495536 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a700d3770 0x7f6a7015b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:25.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.511+0000 7f6a7678d700 1 --2- 172.21.15.7:0/3326495536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a700cf660 0x7f6a7015f410 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:25.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.511+0000 7f6a7678d700 1 --2- 172.21.15.7:0/3326495536 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6a700b67d0 0x7f6a700cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:25.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.511+0000 7f6a7678d700 1 -- 172.21.15.7:0/3326495536 >> 172.21.15.7:0/3326495536 conn(0x7f6a700d2c50 msgr2=0x7f6a700d3030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:25.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.511+0000 7f6a7678d700 1 -- 172.21.15.7:0/3326495536 shutdown_connections 2024-09-13T18:54:25.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:25.511+0000 7f6a7678d700 1 -- 172.21.15.7:0/3326495536 wait complete. 2024-09-13T18:54:25.519 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-09-13T18:54:25.530 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T18:54:25.645 INFO:teuthology.orchestra.run.smithi007.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":46,"msgr_send_messages":36,"msgr_recv_bytes":113762,"msgr_send_bytes":15107,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.010746063,"msgr_running_send_time":0.002221095,"msgr_running_recv_time":0.010720110,"msgr_running_fast_dispatch_time":0.000877094,"msgr_send_messages_queue_lat":{"avgcount":36,"sum":0.004747978,"avgtime":0.000131888},"msgr_handle_ack_lat":{"avgcount":14,"sum":0.000006370,"avgtime":0.000000455},"msgr_recv_encrypted_bytes":104736,"msgr_send_encrypted_bytes":6480},"AsyncMessenger::Worker-1":{"msgr_recv_messages":17,"msgr_send_messages":21,"msgr_recv_bytes":10129,"msgr_send_bytes":95240,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.004516864,"msgr_running_send_time":0.000978655,"msgr_running_recv_time":0.002775052,"msgr_running_fast_dispatch_time":0.000501745,"msgr_send_messages_queue_lat":{"avgcount":19,"sum":0.003412686,"avgtime":0.000179615},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":88976},"AsyncMessenger::Worker-2":{"msgr_recv_messages":21,"msgr_send_messages":23,"msgr_recv_bytes":13455,"msgr_send_bytes":26377,"msgr_created_connections":7,"msgr_active_connections":6,"msgr_running_total_time":0.004904050,"msgr_running_send_time":0.000773106,"msgr_running_recv_time":0.003124703,"msgr_running_fast_dispatch_time":0.000632627,"msgr_send_messages_queue_lat":{"avgcount":22,"sum":0.006890593,"avgtime":0.000313208},"msgr_handle_ack_lat":{"avgcount":2,"sum":0.000000328,"avgtime":0.000000164},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":50,"sum":0.006283779,"avgtime":0.000125675}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000190271,"avgtime":0.000047567}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005761441,"avgtime":0.002880720},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":0,"handle_client_caps_dirty":0,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":3,"evex":0,"evtrm":0,"ev":5,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4198297,"rdpos":4195196,"jlat":{"avgcount":3,"sum":0.010234584,"avgtime":0.003411528},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":39780,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":0,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005288191,"avgtime":0.005288191},"req_getattr_latency":{"avgcount":1,"sum":0.000473250,"avgtime":0.000473250},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":772,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":229159,"buffer_anon_items":64,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5544,"osdmap_items":90,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":51904,"mds_co_items":189,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":47,"op_send_bytes":3197,"op_resend":0,"op_reply":47,"op_latency":{"avgcount":47,"sum":0.067998411,"avgtime":0.001446774},"op_inflight":0,"oplen_avg":{"avgcount":47,"sum":93},"op":47,"op_r":36,"op_w":11,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":3,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":56,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":1,"omap_rd":48,"omap_del":0},"oft":{"omap_total_objs":0,"omap_total_kv_pairs":0,"omap_total_updates":0,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":81,"get_sum":127763,"get_or_fail_fail":0,"get_or_fail_success":81,"take":0,"take_sum":0,"put":81,"put_sum":127763,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":47,"take_sum":4267,"put":47,"put_sum":4267,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":47,"take_sum":47,"put":47,"put_sum":47,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x560106079c20":{"val":0,"max":3758096384,"get_started":0,"get":3,"get_sum":2659,"get_or_fail_fail":0,"get_or_fail_success":3,"take":0,"take_sum":0,"put":3,"put_sum":2659,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T18:54:25.646 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 14, "avgtime": 4.55e-07, "sum": 6.37e-06 }, "msgr_recv_bytes": 113762, "msgr_recv_encrypted_bytes": 104736, "msgr_recv_messages": 46, "msgr_running_fast_dispatch_time": 0.000877094, "msgr_running_recv_time": 0.01072011, "msgr_running_send_time": 0.002221095, "msgr_running_total_time": 0.010746063, "msgr_send_bytes": 15107, "msgr_send_encrypted_bytes": 6480, "msgr_send_messages": 36, "msgr_send_messages_queue_lat": { "avgcount": 36, "avgtime": 0.000131888, "sum": 0.004747978 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 10129, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 17, "msgr_running_fast_dispatch_time": 0.000501745, "msgr_running_recv_time": 0.002775052, "msgr_running_send_time": 0.000978655, "msgr_running_total_time": 0.004516864, "msgr_send_bytes": 95240, "msgr_send_encrypted_bytes": 88976, "msgr_send_messages": 21, "msgr_send_messages_queue_lat": { "avgcount": 19, "avgtime": 0.000179615, "sum": 0.003412686 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 6, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 2, "avgtime": 1.64e-07, "sum": 3.28e-07 }, "msgr_recv_bytes": 13455, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 21, "msgr_running_fast_dispatch_time": 0.000632627, "msgr_running_recv_time": 0.003124703, "msgr_running_send_time": 0.000773106, "msgr_running_total_time": 0.00490405, "msgr_send_bytes": 26377, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 23, "msgr_send_messages_queue_lat": { "avgcount": 22, "avgtime": 0.000313208, "sum": 0.006890593 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 50, "avgtime": 0.000125675, "sum": 0.006283779 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 4.7567e-05, "sum": 0.000190271 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 0, "handle_client_caps_dirty": 0, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00288072, "sum": 0.005761441 }, "request": 4, "root_rbytes": 0, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 5, "evadd": 3, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 3, "avgtime": 0.003411528, "sum": 0.010234584 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4198297 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 39780 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 0, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005288191, "sum": 0.005288191 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.00047325, "sum": 0.00047325 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 772, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 229159, "buffer_anon_items": 64, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 51904, "mds_co_items": 189, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5544, "osdmap_items": 90, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 56, "map_full": 0, "map_inc": 1, "omap_del": 0, "omap_rd": 48, "omap_wr": 1, "op": 47, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 47, "avgtime": 0.001446774, "sum": 0.067998411 }, "op_pg": 0, "op_r": 36, "op_reply": 47, "op_resend": 0, "op_rmw": 0, "op_send": 47, "op_send_bytes": 3197, "op_w": 11, "oplen_avg": { "avgcount": 47, "sum": 93 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 3, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 0, "omap_total_objs": 0, "omap_total_removes": 0, "omap_total_updates": 0 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 81, "get_or_fail_fail": 0, "get_or_fail_success": 81, "get_started": 0, "get_sum": 127763, "max": 104857600, "put": 81, "put_sum": 127763, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 47, "put_sum": 4267, "take": 47, "take_sum": 4267, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 47, "put_sum": 47, "take": 47, "take_sum": 47, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x560106079c20": { "get": 3, "get_or_fail_fail": 0, "get_or_fail_success": 3, "get_started": 0, "get_sum": 2659, "max": 3758096384, "put": 3, "put_sum": 2659, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T18:54:25.646 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120 retry_count=0)... 2024-09-13T18:54:29.335 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:54:29.338 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:54:30.647 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a6b6c700 1 Processor -- start 2024-09-13T18:54:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a6b6c700 1 -- start start 2024-09-13T18:54:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a6b6c700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa5a0070e50 0x7fa5a00ada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a6b6c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa5a00cabf0 0x7fa5a00b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a6b6c700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a00b1aa0 con 0x7fa5a01599b0 2024-09-13T18:54:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a6b6c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a006c3c0 con 0x7fa5a00cabf0 2024-09-13T18:54:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a6b6c700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a006be40 con 0x7fa5a0070e50 2024-09-13T18:54:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a636b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa5a00cabf0 0x7fa5a00b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a5b6a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa5a01599b0 0x7fa5a0070740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53032/0 (socket says 172.21.15.7:53032) 2024-09-13T18:54:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a5b6a700 1 -- 172.21.15.7:0/1416184959 learned_addr learned my addr 172.21.15.7:0/1416184959 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a5369700 1 --2- 172.21.15.7:0/1416184959 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa5a0070e50 0x7fa5a00ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a636b700 1 -- 172.21.15.7:0/1416184959 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa5a0070e50 msgr2=0x7fa5a00ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a636b700 1 --2- 172.21.15.7:0/1416184959 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa5a0070e50 0x7fa5a00ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a636b700 1 -- 172.21.15.7:0/1416184959 >> v1:172.21.15.7:6789/0 conn(0x7fa5a01599b0 legacy=0x7fa5a0070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a636b700 1 -- 172.21.15.7:0/1416184959 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a00cb140 con 0x7fa5a00cabf0 2024-09-13T18:54:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a5369700 1 --2- 172.21.15.7:0/1416184959 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa5a0070e50 0x7fa5a00ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a636b700 1 --2- 172.21.15.7:0/1416184959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5a00cabf0 0x7fa5a00b1f30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa598009d90 tx=0x7fa59800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63569e478a0a054f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a4b68700 1 -- 172.21.15.7:0/1416184959 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59800e070 con 0x7fa5a00cabf0 2024-09-13T18:54:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a4b68700 1 -- 172.21.15.7:0/1416184959 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59800bbd0 con 0x7fa5a00cabf0 2024-09-13T18:54:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.767+0000 7fa5a4b68700 1 -- 172.21.15.7:0/1416184959 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598012430 con 0x7fa5a00cabf0 2024-09-13T18:54:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.771+0000 7fa5a6b6c700 1 -- 172.21.15.7:0/1416184959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5a00cabf0 msgr2=0x7fa5a00b1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.771+0000 7fa5a6b6c700 1 --2- 172.21.15.7:0/1416184959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5a00cabf0 0x7fa5a00b1f30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa598009d90 tx=0x7fa59800b330 comp rx=0 tx=0).stop 2024-09-13T18:54:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.771+0000 7fa5a6b6c700 1 -- 172.21.15.7:0/1416184959 shutdown_connections 2024-09-13T18:54:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.771+0000 7fa5a6b6c700 1 --2- 172.21.15.7:0/1416184959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5a00cabf0 0x7fa5a00b1f30 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.771+0000 7fa5a6b6c700 1 --2- 172.21.15.7:0/1416184959 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa5a0070e50 0x7fa5a00ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.771+0000 7fa5a6b6c700 1 -- 172.21.15.7:0/1416184959 >> 172.21.15.7:0/1416184959 conn(0x7fa5a00b0570 msgr2=0x7fa5a00b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.771+0000 7fa5a6b6c700 1 -- 172.21.15.7:0/1416184959 shutdown_connections 2024-09-13T18:54:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.771+0000 7fa5a6b6c700 1 -- 172.21.15.7:0/1416184959 wait complete. 2024-09-13T18:54:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.771+0000 7fa5a6b6c700 1 Processor -- start 2024-09-13T18:54:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.771+0000 7fa5a6b6c700 1 -- start start 2024-09-13T18:54:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a6b6c700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa5a0070e50 0x7fa5a0148330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a6b6c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5a00cabf0 0x7fa5a014cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a6b6c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5a01599b0 0x7fa5a01442c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a6b6c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a0148870 con 0x7fa5a00cabf0 2024-09-13T18:54:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a6b6c700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a0149ac0 con 0x7fa5a01599b0 2024-09-13T18:54:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a6b6c700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a0148a70 con 0x7fa5a0070e50 2024-09-13T18:54:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a5369700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5a00cabf0 0x7fa5a014cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a5369700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5a00cabf0 0x7fa5a014cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36982/0 (socket says 172.21.15.7:36982) 2024-09-13T18:54:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a5369700 1 -- 172.21.15.7:0/3649268330 learned_addr learned my addr 172.21.15.7:0/3649268330 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a5b6a700 1 --2- 172.21.15.7:0/3649268330 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa5a0070e50 0x7fa5a0148330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a636b700 1 --2- 172.21.15.7:0/3649268330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5a01599b0 0x7fa5a01442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a5369700 1 -- 172.21.15.7:0/3649268330 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa5a0070e50 msgr2=0x7fa5a0148330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a5369700 1 --2- 172.21.15.7:0/3649268330 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa5a0070e50 0x7fa5a0148330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a5369700 1 -- 172.21.15.7:0/3649268330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5a01599b0 msgr2=0x7fa5a01442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a5369700 1 --2- 172.21.15.7:0/3649268330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5a01599b0 0x7fa5a01442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a5369700 1 -- 172.21.15.7:0/3649268330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa598009a70 con 0x7fa5a00cabf0 2024-09-13T18:54:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a636b700 1 --2- 172.21.15.7:0/3649268330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5a01599b0 0x7fa5a01442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a5369700 1 --2- 172.21.15.7:0/3649268330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5a00cabf0 0x7fa5a014cb30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00e790 tx=0x7fa59c00eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5967fc700 1 -- 172.21.15.7:0/3649268330 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c013610 con 0x7fa5a00cabf0 2024-09-13T18:54:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5967fc700 1 -- 172.21.15.7:0/3649268330 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59c013790 con 0x7fa5a00cabf0 2024-09-13T18:54:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a6b6c700 1 -- 172.21.15.7:0/3649268330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a0148d50 con 0x7fa5a00cabf0 2024-09-13T18:54:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5967fc700 1 -- 172.21.15.7:0/3649268330 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c013b40 con 0x7fa5a00cabf0 2024-09-13T18:54:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.775+0000 7fa5a6b6c700 1 -- 172.21.15.7:0/3649268330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a0149310 con 0x7fa5a00cabf0 2024-09-13T18:54:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.779+0000 7fa5a6b6c700 1 -- 172.21.15.7:0/3649268330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a00ada80 con 0x7fa5a00cabf0 2024-09-13T18:54:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.779+0000 7fa5967fc700 1 -- 172.21.15.7:0/3649268330 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa59c013ca0 con 0x7fa5a00cabf0 2024-09-13T18:54:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.779+0000 7fa5967fc700 1 --2- 172.21.15.7:0/3649268330 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa584066730 0x7fa584068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.779+0000 7fa5967fc700 1 -- 172.21.15.7:0/3649268330 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa59c083b40 con 0x7fa5a00cabf0 2024-09-13T18:54:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.779+0000 7fa5a5b6a700 1 --2- 172.21.15.7:0/3649268330 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa584066730 0x7fa584068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:30.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.779+0000 7fa5a5b6a700 1 --2- 172.21.15.7:0/3649268330 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa584066730 0x7fa584068bf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa590000f60 tx=0x7fa590007f30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:30.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.787+0000 7fa5967fc700 1 -- 172.21.15.7:0/3649268330 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa59c050dc0 con 0x7fa5a00cabf0 2024-09-13T18:54:31.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.995+0000 7fa5a6b6c700 1 -- 172.21.15.7:0/3649268330 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5a00b1290 con 0x7fa5a00cabf0 2024-09-13T18:54:31.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.995+0000 7fa5967fc700 1 -- 172.21.15.7:0/3649268330 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4735 (secure 0 0 0) 0x7fa59c0543e0 con 0x7fa5a00cabf0 2024-09-13T18:54:31.002 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:31.002 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1829974483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1829974483},{"type":"v1","addr":"172.21.15.171:6835","nonce":1829974483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1948505526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1948505526},{"type":"v1","addr":"172.21.15.171:6837","nonce":1948505526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3617964311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3617964311},{"type":"v1","addr":"172.21.15.7:6833","nonce":3617964311}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:21.419627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/3768750832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3768750832},{"type":"v1","addr":"172.21.15.7:6835","nonce":3768750832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:31.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.999+0000 7fa5a6b6c700 1 -- 172.21.15.7:0/3649268330 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa584066730 msgr2=0x7fa584068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:31.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.999+0000 7fa5a6b6c700 1 --2- 172.21.15.7:0/3649268330 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa584066730 0x7fa584068bf0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa590000f60 tx=0x7fa590007f30 comp rx=0 tx=0).stop 2024-09-13T18:54:31.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.999+0000 7fa5a6b6c700 1 -- 172.21.15.7:0/3649268330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5a00cabf0 msgr2=0x7fa5a014cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:31.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.999+0000 7fa5a6b6c700 1 --2- 172.21.15.7:0/3649268330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5a00cabf0 0x7fa5a014cb30 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00e790 tx=0x7fa59c00eaa0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.999+0000 7fa5a6b6c700 1 -- 172.21.15.7:0/3649268330 shutdown_connections 2024-09-13T18:54:31.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.999+0000 7fa5a6b6c700 1 --2- 172.21.15.7:0/3649268330 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa584066730 0x7fa584068bf0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.999+0000 7fa5a6b6c700 1 --2- 172.21.15.7:0/3649268330 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5a01599b0 0x7fa5a01442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.999+0000 7fa5a6b6c700 1 --2- 172.21.15.7:0/3649268330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5a00cabf0 0x7fa5a014cb30 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.999+0000 7fa5a6b6c700 1 --2- 172.21.15.7:0/3649268330 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa5a0070e50 0x7fa5a0148330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.999+0000 7fa5a6b6c700 1 -- 172.21.15.7:0/3649268330 >> 172.21.15.7:0/3649268330 conn(0x7fa5a00b0570 msgr2=0x7fa5a00af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:31.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.999+0000 7fa5a6b6c700 1 -- 172.21.15.7:0/3649268330 shutdown_connections 2024-09-13T18:54:31.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:30.999+0000 7fa5a6b6c700 1 -- 172.21.15.7:0/3649268330 wait complete. 2024-09-13T18:54:31.006 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-09-13T18:54:31.017 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T18:54:31.136 INFO:teuthology.orchestra.run.smithi007.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":53,"msgr_send_messages":37,"msgr_recv_bytes":116474,"msgr_send_bytes":15699,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.012223405,"msgr_running_send_time":0.002795201,"msgr_running_recv_time":0.011120756,"msgr_running_fast_dispatch_time":0.001371606,"msgr_send_messages_queue_lat":{"avgcount":37,"sum":0.004802356,"avgtime":0.000129793},"msgr_handle_ack_lat":{"avgcount":20,"sum":0.000008062,"avgtime":0.000000403},"msgr_recv_encrypted_bytes":105008,"msgr_send_encrypted_bytes":7072},"AsyncMessenger::Worker-1":{"msgr_recv_messages":17,"msgr_send_messages":22,"msgr_recv_bytes":10129,"msgr_send_bytes":97016,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.004601186,"msgr_running_send_time":0.001048174,"msgr_running_recv_time":0.002775052,"msgr_running_fast_dispatch_time":0.000501745,"msgr_send_messages_queue_lat":{"avgcount":20,"sum":0.003462442,"avgtime":0.000173122},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":90752},"AsyncMessenger::Worker-2":{"msgr_recv_messages":25,"msgr_send_messages":27,"msgr_recv_bytes":14749,"msgr_send_bytes":28439,"msgr_created_connections":7,"msgr_active_connections":6,"msgr_running_total_time":0.005550922,"msgr_running_send_time":0.001022522,"msgr_running_recv_time":0.003260508,"msgr_running_fast_dispatch_time":0.000865445,"msgr_send_messages_queue_lat":{"avgcount":26,"sum":0.007094455,"avgtime":0.000272863},"msgr_handle_ack_lat":{"avgcount":2,"sum":0.000000328,"avgtime":0.000000164},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":52,"sum":0.006455826,"avgtime":0.000124150}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000190271,"avgtime":0.000047567}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005761441,"avgtime":0.002880720},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":4,"evex":0,"evtrm":0,"ev":6,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4200014,"rdpos":4195196,"jlat":{"avgcount":3,"sum":0.010234584,"avgtime":0.003411528},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":40044,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":0,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005288191,"avgtime":0.005288191},"req_getattr_latency":{"avgcount":1,"sum":0.000473250,"avgtime":0.000473250},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":777,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":233729,"buffer_anon_items":70,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5544,"osdmap_items":90,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":55175,"mds_co_items":229,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":51,"op_send_bytes":3733,"op_resend":0,"op_reply":51,"op_latency":{"avgcount":51,"sum":0.075998225,"avgtime":0.001490161},"op_inflight":0,"oplen_avg":{"avgcount":51,"sum":104},"op":51,"op_r":36,"op_w":15,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":3,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":56,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":9,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":92,"get_sum":130727,"get_or_fail_fail":0,"get_or_fail_success":92,"take":0,"take_sum":0,"put":92,"put_sum":130727,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":51,"take_sum":4803,"put":51,"put_sum":4803,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":51,"take_sum":51,"put":51,"put_sum":51,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x560106079c20":{"val":1717,"max":3758096384,"get_started":0,"get":4,"get_sum":4376,"get_or_fail_fail":0,"get_or_fail_success":4,"take":0,"take_sum":0,"put":3,"put_sum":2659,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T18:54:31.137 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 20, "avgtime": 4.03e-07, "sum": 8.062e-06 }, "msgr_recv_bytes": 116474, "msgr_recv_encrypted_bytes": 105008, "msgr_recv_messages": 53, "msgr_running_fast_dispatch_time": 0.001371606, "msgr_running_recv_time": 0.011120756, "msgr_running_send_time": 0.002795201, "msgr_running_total_time": 0.012223405, "msgr_send_bytes": 15699, "msgr_send_encrypted_bytes": 7072, "msgr_send_messages": 37, "msgr_send_messages_queue_lat": { "avgcount": 37, "avgtime": 0.000129793, "sum": 0.004802356 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 10129, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 17, "msgr_running_fast_dispatch_time": 0.000501745, "msgr_running_recv_time": 0.002775052, "msgr_running_send_time": 0.001048174, "msgr_running_total_time": 0.004601186, "msgr_send_bytes": 97016, "msgr_send_encrypted_bytes": 90752, "msgr_send_messages": 22, "msgr_send_messages_queue_lat": { "avgcount": 20, "avgtime": 0.000173122, "sum": 0.003462442 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 6, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 2, "avgtime": 1.64e-07, "sum": 3.28e-07 }, "msgr_recv_bytes": 14749, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 25, "msgr_running_fast_dispatch_time": 0.000865445, "msgr_running_recv_time": 0.003260508, "msgr_running_send_time": 0.001022522, "msgr_running_total_time": 0.005550922, "msgr_send_bytes": 28439, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 27, "msgr_send_messages_queue_lat": { "avgcount": 26, "avgtime": 0.000272863, "sum": 0.007094455 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 52, "avgtime": 0.00012415, "sum": 0.006455826 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 4.7567e-05, "sum": 0.000190271 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 1, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00288072, "sum": 0.005761441 }, "request": 4, "root_rbytes": 0, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 6, "evadd": 4, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 3, "avgtime": 0.003411528, "sum": 0.010234584 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4200014 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 40044 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 0, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005288191, "sum": 0.005288191 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.00047325, "sum": 0.00047325 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 777, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 233729, "buffer_anon_items": 70, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 55175, "mds_co_items": 229, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5544, "osdmap_items": 90, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 56, "map_full": 0, "map_inc": 1, "omap_del": 3, "omap_rd": 48, "omap_wr": 9, "op": 51, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 51, "avgtime": 0.001490161, "sum": 0.075998225 }, "op_pg": 0, "op_r": 36, "op_reply": 51, "op_resend": 0, "op_rmw": 0, "op_send": 51, "op_send_bytes": 3733, "op_w": 15, "oplen_avg": { "avgcount": 51, "sum": 104 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 3, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 92, "get_or_fail_fail": 0, "get_or_fail_success": 92, "get_started": 0, "get_sum": 130727, "max": 104857600, "put": 92, "put_sum": 130727, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 51, "put_sum": 4803, "take": 51, "take_sum": 4803, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 51, "put_sum": 51, "take": 51, "take_sum": 51, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x560106079c20": { "get": 4, "get_or_fail_fail": 0, "get_or_fail_success": 4, "get_started": 0, "get_sum": 4376, "max": 3758096384, "put": 3, "put_sum": 2659, "take": 0, "take_sum": 0, "val": 1717, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T18:54:31.137 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-09-13T18:54:31.137 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:31.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.247+0000 7f4461de2700 1 Processor -- start 2024-09-13T18:54:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.247+0000 7f4461de2700 1 -- start start 2024-09-13T18:54:31.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f4461de2700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f445c0ada40 0x7f445c070020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f4461de2700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f445c06cdc0 0x7f445c164b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f4461de2700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f445c0b3b40 con 0x7f445c160ff0 2024-09-13T18:54:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f4461de2700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f445c0b3860 con 0x7f445c0ada40 2024-09-13T18:54:31.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f4461de2700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445c071a20 con 0x7f445c06cdc0 2024-09-13T18:54:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f4460de0700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f445c160ff0 0x7f445c06c6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:51668/0 (socket says 172.21.15.7:51668) 2024-09-13T18:54:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f4460de0700 1 -- 172.21.15.7:0/1255392986 learned_addr learned my addr 172.21.15.7:0/1255392986 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f44615e1700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f445c06cdc0 0x7f445c164b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f445bfff700 1 --2- 172.21.15.7:0/1255392986 >> v2:172.21.15.171:3300/0 conn(0x7f445c0ada40 0x7f445c070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f445b7fe700 1 -- 172.21.15.7:0/1255392986 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1148870936 0 0) 0x7f445c0b3b40 con 0x7f445c160ff0 2024-09-13T18:54:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f445b7fe700 1 -- 172.21.15.7:0/1255392986 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f444c0035d0 con 0x7f445c160ff0 2024-09-13T18:54:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f44615e1700 1 -- 172.21.15.7:0/1255392986 >> v2:172.21.15.171:3300/0 conn(0x7f445c0ada40 msgr2=0x7f445c070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f44615e1700 1 --2- 172.21.15.7:0/1255392986 >> v2:172.21.15.171:3300/0 conn(0x7f445c0ada40 0x7f445c070020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f44615e1700 1 -- 172.21.15.7:0/1255392986 >> v1:172.21.15.171:6789/0 conn(0x7f445c160ff0 legacy=0x7f445c06c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:31.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f44615e1700 1 -- 172.21.15.7:0/1255392986 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c071d00 con 0x7f445c06cdc0 2024-09-13T18:54:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f44615e1700 1 --2- 172.21.15.7:0/1255392986 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f445c06cdc0 0x7f445c164b20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4454009d90 tx=0x7f445400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deea79976c032170 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f445b7fe700 1 -- 172.21.15.7:0/1255392986 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445400e070 con 0x7f445c06cdc0 2024-09-13T18:54:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f445b7fe700 1 -- 172.21.15.7:0/1255392986 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445400bb20 con 0x7f445c06cdc0 2024-09-13T18:54:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f445b7fe700 1 -- 172.21.15.7:0/1255392986 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445400be90 con 0x7f445c06cdc0 2024-09-13T18:54:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f4461de2700 1 -- 172.21.15.7:0/1255392986 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f445c06cdc0 msgr2=0x7f445c164b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f4461de2700 1 --2- 172.21.15.7:0/1255392986 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f445c06cdc0 0x7f445c164b20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4454009d90 tx=0x7f445400b330 comp rx=0 tx=0).stop 2024-09-13T18:54:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f4461de2700 1 -- 172.21.15.7:0/1255392986 shutdown_connections 2024-09-13T18:54:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f4461de2700 1 --2- 172.21.15.7:0/1255392986 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f445c06cdc0 0x7f445c164b20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f4461de2700 1 --2- 172.21.15.7:0/1255392986 >> v2:172.21.15.171:3300/0 conn(0x7f445c0ada40 0x7f445c070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f4461de2700 1 -- 172.21.15.7:0/1255392986 >> 172.21.15.7:0/1255392986 conn(0x7f445c0b0570 msgr2=0x7f445c0b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f4461de2700 1 -- 172.21.15.7:0/1255392986 shutdown_connections 2024-09-13T18:54:31.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.251+0000 7f4461de2700 1 -- 172.21.15.7:0/1255392986 wait complete. 2024-09-13T18:54:31.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f4461de2700 1 Processor -- start 2024-09-13T18:54:31.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f4461de2700 1 -- start start 2024-09-13T18:54:31.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f4461de2700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f445c06cdc0 0x7f445c14da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:31.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f4461de2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c0ada40 0x7f445c0aab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:31.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f4461de2700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f445c160ff0 0x7f445c0ab090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:31.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f4461de2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c0a89d0 con 0x7f445c0ada40 2024-09-13T18:54:31.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f4461de2700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c1641e0 con 0x7f445c06cdc0 2024-09-13T18:54:31.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f4461de2700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445c0ad170 con 0x7f445c160ff0 2024-09-13T18:54:31.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f445bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c0ada40 0x7f445c0aab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:31.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f445bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c0ada40 0x7f445c0aab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49444/0 (socket says 172.21.15.7:49444) 2024-09-13T18:54:31.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f445bfff700 1 -- 172.21.15.7:0/2045047729 learned_addr learned my addr 172.21.15.7:0/2045047729 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:31.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f44615e1700 1 --2- 172.21.15.7:0/2045047729 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f445c160ff0 0x7f445c0ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:31.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f4460de0700 1 --2- 172.21.15.7:0/2045047729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f445c06cdc0 0x7f445c14da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:31.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f445bfff700 1 -- 172.21.15.7:0/2045047729 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f445c160ff0 msgr2=0x7f445c0ab090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:31.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f445bfff700 1 --2- 172.21.15.7:0/2045047729 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f445c160ff0 0x7f445c0ab090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f445bfff700 1 -- 172.21.15.7:0/2045047729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f445c06cdc0 msgr2=0x7f445c14da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f445bfff700 1 --2- 172.21.15.7:0/2045047729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f445c06cdc0 0x7f445c14da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f445bfff700 1 -- 172.21.15.7:0/2045047729 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4454009a70 con 0x7f445c0ada40 2024-09-13T18:54:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.255+0000 7f44615e1700 1 --2- 172.21.15.7:0/2045047729 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f445c160ff0 0x7f445c0ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.259+0000 7f445bfff700 1 --2- 172.21.15.7:0/2045047729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c0ada40 0x7f445c0aab50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f444800e9d0 tx=0x7f444800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.259+0000 7f44597fa700 1 -- 172.21.15.7:0/2045047729 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448009720 con 0x7f445c0ada40 2024-09-13T18:54:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.259+0000 7f44597fa700 1 -- 172.21.15.7:0/2045047729 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444800c850 con 0x7f445c0ada40 2024-09-13T18:54:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.259+0000 7f4461de2700 1 -- 172.21.15.7:0/2045047729 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445c163c20 con 0x7f445c0ada40 2024-09-13T18:54:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.259+0000 7f44597fa700 1 -- 172.21.15.7:0/2045047729 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444800cbc0 con 0x7f445c0ada40 2024-09-13T18:54:31.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.259+0000 7f4461de2700 1 -- 172.21.15.7:0/2045047729 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445c0a85f0 con 0x7f445c0ada40 2024-09-13T18:54:31.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.259+0000 7f4461de2700 1 -- 172.21.15.7:0/2045047729 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445c0a8450 con 0x7f445c0ada40 2024-09-13T18:54:31.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.259+0000 7f44597fa700 1 -- 172.21.15.7:0/2045047729 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f444800cd20 con 0x7f445c0ada40 2024-09-13T18:54:31.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.263+0000 7f44597fa700 1 --2- 172.21.15.7:0/2045047729 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4440066730 0x7f4440068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.263+0000 7f44597fa700 1 -- 172.21.15.7:0/2045047729 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f44480842f0 con 0x7f445c0ada40 2024-09-13T18:54:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.263+0000 7f4460de0700 1 --2- 172.21.15.7:0/2045047729 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4440066730 0x7f4440068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.263+0000 7f4460de0700 1 --2- 172.21.15.7:0/2045047729 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4440066730 0x7f4440068bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4450007fd0 tx=0x7f4450007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:31.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.271+0000 7f44597fa700 1 -- 172.21.15.7:0/2045047729 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f445c0a8450 con 0x7f445c0ada40 2024-09-13T18:54:31.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.483+0000 7f4461de2700 1 -- 172.21.15.7:0/2045047729 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f445c0a8b10 con 0x7f445c0ada40 2024-09-13T18:54:31.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.483+0000 7f44597fa700 1 -- 172.21.15.7:0/2045047729 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f445c0a8b10 con 0x7f445c0ada40 2024-09-13T18:54:31.489 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:31.489 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1829974483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1829974483},{"type":"v1","addr":"172.21.15.171:6835","nonce":1829974483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1948505526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1948505526},{"type":"v1","addr":"172.21.15.171:6837","nonce":1948505526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3617964311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3617964311},{"type":"v1","addr":"172.21.15.7:6833","nonce":3617964311}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:21.419627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/3768750832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3768750832},{"type":"v1","addr":"172.21.15.7:6835","nonce":3768750832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:31.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.487+0000 7f4461de2700 1 -- 172.21.15.7:0/2045047729 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4440066730 msgr2=0x7f4440068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:31.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.487+0000 7f4461de2700 1 --2- 172.21.15.7:0/2045047729 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4440066730 0x7f4440068bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4450007fd0 tx=0x7f4450007de0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.487+0000 7f4461de2700 1 -- 172.21.15.7:0/2045047729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c0ada40 msgr2=0x7f445c0aab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:31.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.487+0000 7f4461de2700 1 --2- 172.21.15.7:0/2045047729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c0ada40 0x7f445c0aab50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f444800e9d0 tx=0x7f444800ece0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.487+0000 7f4461de2700 1 -- 172.21.15.7:0/2045047729 shutdown_connections 2024-09-13T18:54:31.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.487+0000 7f4461de2700 1 --2- 172.21.15.7:0/2045047729 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4440066730 0x7f4440068bf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.487+0000 7f4461de2700 1 --2- 172.21.15.7:0/2045047729 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f445c160ff0 0x7f445c0ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.487+0000 7f4461de2700 1 --2- 172.21.15.7:0/2045047729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c0ada40 0x7f445c0aab50 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.487+0000 7f4461de2700 1 --2- 172.21.15.7:0/2045047729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f445c06cdc0 0x7f445c14da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.487+0000 7f4461de2700 1 -- 172.21.15.7:0/2045047729 >> 172.21.15.7:0/2045047729 conn(0x7f445c0b0570 msgr2=0x7f445c15bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:31.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.487+0000 7f4461de2700 1 -- 172.21.15.7:0/2045047729 shutdown_connections 2024-09-13T18:54:31.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.487+0000 7f4461de2700 1 -- 172.21.15.7:0/2045047729 wait complete. 2024-09-13T18:54:31.494 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-09-13T18:54:31.507 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T18:54:31.628 INFO:teuthology.orchestra.run.smithi007.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":54,"msgr_send_messages":37,"msgr_recv_bytes":116892,"msgr_send_bytes":15699,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.012391537,"msgr_running_send_time":0.002852932,"msgr_running_recv_time":0.011159084,"msgr_running_fast_dispatch_time":0.001440492,"msgr_send_messages_queue_lat":{"avgcount":37,"sum":0.004802356,"avgtime":0.000129793},"msgr_handle_ack_lat":{"avgcount":21,"sum":0.000008244,"avgtime":0.000000392},"msgr_recv_encrypted_bytes":105008,"msgr_send_encrypted_bytes":7072},"AsyncMessenger::Worker-1":{"msgr_recv_messages":17,"msgr_send_messages":22,"msgr_recv_bytes":10129,"msgr_send_bytes":97016,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.004601186,"msgr_running_send_time":0.001048174,"msgr_running_recv_time":0.002775052,"msgr_running_fast_dispatch_time":0.000501745,"msgr_send_messages_queue_lat":{"avgcount":20,"sum":0.003462442,"avgtime":0.000173122},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":90752},"AsyncMessenger::Worker-2":{"msgr_recv_messages":25,"msgr_send_messages":27,"msgr_recv_bytes":14749,"msgr_send_bytes":28439,"msgr_created_connections":7,"msgr_active_connections":6,"msgr_running_total_time":0.005550922,"msgr_running_send_time":0.001022522,"msgr_running_recv_time":0.003260508,"msgr_running_fast_dispatch_time":0.000865445,"msgr_send_messages_queue_lat":{"avgcount":26,"sum":0.007094455,"avgtime":0.000272863},"msgr_handle_ack_lat":{"avgcount":2,"sum":0.000000328,"avgtime":0.000000164},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":52,"sum":0.006455826,"avgtime":0.000124150}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000190271,"avgtime":0.000047567}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005761441,"avgtime":0.002880720},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":4,"evex":0,"evtrm":0,"ev":6,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4200014,"rdpos":4195196,"jlat":{"avgcount":3,"sum":0.010234584,"avgtime":0.003411528},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":40308,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":0,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005288191,"avgtime":0.005288191},"req_getattr_latency":{"avgcount":1,"sum":0.000473250,"avgtime":0.000473250},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":777,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":233729,"buffer_anon_items":70,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5544,"osdmap_items":90,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":55175,"mds_co_items":229,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":51,"op_send_bytes":3733,"op_resend":0,"op_reply":51,"op_latency":{"avgcount":51,"sum":0.075998225,"avgtime":0.001490161},"op_inflight":0,"oplen_avg":{"avgcount":51,"sum":104},"op":51,"op_r":36,"op_w":15,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":3,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":56,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":9,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":93,"get_sum":131055,"get_or_fail_fail":0,"get_or_fail_success":93,"take":0,"take_sum":0,"put":93,"put_sum":131055,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":51,"take_sum":4803,"put":51,"put_sum":4803,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":51,"take_sum":51,"put":51,"put_sum":51,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x560106079c20":{"val":1717,"max":3758096384,"get_started":0,"get":4,"get_sum":4376,"get_or_fail_fail":0,"get_or_fail_success":4,"take":0,"take_sum":0,"put":3,"put_sum":2659,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T18:54:31.629 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 21, "avgtime": 3.92e-07, "sum": 8.244e-06 }, "msgr_recv_bytes": 116892, "msgr_recv_encrypted_bytes": 105008, "msgr_recv_messages": 54, "msgr_running_fast_dispatch_time": 0.001440492, "msgr_running_recv_time": 0.011159084, "msgr_running_send_time": 0.002852932, "msgr_running_total_time": 0.012391537, "msgr_send_bytes": 15699, "msgr_send_encrypted_bytes": 7072, "msgr_send_messages": 37, "msgr_send_messages_queue_lat": { "avgcount": 37, "avgtime": 0.000129793, "sum": 0.004802356 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 10129, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 17, "msgr_running_fast_dispatch_time": 0.000501745, "msgr_running_recv_time": 0.002775052, "msgr_running_send_time": 0.001048174, "msgr_running_total_time": 0.004601186, "msgr_send_bytes": 97016, "msgr_send_encrypted_bytes": 90752, "msgr_send_messages": 22, "msgr_send_messages_queue_lat": { "avgcount": 20, "avgtime": 0.000173122, "sum": 0.003462442 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 6, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 2, "avgtime": 1.64e-07, "sum": 3.28e-07 }, "msgr_recv_bytes": 14749, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 25, "msgr_running_fast_dispatch_time": 0.000865445, "msgr_running_recv_time": 0.003260508, "msgr_running_send_time": 0.001022522, "msgr_running_total_time": 0.005550922, "msgr_send_bytes": 28439, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 27, "msgr_send_messages_queue_lat": { "avgcount": 26, "avgtime": 0.000272863, "sum": 0.007094455 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 52, "avgtime": 0.00012415, "sum": 0.006455826 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 4.7567e-05, "sum": 0.000190271 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 1, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00288072, "sum": 0.005761441 }, "request": 4, "root_rbytes": 0, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 6, "evadd": 4, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 3, "avgtime": 0.003411528, "sum": 0.010234584 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4200014 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 40308 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 0, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005288191, "sum": 0.005288191 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.00047325, "sum": 0.00047325 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 777, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 233729, "buffer_anon_items": 70, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 55175, "mds_co_items": 229, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5544, "osdmap_items": 90, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 56, "map_full": 0, "map_inc": 1, "omap_del": 3, "omap_rd": 48, "omap_wr": 9, "op": 51, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 51, "avgtime": 0.001490161, "sum": 0.075998225 }, "op_pg": 0, "op_r": 36, "op_reply": 51, "op_resend": 0, "op_rmw": 0, "op_send": 51, "op_send_bytes": 3733, "op_w": 15, "oplen_avg": { "avgcount": 51, "sum": 104 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 3, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 93, "get_or_fail_fail": 0, "get_or_fail_success": 93, "get_started": 0, "get_sum": 131055, "max": 104857600, "put": 93, "put_sum": 131055, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 51, "put_sum": 4803, "take": 51, "take_sum": 4803, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 51, "put_sum": 51, "take": 51, "take_sum": 51, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x560106079c20": { "get": 4, "get_or_fail_fail": 0, "get_or_fail_success": 4, "get_started": 0, "get_sum": 4376, "max": 3758096384, "put": 3, "put_sum": 2659, "take": 0, "take_sum": 0, "val": 1717, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T18:54:31.629 INFO:tasks.cephfs.test_openfiletable:omap_total_updates_1:2 2024-09-13T18:54:31.641 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:31.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.751+0000 7f24ba4e5700 1 Processor -- start 2024-09-13T18:54:31.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.751+0000 7f24ba4e5700 1 -- start start 2024-09-13T18:54:31.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.751+0000 7f24ba4e5700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f24b415ac60 0x7f24b40cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:31.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.751+0000 7f24ba4e5700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24b406e820 0x7f24b406ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:31.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.751+0000 7f24ba4e5700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b406e040 con 0x7f24b40b3030 2024-09-13T18:54:31.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.751+0000 7f24ba4e5700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b40cc3e0 con 0x7f24b415ac60 2024-09-13T18:54:31.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.751+0000 7f24ba4e5700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b40cc6c0 con 0x7f24b406e820 2024-09-13T18:54:31.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.751+0000 7f24b8ce2700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f24b40b3030 0x7f24b40cefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37050/0 (socket says 172.21.15.7:37050) 2024-09-13T18:54:31.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.751+0000 7f24b8ce2700 1 -- 172.21.15.7:0/847012943 learned_addr learned my addr 172.21.15.7:0/847012943 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:31.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.751+0000 7f24b9ce4700 1 --2- 172.21.15.7:0/847012943 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24b406e820 0x7f24b406ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:31.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.751+0000 7f24b94e3700 1 --2- 172.21.15.7:0/847012943 >> v2:172.21.15.171:3300/0 conn(0x7f24b415ac60 0x7f24b40cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:31.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24abfff700 1 -- 172.21.15.7:0/847012943 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2752835406 0 0) 0x7f24b406e040 con 0x7f24b40b3030 2024-09-13T18:54:31.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24abfff700 1 -- 172.21.15.7:0/847012943 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24a00035d0 con 0x7f24b40b3030 2024-09-13T18:54:31.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24b9ce4700 1 -- 172.21.15.7:0/847012943 >> v2:172.21.15.171:3300/0 conn(0x7f24b415ac60 msgr2=0x7f24b40cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:31.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24b9ce4700 1 --2- 172.21.15.7:0/847012943 >> v2:172.21.15.171:3300/0 conn(0x7f24b415ac60 0x7f24b40cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24b9ce4700 1 -- 172.21.15.7:0/847012943 >> v1:172.21.15.7:6789/0 conn(0x7f24b40b3030 legacy=0x7f24b40cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:31.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24b9ce4700 1 -- 172.21.15.7:0/847012943 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b406e300 con 0x7f24b406e820 2024-09-13T18:54:31.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24b94e3700 1 --2- 172.21.15.7:0/847012943 >> v2:172.21.15.171:3300/0 conn(0x7f24b415ac60 0x7f24b40cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:31.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24b9ce4700 1 --2- 172.21.15.7:0/847012943 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24b406e820 0x7f24b406ec90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009d90 tx=0x7f24ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fcc2f08cd18ee86 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:31.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24abfff700 1 -- 172.21.15.7:0/847012943 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac00e070 con 0x7f24b406e820 2024-09-13T18:54:31.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24abfff700 1 -- 172.21.15.7:0/847012943 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ac00bb20 con 0x7f24b406e820 2024-09-13T18:54:31.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24abfff700 1 -- 172.21.15.7:0/847012943 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ac012430 con 0x7f24b406e820 2024-09-13T18:54:31.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24ba4e5700 1 -- 172.21.15.7:0/847012943 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24b406e820 msgr2=0x7f24b406ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:31.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24ba4e5700 1 --2- 172.21.15.7:0/847012943 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24b406e820 0x7f24b406ec90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009d90 tx=0x7f24ac00b330 comp rx=0 tx=0).stop 2024-09-13T18:54:31.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24ba4e5700 1 -- 172.21.15.7:0/847012943 shutdown_connections 2024-09-13T18:54:31.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24ba4e5700 1 --2- 172.21.15.7:0/847012943 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24b406e820 0x7f24b406ec90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24ba4e5700 1 --2- 172.21.15.7:0/847012943 >> v2:172.21.15.171:3300/0 conn(0x7f24b415ac60 0x7f24b40cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24ba4e5700 1 -- 172.21.15.7:0/847012943 >> 172.21.15.7:0/847012943 conn(0x7f24b40b2510 msgr2=0x7f24b40b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:31.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24ba4e5700 1 -- 172.21.15.7:0/847012943 shutdown_connections 2024-09-13T18:54:31.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.755+0000 7f24ba4e5700 1 -- 172.21.15.7:0/847012943 wait complete. 2024-09-13T18:54:31.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24ba4e5700 1 Processor -- start 2024-09-13T18:54:31.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24ba4e5700 1 -- start start 2024-09-13T18:54:31.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24ba4e5700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24b406e820 0x7f24b40c51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:31.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24ba4e5700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24b40b3030 0x7f24b415d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:31.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24ba4e5700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24b415ac60 0x7f24b4159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:31.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24ba4e5700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b40c5700 con 0x7f24b406e820 2024-09-13T18:54:31.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24ba4e5700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b40c58b0 con 0x7f24b415ac60 2024-09-13T18:54:31.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24ba4e5700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b40c5e30 con 0x7f24b40b3030 2024-09-13T18:54:31.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24b94e3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24b406e820 0x7f24b40c51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:31.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24b94e3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24b406e820 0x7f24b40c51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49448/0 (socket says 172.21.15.7:49448) 2024-09-13T18:54:31.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24b8ce2700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24b40b3030 0x7f24b415d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:31.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24b94e3700 1 -- 172.21.15.7:0/927846830 learned_addr learned my addr 172.21.15.7:0/927846830 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:31.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24b9ce4700 1 --2- 172.21.15.7:0/927846830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24b415ac60 0x7f24b4159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:31.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24b94e3700 1 -- 172.21.15.7:0/927846830 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24b40b3030 msgr2=0x7f24b415d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:31.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24b94e3700 1 --2- 172.21.15.7:0/927846830 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24b40b3030 0x7f24b415d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24b94e3700 1 -- 172.21.15.7:0/927846830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24b415ac60 msgr2=0x7f24b4159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:31.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24b94e3700 1 --2- 172.21.15.7:0/927846830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24b415ac60 0x7f24b4159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24b94e3700 1 -- 172.21.15.7:0/927846830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a40094b0 con 0x7f24b406e820 2024-09-13T18:54:31.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24b94e3700 1 --2- 172.21.15.7:0/927846830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24b406e820 0x7f24b40c51c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f24a400ebe0 tx=0x7f24a400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:31.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24a9ffb700 1 -- 172.21.15.7:0/927846830 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a4009770 con 0x7f24b406e820 2024-09-13T18:54:31.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24a9ffb700 1 -- 172.21.15.7:0/927846830 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a400c850 con 0x7f24b406e820 2024-09-13T18:54:31.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24a9ffb700 1 -- 172.21.15.7:0/927846830 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a400cbe0 con 0x7f24b406e820 2024-09-13T18:54:31.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24ba4e5700 1 -- 172.21.15.7:0/927846830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24ac009a70 con 0x7f24b406e820 2024-09-13T18:54:31.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.759+0000 7f24ba4e5700 1 -- 172.21.15.7:0/927846830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b415a1b0 con 0x7f24b406e820 2024-09-13T18:54:31.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.763+0000 7f24ba4e5700 1 -- 172.21.15.7:0/927846830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b415ddb0 con 0x7f24b406e820 2024-09-13T18:54:31.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.763+0000 7f24a9ffb700 1 -- 172.21.15.7:0/927846830 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f24a400cd40 con 0x7f24b406e820 2024-09-13T18:54:31.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.763+0000 7f24a9ffb700 1 --2- 172.21.15.7:0/927846830 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2498066730 0x7f2498068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:31.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.763+0000 7f24a9ffb700 1 -- 172.21.15.7:0/927846830 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f24a40841b0 con 0x7f24b406e820 2024-09-13T18:54:31.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.763+0000 7f24b8ce2700 1 --2- 172.21.15.7:0/927846830 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2498066730 0x7f2498068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:31.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.767+0000 7f24b8ce2700 1 --2- 172.21.15.7:0/927846830 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2498066730 0x7f2498068bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f24b40c7820 tx=0x7f24b0007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.771+0000 7f24a9ffb700 1 -- 172.21.15.7:0/927846830 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f24a40514b0 con 0x7f24b406e820 2024-09-13T18:54:31.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.979+0000 7f24ba4e5700 1 -- 172.21.15.7:0/927846830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24b40cea60 con 0x7f24b406e820 2024-09-13T18:54:31.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.979+0000 7f24a9ffb700 1 -- 172.21.15.7:0/927846830 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f24a4054ad0 con 0x7f24b406e820 2024-09-13T18:54:31.984 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:31.984 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1829974483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1829974483},{"type":"v1","addr":"172.21.15.171:6835","nonce":1829974483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1948505526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1948505526},{"type":"v1","addr":"172.21.15.171:6837","nonce":1948505526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3617964311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3617964311},{"type":"v1","addr":"172.21.15.7:6833","nonce":3617964311}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:21.419627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/3768750832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3768750832},{"type":"v1","addr":"172.21.15.7:6835","nonce":3768750832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:31.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.983+0000 7f24ba4e5700 1 -- 172.21.15.7:0/927846830 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2498066730 msgr2=0x7f2498068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:31.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.983+0000 7f24ba4e5700 1 --2- 172.21.15.7:0/927846830 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2498066730 0x7f2498068bf0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f24b40c7820 tx=0x7f24b0007de0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.983+0000 7f24ba4e5700 1 -- 172.21.15.7:0/927846830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24b406e820 msgr2=0x7f24b40c51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:31.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.983+0000 7f24ba4e5700 1 --2- 172.21.15.7:0/927846830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24b406e820 0x7f24b40c51c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f24a400ebe0 tx=0x7f24a400eef0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.983+0000 7f24ba4e5700 1 -- 172.21.15.7:0/927846830 shutdown_connections 2024-09-13T18:54:31.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.983+0000 7f24ba4e5700 1 --2- 172.21.15.7:0/927846830 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2498066730 0x7f2498068bf0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.983+0000 7f24ba4e5700 1 --2- 172.21.15.7:0/927846830 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24b415ac60 0x7f24b4159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.983+0000 7f24ba4e5700 1 --2- 172.21.15.7:0/927846830 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24b40b3030 0x7f24b415d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.983+0000 7f24ba4e5700 1 --2- 172.21.15.7:0/927846830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24b406e820 0x7f24b40c51c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:31.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.983+0000 7f24ba4e5700 1 -- 172.21.15.7:0/927846830 >> 172.21.15.7:0/927846830 conn(0x7f24b40b2510 msgr2=0x7f24b40cec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:31.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.983+0000 7f24ba4e5700 1 -- 172.21.15.7:0/927846830 shutdown_connections 2024-09-13T18:54:31.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:31.983+0000 7f24ba4e5700 1 -- 172.21.15.7:0/927846830 wait complete. 2024-09-13T18:54:31.988 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-09-13T18:54:31.999 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T18:54:32.121 INFO:teuthology.orchestra.run.smithi007.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":54,"msgr_send_messages":37,"msgr_recv_bytes":116892,"msgr_send_bytes":15699,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.012391537,"msgr_running_send_time":0.002852932,"msgr_running_recv_time":0.011159084,"msgr_running_fast_dispatch_time":0.001440492,"msgr_send_messages_queue_lat":{"avgcount":37,"sum":0.004802356,"avgtime":0.000129793},"msgr_handle_ack_lat":{"avgcount":21,"sum":0.000008244,"avgtime":0.000000392},"msgr_recv_encrypted_bytes":105008,"msgr_send_encrypted_bytes":7072},"AsyncMessenger::Worker-1":{"msgr_recv_messages":17,"msgr_send_messages":22,"msgr_recv_bytes":10129,"msgr_send_bytes":97016,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.004601186,"msgr_running_send_time":0.001048174,"msgr_running_recv_time":0.002775052,"msgr_running_fast_dispatch_time":0.000501745,"msgr_send_messages_queue_lat":{"avgcount":20,"sum":0.003462442,"avgtime":0.000173122},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":90752},"AsyncMessenger::Worker-2":{"msgr_recv_messages":25,"msgr_send_messages":27,"msgr_recv_bytes":14749,"msgr_send_bytes":28439,"msgr_created_connections":7,"msgr_active_connections":6,"msgr_running_total_time":0.005550922,"msgr_running_send_time":0.001022522,"msgr_running_recv_time":0.003260508,"msgr_running_fast_dispatch_time":0.000865445,"msgr_send_messages_queue_lat":{"avgcount":26,"sum":0.007094455,"avgtime":0.000272863},"msgr_handle_ack_lat":{"avgcount":2,"sum":0.000000328,"avgtime":0.000000164},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":52,"sum":0.006455826,"avgtime":0.000124150}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000190271,"avgtime":0.000047567}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005761441,"avgtime":0.002880720},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":0,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":1,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":4,"evex":0,"evtrm":0,"ev":6,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4200014,"rdpos":4195196,"jlat":{"avgcount":3,"sum":0.010234584,"avgtime":0.003411528},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":40308,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":0,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005288191,"avgtime":0.005288191},"req_getattr_latency":{"avgcount":1,"sum":0.000473250,"avgtime":0.000473250},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":777,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":233729,"buffer_anon_items":70,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5544,"osdmap_items":90,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":55175,"mds_co_items":229,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":51,"op_send_bytes":3733,"op_resend":0,"op_reply":51,"op_latency":{"avgcount":51,"sum":0.075998225,"avgtime":0.001490161},"op_inflight":0,"oplen_avg":{"avgcount":51,"sum":104},"op":51,"op_r":36,"op_w":15,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":3,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":56,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":9,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":93,"get_sum":131055,"get_or_fail_fail":0,"get_or_fail_success":93,"take":0,"take_sum":0,"put":93,"put_sum":131055,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":51,"take_sum":4803,"put":51,"put_sum":4803,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":51,"take_sum":51,"put":51,"put_sum":51,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x560106079c20":{"val":1717,"max":3758096384,"get_started":0,"get":4,"get_sum":4376,"get_or_fail_fail":0,"get_or_fail_success":4,"take":0,"take_sum":0,"put":3,"put_sum":2659,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T18:54:32.122 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 21, "avgtime": 3.92e-07, "sum": 8.244e-06 }, "msgr_recv_bytes": 116892, "msgr_recv_encrypted_bytes": 105008, "msgr_recv_messages": 54, "msgr_running_fast_dispatch_time": 0.001440492, "msgr_running_recv_time": 0.011159084, "msgr_running_send_time": 0.002852932, "msgr_running_total_time": 0.012391537, "msgr_send_bytes": 15699, "msgr_send_encrypted_bytes": 7072, "msgr_send_messages": 37, "msgr_send_messages_queue_lat": { "avgcount": 37, "avgtime": 0.000129793, "sum": 0.004802356 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 10129, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 17, "msgr_running_fast_dispatch_time": 0.000501745, "msgr_running_recv_time": 0.002775052, "msgr_running_send_time": 0.001048174, "msgr_running_total_time": 0.004601186, "msgr_send_bytes": 97016, "msgr_send_encrypted_bytes": 90752, "msgr_send_messages": 22, "msgr_send_messages_queue_lat": { "avgcount": 20, "avgtime": 0.000173122, "sum": 0.003462442 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 6, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 2, "avgtime": 1.64e-07, "sum": 3.28e-07 }, "msgr_recv_bytes": 14749, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 25, "msgr_running_fast_dispatch_time": 0.000865445, "msgr_running_recv_time": 0.003260508, "msgr_running_send_time": 0.001022522, "msgr_running_total_time": 0.005550922, "msgr_send_bytes": 28439, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 27, "msgr_send_messages_queue_lat": { "avgcount": 26, "avgtime": 0.000272863, "sum": 0.007094455 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 52, "avgtime": 0.00012415, "sum": 0.006455826 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 4.7567e-05, "sum": 0.000190271 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 1, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00288072, "sum": 0.005761441 }, "request": 4, "root_rbytes": 0, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 6, "evadd": 4, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 3, "avgtime": 0.003411528, "sum": 0.010234584 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4200014 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 40308 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 0, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005288191, "sum": 0.005288191 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.00047325, "sum": 0.00047325 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 777, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 233729, "buffer_anon_items": 70, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 55175, "mds_co_items": 229, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5544, "osdmap_items": 90, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 56, "map_full": 0, "map_inc": 1, "omap_del": 3, "omap_rd": 48, "omap_wr": 9, "op": 51, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 51, "avgtime": 0.001490161, "sum": 0.075998225 }, "op_pg": 0, "op_r": 36, "op_reply": 51, "op_resend": 0, "op_rmw": 0, "op_send": 51, "op_send_bytes": 3733, "op_w": 15, "oplen_avg": { "avgcount": 51, "sum": 104 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 3, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 93, "get_or_fail_fail": 0, "get_or_fail_success": 93, "get_started": 0, "get_sum": 131055, "max": 104857600, "put": 93, "put_sum": 131055, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 51, "put_sum": 4803, "take": 51, "take_sum": 4803, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 51, "put_sum": 51, "take": 51, "take_sum": 51, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x560106079c20": { "get": 4, "get_or_fail_fail": 0, "get_or_fail_success": 4, "get_started": 0, "get_sum": 4376, "max": 3758096384, "put": 3, "put_sum": 2659, "take": 0, "take_sum": 0, "val": 1717, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T18:54:32.123 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120 retry_count=0)... 2024-09-13T18:54:37.123 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:37.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.242+0000 7f8eea701700 1 Processor -- start 2024-09-13T18:54:37.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.242+0000 7f8eea701700 1 -- start start 2024-09-13T18:54:37.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.242+0000 7f8eea701700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f8ee415bbe0 0x7f8ee40718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:37.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.242+0000 7f8eea701700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee415c020 con 0x7f8ee40ccbb0 2024-09-13T18:54:37.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.242+0000 7f8eea701700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee406fc80 con 0x7f8ee40ad6b0 2024-09-13T18:54:37.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.242+0000 7f8eea701700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ee4071e10 con 0x7f8ee415bbe0 2024-09-13T18:54:37.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.242+0000 7f8ee9f00700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8ee40ccbb0 0x7f8ee40cd040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37066/0 (socket says 172.21.15.7:37066) 2024-09-13T18:54:37.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.242+0000 7f8ee9f00700 1 -- 172.21.15.7:0/1730212574 learned_addr learned my addr 172.21.15.7:0/1730212574 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:37.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ee8efe700 1 --2- 172.21.15.7:0/1730212574 >> v2:172.21.15.171:3300/0 conn(0x7f8ee415bbe0 0x7f8ee40718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:37.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ed7fff700 1 -- 172.21.15.7:0/1730212574 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152341144 0 0) 0x7f8ee415c020 con 0x7f8ee40ccbb0 2024-09-13T18:54:37.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ed7fff700 1 -- 172.21.15.7:0/1730212574 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ed00035d0 con 0x7f8ee40ccbb0 2024-09-13T18:54:37.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ed7fff700 1 -- 172.21.15.7:0/1730212574 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3208515971 0 0) 0x7f8ed00035d0 con 0x7f8ee40ccbb0 2024-09-13T18:54:37.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ed7fff700 1 -- 172.21.15.7:0/1730212574 >> v2:172.21.15.171:3300/0 conn(0x7f8ee415bbe0 msgr2=0x7f8ee40718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:37.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ed7fff700 1 --2- 172.21.15.7:0/1730212574 >> v2:172.21.15.171:3300/0 conn(0x7f8ee415bbe0 0x7f8ee40718d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:37.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ed7fff700 1 -- 172.21.15.7:0/1730212574 >> v1:172.21.15.171:6789/0 conn(0x7f8ee40ad6b0 legacy=0x7f8ee40ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ed7fff700 1 -- 172.21.15.7:0/1730212574 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee406fed0 con 0x7f8ee40ccbb0 2024-09-13T18:54:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ed7fff700 1 -- 172.21.15.7:0/1730212574 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7f8edc0019f0 con 0x7f8ee40ccbb0 2024-09-13T18:54:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ed7fff700 1 -- 172.21.15.7:0/1730212574 >> v1:172.21.15.7:6789/0 conn(0x7f8ee40ccbb0 legacy=0x7f8ee40cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ed7fff700 1 --2- 172.21.15.7:0/1730212574 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8ed0003a20 0x7f8ed0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ed7fff700 1 --2- 172.21.15.7:0/1730212574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed0004870 0x7f8ed0004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ed7fff700 1 --2- 172.21.15.7:0/1730212574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ed0005230 0x7f8ed000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ed7fff700 1 -- 172.21.15.7:0/1730212574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee4071e10 con 0x7f8ed0004870 2024-09-13T18:54:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ed7fff700 1 -- 172.21.15.7:0/1730212574 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed000c0d0 con 0x7f8ed0005230 2024-09-13T18:54:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ed7fff700 1 -- 172.21.15.7:0/1730212574 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed00106a0 con 0x7f8ed0003a20 2024-09-13T18:54:37.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ee8efe700 1 --2- 172.21.15.7:0/1730212574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed0004870 0x7f8ed0004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ee96ff700 1 --2- 172.21.15.7:0/1730212574 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8ed0003a20 0x7f8ed0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ee9f00700 1 --2- 172.21.15.7:0/1730212574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ed0005230 0x7f8ed000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ee8efe700 1 -- 172.21.15.7:0/1730212574 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8ed0003a20 msgr2=0x7f8ed0003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ee8efe700 1 --2- 172.21.15.7:0/1730212574 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8ed0003a20 0x7f8ed0003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ee8efe700 1 -- 172.21.15.7:0/1730212574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ed0005230 msgr2=0x7f8ed000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:37.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ee8efe700 1 --2- 172.21.15.7:0/1730212574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ed0005230 0x7f8ed000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ee8efe700 1 -- 172.21.15.7:0/1730212574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed00035d0 con 0x7f8ed0004870 2024-09-13T18:54:37.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ee9f00700 1 --2- 172.21.15.7:0/1730212574 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ed0005230 0x7f8ed000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:54:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ee9f00700 1 -- 172.21.15.7:0/1730212574 reap_dead start 2024-09-13T18:54:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ee8efe700 1 --2- 172.21.15.7:0/1730212574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed0004870 0x7f8ed0004cf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009b70 tx=0x7f8ed00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d956b03baf4114c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ed7fff700 1 -- 172.21.15.7:0/1730212574 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee000e6c0 con 0x7f8ed0004870 2024-09-13T18:54:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ed7fff700 1 -- 172.21.15.7:0/1730212574 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee000e820 con 0x7f8ed0004870 2024-09-13T18:54:37.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8ed7fff700 1 -- 172.21.15.7:0/1730212574 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee000eb90 con 0x7f8ed0004870 2024-09-13T18:54:37.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8eea701700 1 -- 172.21.15.7:0/1730212574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed0004870 msgr2=0x7f8ed0004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:37.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8eea701700 1 --2- 172.21.15.7:0/1730212574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed0004870 0x7f8ed0004cf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009b70 tx=0x7f8ed00058a0 comp rx=0 tx=0).stop 2024-09-13T18:54:37.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8eea701700 1 -- 172.21.15.7:0/1730212574 shutdown_connections 2024-09-13T18:54:37.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8eea701700 1 --2- 172.21.15.7:0/1730212574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed0004870 0x7f8ed0004cf0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:37.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8eea701700 1 -- 172.21.15.7:0/1730212574 >> 172.21.15.7:0/1730212574 conn(0x7f8ee40b0570 msgr2=0x7f8ee40b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:37.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8eea701700 1 -- 172.21.15.7:0/1730212574 shutdown_connections 2024-09-13T18:54:37.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.246+0000 7f8eea701700 1 -- 172.21.15.7:0/1730212574 wait complete. 2024-09-13T18:54:37.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8eea701700 1 Processor -- start 2024-09-13T18:54:37.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8eea701700 1 -- start start 2024-09-13T18:54:37.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8eea701700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed0004870 0x7f8ee4147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:37.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8eea701700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8ee414dfd0 0x7f8ee414c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:37.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8eea701700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee4148200 0x7f8ee415d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:37.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8eea701700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee414e4b0 con 0x7f8ed0004870 2024-09-13T18:54:37.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8eea701700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee414e790 con 0x7f8ee4148200 2024-09-13T18:54:37.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8eea701700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee414ea70 con 0x7f8ee414dfd0 2024-09-13T18:54:37.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8ee96ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed0004870 0x7f8ee4147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:37.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8ee96ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed0004870 0x7f8ee4147b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49462/0 (socket says 172.21.15.7:49462) 2024-09-13T18:54:37.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8ee96ff700 1 -- 172.21.15.7:0/234264571 learned_addr learned my addr 172.21.15.7:0/234264571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:37.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8ee8efe700 1 --2- 172.21.15.7:0/234264571 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8ee414dfd0 0x7f8ee414c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:37.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8ee9f00700 1 --2- 172.21.15.7:0/234264571 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee4148200 0x7f8ee415d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:37.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8ee96ff700 1 -- 172.21.15.7:0/234264571 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8ee414dfd0 msgr2=0x7f8ee414c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:37.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8ee96ff700 1 --2- 172.21.15.7:0/234264571 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8ee414dfd0 0x7f8ee414c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:37.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8ee96ff700 1 -- 172.21.15.7:0/234264571 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee4148200 msgr2=0x7f8ee415d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:37.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8ee96ff700 1 --2- 172.21.15.7:0/234264571 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee4148200 0x7f8ee415d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:37.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8ee96ff700 1 -- 172.21.15.7:0/234264571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee414ed50 con 0x7f8ed0004870 2024-09-13T18:54:37.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8ee96ff700 1 --2- 172.21.15.7:0/234264571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed0004870 0x7f8ee4147b20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800e580 tx=0x7f8ed800e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:37.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8ed5ffb700 1 -- 172.21.15.7:0/234264571 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed80052e0 con 0x7f8ed0004870 2024-09-13T18:54:37.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8ed5ffb700 1 -- 172.21.15.7:0/234264571 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed8005440 con 0x7f8ed0004870 2024-09-13T18:54:37.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8ed5ffb700 1 -- 172.21.15.7:0/234264571 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed80038c0 con 0x7f8ed0004870 2024-09-13T18:54:37.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8eea701700 1 -- 172.21.15.7:0/234264571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee414f030 con 0x7f8ed0004870 2024-09-13T18:54:37.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.250+0000 7f8eea701700 1 -- 172.21.15.7:0/234264571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee415dc10 con 0x7f8ed0004870 2024-09-13T18:54:37.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.254+0000 7f8eea701700 1 -- 172.21.15.7:0/234264571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee4070c90 con 0x7f8ed0004870 2024-09-13T18:54:37.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.254+0000 7f8ed5ffb700 1 -- 172.21.15.7:0/234264571 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8ed8003a20 con 0x7f8ed0004870 2024-09-13T18:54:37.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.254+0000 7f8ed5ffb700 1 --2- 172.21.15.7:0/234264571 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8ec8066730 0x7f8ec8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:37.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.254+0000 7f8ed5ffb700 1 -- 172.21.15.7:0/234264571 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8ed8083790 con 0x7f8ed0004870 2024-09-13T18:54:37.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.254+0000 7f8ee8efe700 1 --2- 172.21.15.7:0/234264571 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8ec8066730 0x7f8ec8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:37.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.254+0000 7f8ee8efe700 1 --2- 172.21.15.7:0/234264571 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8ec8066730 0x7f8ec8068bf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00090b0 tx=0x7f8ee001f040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.262+0000 7f8ed5ffb700 1 -- 172.21.15.7:0/234264571 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8ed8050b50 con 0x7f8ed0004870 2024-09-13T18:54:37.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.470+0000 7f8eea701700 1 -- 172.21.15.7:0/234264571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ee415de00 con 0x7f8ed0004870 2024-09-13T18:54:37.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.474+0000 7f8ed5ffb700 1 -- 172.21.15.7:0/234264571 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f8ed8019090 con 0x7f8ed0004870 2024-09-13T18:54:37.479 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:37.479 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1829974483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1829974483},{"type":"v1","addr":"172.21.15.171:6835","nonce":1829974483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1948505526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1948505526},{"type":"v1","addr":"172.21.15.171:6837","nonce":1948505526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3617964311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3617964311},{"type":"v1","addr":"172.21.15.7:6833","nonce":3617964311}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:21.419627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/3768750832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3768750832},{"type":"v1","addr":"172.21.15.7:6835","nonce":3768750832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:37.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.478+0000 7f8eea701700 1 -- 172.21.15.7:0/234264571 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8ec8066730 msgr2=0x7f8ec8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:37.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.478+0000 7f8eea701700 1 --2- 172.21.15.7:0/234264571 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8ec8066730 0x7f8ec8068bf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00090b0 tx=0x7f8ee001f040 comp rx=0 tx=0).stop 2024-09-13T18:54:37.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.478+0000 7f8eea701700 1 -- 172.21.15.7:0/234264571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed0004870 msgr2=0x7f8ee4147b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:37.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.478+0000 7f8eea701700 1 --2- 172.21.15.7:0/234264571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed0004870 0x7f8ee4147b20 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800e580 tx=0x7f8ed800e890 comp rx=0 tx=0).stop 2024-09-13T18:54:37.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.478+0000 7f8eea701700 1 -- 172.21.15.7:0/234264571 shutdown_connections 2024-09-13T18:54:37.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.478+0000 7f8eea701700 1 --2- 172.21.15.7:0/234264571 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8ec8066730 0x7f8ec8068bf0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:37.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.478+0000 7f8eea701700 1 --2- 172.21.15.7:0/234264571 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ee4148200 0x7f8ee415d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:37.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.478+0000 7f8eea701700 1 --2- 172.21.15.7:0/234264571 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f8ee414dfd0 0x7f8ee414c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:37.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.478+0000 7f8eea701700 1 --2- 172.21.15.7:0/234264571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ed0004870 0x7f8ee4147b20 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:37.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.478+0000 7f8eea701700 1 -- 172.21.15.7:0/234264571 >> 172.21.15.7:0/234264571 conn(0x7f8ee40b0570 msgr2=0x7f8ee415ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:37.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.478+0000 7f8eea701700 1 -- 172.21.15.7:0/234264571 shutdown_connections 2024-09-13T18:54:37.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:37.478+0000 7f8eea701700 1 -- 172.21.15.7:0/234264571 wait complete. 2024-09-13T18:54:37.487 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-09-13T18:54:37.504 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T18:54:37.620 INFO:teuthology.orchestra.run.smithi007.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":65,"msgr_send_messages":42,"msgr_recv_bytes":120658,"msgr_send_bytes":19379,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.014271128,"msgr_running_send_time":0.003592364,"msgr_running_recv_time":0.011825678,"msgr_running_fast_dispatch_time":0.001969763,"msgr_send_messages_queue_lat":{"avgcount":42,"sum":0.005083770,"avgtime":0.000121042},"msgr_handle_ack_lat":{"avgcount":31,"sum":0.000011096,"avgtime":0.000000357},"msgr_recv_encrypted_bytes":105552,"msgr_send_encrypted_bytes":8256},"AsyncMessenger::Worker-1":{"msgr_recv_messages":17,"msgr_send_messages":24,"msgr_recv_bytes":10129,"msgr_send_bytes":100568,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.004755521,"msgr_running_send_time":0.001175886,"msgr_running_recv_time":0.002775052,"msgr_running_fast_dispatch_time":0.000501745,"msgr_send_messages_queue_lat":{"avgcount":22,"sum":0.003551007,"avgtime":0.000161409},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":94304},"AsyncMessenger::Worker-2":{"msgr_recv_messages":25,"msgr_send_messages":27,"msgr_recv_bytes":14749,"msgr_send_bytes":28439,"msgr_created_connections":7,"msgr_active_connections":6,"msgr_running_total_time":0.005550922,"msgr_running_send_time":0.001022522,"msgr_running_recv_time":0.003260508,"msgr_running_fast_dispatch_time":0.000865445,"msgr_send_messages_queue_lat":{"avgcount":26,"sum":0.007094455,"avgtime":0.000272863},"msgr_handle_ack_lat":{"avgcount":2,"sum":0.000000328,"avgtime":0.000000164},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":54,"sum":0.006954078,"avgtime":0.000128779}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000190271,"avgtime":0.000047567}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005761441,"avgtime":0.002880720},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":7,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":2,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":5,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4201668,"rdpos":4195196,"jlat":{"avgcount":4,"sum":0.014848683,"avgtime":0.003712170},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":40572,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":1,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005288191,"avgtime":0.005288191},"req_getattr_latency":{"avgcount":1,"sum":0.000473250,"avgtime":0.000473250},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":787,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":229721,"buffer_anon_items":69,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5544,"osdmap_items":90,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":55215,"mds_co_items":234,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":52,"op_send_bytes":5450,"op_resend":0,"op_reply":52,"op_latency":{"avgcount":52,"sum":0.079998131,"avgtime":0.001538425},"op_inflight":0,"oplen_avg":{"avgcount":52,"sum":105},"op":52,"op_r":36,"op_w":16,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":4,"osdop_writefull":1,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":56,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":9,"omap_rd":48,"omap_del":3},"oft":{"omap_total_objs":2,"omap_total_kv_pairs":2,"omap_total_updates":2,"omap_total_removes":0},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":104,"get_sum":133727,"get_or_fail_fail":0,"get_or_fail_success":104,"take":0,"take_sum":0,"put":104,"put_sum":133727,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":52,"take_sum":6520,"put":52,"put_sum":6520,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":52,"take_sum":52,"put":52,"put_sum":52,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x560106079c20":{"val":1654,"max":3758096384,"get_started":0,"get":5,"get_sum":6030,"get_or_fail_fail":0,"get_or_fail_success":5,"take":0,"take_sum":0,"put":4,"put_sum":4376,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T18:54:37.621 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 31, "avgtime": 3.57e-07, "sum": 1.1096e-05 }, "msgr_recv_bytes": 120658, "msgr_recv_encrypted_bytes": 105552, "msgr_recv_messages": 65, "msgr_running_fast_dispatch_time": 0.001969763, "msgr_running_recv_time": 0.011825678, "msgr_running_send_time": 0.003592364, "msgr_running_total_time": 0.014271128, "msgr_send_bytes": 19379, "msgr_send_encrypted_bytes": 8256, "msgr_send_messages": 42, "msgr_send_messages_queue_lat": { "avgcount": 42, "avgtime": 0.000121042, "sum": 0.00508377 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 10129, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 17, "msgr_running_fast_dispatch_time": 0.000501745, "msgr_running_recv_time": 0.002775052, "msgr_running_send_time": 0.001175886, "msgr_running_total_time": 0.004755521, "msgr_send_bytes": 100568, "msgr_send_encrypted_bytes": 94304, "msgr_send_messages": 24, "msgr_send_messages_queue_lat": { "avgcount": 22, "avgtime": 0.000161409, "sum": 0.003551007 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 6, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 2, "avgtime": 1.64e-07, "sum": 3.28e-07 }, "msgr_recv_bytes": 14749, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 25, "msgr_running_fast_dispatch_time": 0.000865445, "msgr_running_recv_time": 0.003260508, "msgr_running_send_time": 0.001022522, "msgr_running_total_time": 0.005550922, "msgr_send_bytes": 28439, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 27, "msgr_send_messages_queue_lat": { "avgcount": 26, "avgtime": 0.000272863, "sum": 0.007094455 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 54, "avgtime": 0.000128779, "sum": 0.006954078 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 4.7567e-05, "sum": 0.000190271 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 2, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00288072, "sum": 0.005761441 }, "request": 4, "root_rbytes": 7, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 5, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 4, "avgtime": 0.00371217, "sum": 0.014848683 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4201668 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 40572 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 1, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005288191, "sum": 0.005288191 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.00047325, "sum": 0.00047325 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 787, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 229721, "buffer_anon_items": 69, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 55215, "mds_co_items": 234, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5544, "osdmap_items": 90, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 56, "map_full": 0, "map_inc": 1, "omap_del": 3, "omap_rd": 48, "omap_wr": 9, "op": 52, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 52, "avgtime": 0.001538425, "sum": 0.079998131 }, "op_pg": 0, "op_r": 36, "op_reply": 52, "op_resend": 0, "op_rmw": 0, "op_send": 52, "op_send_bytes": 5450, "op_w": 16, "oplen_avg": { "avgcount": 52, "sum": 105 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 4, "osdop_writefull": 1, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 2, "omap_total_objs": 2, "omap_total_removes": 0, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 104, "get_or_fail_fail": 0, "get_or_fail_success": 104, "get_started": 0, "get_sum": 133727, "max": 104857600, "put": 104, "put_sum": 133727, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 52, "put_sum": 6520, "take": 52, "take_sum": 6520, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 52, "put_sum": 52, "take": 52, "take_sum": 52, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x560106079c20": { "get": 5, "get_or_fail_fail": 0, "get_or_fail_success": 5, "get_started": 0, "get_sum": 6030, "max": 3758096384, "put": 4, "put_sum": 4376, "take": 0, "take_sum": 0, "val": 1654, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T18:54:37.621 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=120 retry_count=0)... 2024-09-13T18:54:42.623 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.738+0000 7fb9af59e700 1 Processor -- start 2024-09-13T18:54:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.738+0000 7fb9af59e700 1 -- start start 2024-09-13T18:54:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9af59e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fb9b01599b0 0x7fb9b00ada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9af59e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9b00b1aa0 con 0x7fb9b00cabf0 2024-09-13T18:54:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9af59e700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9b006be40 con 0x7fb9b0070e50 2024-09-13T18:54:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9af59e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b006c3c0 con 0x7fb9b01599b0 2024-09-13T18:54:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9add9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb9b00cabf0 0x7fb9b0070740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51424/0 (socket says 172.21.15.7:51424) 2024-09-13T18:54:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9add9b700 1 -- 172.21.15.7:0/2399775783 learned_addr learned my addr 172.21.15.7:0/2399775783 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:42.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ae59c700 1 --2- 172.21.15.7:0/2399775783 >> v2:172.21.15.171:3300/0 conn(0x7fb9b01599b0 0x7fb9b00ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:42.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ad59a700 1 -- 172.21.15.7:0/2399775783 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3200452898 0 0) 0x7fb9b00b1aa0 con 0x7fb9b00cabf0 2024-09-13T18:54:42.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ad59a700 1 -- 172.21.15.7:0/2399775783 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9980057e0 con 0x7fb9b00cabf0 2024-09-13T18:54:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ad59a700 1 -- 172.21.15.7:0/2399775783 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3889882420 0 0) 0x7fb9980057e0 con 0x7fb9b00cabf0 2024-09-13T18:54:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ad59a700 1 -- 172.21.15.7:0/2399775783 >> v2:172.21.15.171:3300/0 conn(0x7fb9b01599b0 msgr2=0x7fb9b00ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ad59a700 1 --2- 172.21.15.7:0/2399775783 >> v2:172.21.15.171:3300/0 conn(0x7fb9b01599b0 0x7fb9b00ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ad59a700 1 -- 172.21.15.7:0/2399775783 >> v1:172.21.15.171:6789/0 conn(0x7fb9b0070e50 legacy=0x7fb9b00b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ad59a700 1 -- 172.21.15.7:0/2399775783 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b00cb140 con 0x7fb9b00cabf0 2024-09-13T18:54:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ad59a700 1 -- 172.21.15.7:0/2399775783 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7fb9a0001670 con 0x7fb9b00cabf0 2024-09-13T18:54:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ad59a700 1 -- 172.21.15.7:0/2399775783 >> v1:172.21.15.7:6789/0 conn(0x7fb9b00cabf0 legacy=0x7fb9b0070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ad59a700 1 --2- 172.21.15.7:0/2399775783 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb998005c30 0x7fb998006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ad59a700 1 --2- 172.21.15.7:0/2399775783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb998006a80 0x7fb998006f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ad59a700 1 --2- 172.21.15.7:0/2399775783 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb998007440 0x7fb99800dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ad59a700 1 -- 172.21.15.7:0/2399775783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b006c3c0 con 0x7fb998006a80 2024-09-13T18:54:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ad59a700 1 -- 172.21.15.7:0/2399775783 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99800e2e0 con 0x7fb998005c30 2024-09-13T18:54:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ad59a700 1 -- 172.21.15.7:0/2399775783 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99800e4a0 con 0x7fb998007440 2024-09-13T18:54:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9add9b700 1 --2- 172.21.15.7:0/2399775783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb998006a80 0x7fb998006f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9aed9d700 1 --2- 172.21.15.7:0/2399775783 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb998007440 0x7fb99800dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ae59c700 1 --2- 172.21.15.7:0/2399775783 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb998005c30 0x7fb998006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9add9b700 1 -- 172.21.15.7:0/2399775783 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb998007440 msgr2=0x7fb99800dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9add9b700 1 --2- 172.21.15.7:0/2399775783 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb998007440 0x7fb99800dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9add9b700 1 -- 172.21.15.7:0/2399775783 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb998005c30 msgr2=0x7fb998006050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9add9b700 1 --2- 172.21.15.7:0/2399775783 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb998005c30 0x7fb998006050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9add9b700 1 -- 172.21.15.7:0/2399775783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9980057e0 con 0x7fb998006a80 2024-09-13T18:54:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9aed9d700 1 --2- 172.21.15.7:0/2399775783 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb998007440 0x7fb99800dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9aed9d700 1 -- 172.21.15.7:0/2399775783 reap_dead start 2024-09-13T18:54:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9add9b700 1 --2- 172.21.15.7:0/2399775783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb998006a80 0x7fb998006f00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000c320 tx=0x7fb998007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda4dcb86e9daaac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ad59a700 1 -- 172.21.15.7:0/2399775783 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0007c30 con 0x7fb998006a80 2024-09-13T18:54:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ad59a700 1 -- 172.21.15.7:0/2399775783 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a000ce80 con 0x7fb998006a80 2024-09-13T18:54:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.742+0000 7fb9ad59a700 1 -- 172.21.15.7:0/2399775783 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0005bc0 con 0x7fb998006a80 2024-09-13T18:54:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.746+0000 7fb9af59e700 1 -- 172.21.15.7:0/2399775783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb998006a80 msgr2=0x7fb998006f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.746+0000 7fb9af59e700 1 --2- 172.21.15.7:0/2399775783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb998006a80 0x7fb998006f00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000c320 tx=0x7fb998007ab0 comp rx=0 tx=0).stop 2024-09-13T18:54:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.746+0000 7fb9af59e700 1 -- 172.21.15.7:0/2399775783 shutdown_connections 2024-09-13T18:54:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.746+0000 7fb9af59e700 1 --2- 172.21.15.7:0/2399775783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb998006a80 0x7fb998006f00 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.746+0000 7fb9af59e700 1 -- 172.21.15.7:0/2399775783 >> 172.21.15.7:0/2399775783 conn(0x7fb9b00b0570 msgr2=0x7fb9b00b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.746+0000 7fb9af59e700 1 -- 172.21.15.7:0/2399775783 shutdown_connections 2024-09-13T18:54:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.746+0000 7fb9af59e700 1 -- 172.21.15.7:0/2399775783 wait complete. 2024-09-13T18:54:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.746+0000 7fb9af59e700 1 Processor -- start 2024-09-13T18:54:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.746+0000 7fb9af59e700 1 -- start start 2024-09-13T18:54:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9af59e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb998006a80 0x7fb9b015a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9af59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9b015c940 0x7fb9b01f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9af59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb9b0143140 0x7fb9b00ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9af59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b014fde0 con 0x7fb9b015c940 2024-09-13T18:54:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9af59e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b015cdc0 con 0x7fb998006a80 2024-09-13T18:54:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9af59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b015d030 con 0x7fb9b0143140 2024-09-13T18:54:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9add9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9b015c940 0x7fb9b01f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9add9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9b015c940 0x7fb9b01f6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40160/0 (socket says 172.21.15.7:40160) 2024-09-13T18:54:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9add9b700 1 -- 172.21.15.7:0/1427455081 learned_addr learned my addr 172.21.15.7:0/1427455081 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9aed9d700 1 --2- 172.21.15.7:0/1427455081 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb9b0143140 0x7fb9b00ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9ae59c700 1 --2- 172.21.15.7:0/1427455081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb998006a80 0x7fb9b015a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9add9b700 1 -- 172.21.15.7:0/1427455081 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb9b0143140 msgr2=0x7fb9b00ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9add9b700 1 --2- 172.21.15.7:0/1427455081 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb9b0143140 0x7fb9b00ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9add9b700 1 -- 172.21.15.7:0/1427455081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb998006a80 msgr2=0x7fb9b015a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:42.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9add9b700 1 --2- 172.21.15.7:0/1427455081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb998006a80 0x7fb9b015a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9add9b700 1 -- 172.21.15.7:0/1427455081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b014f4b0 con 0x7fb9b015c940 2024-09-13T18:54:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9aed9d700 1 --2- 172.21.15.7:0/1427455081 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb9b0143140 0x7fb9b00ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9add9b700 1 --2- 172.21.15.7:0/1427455081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9b015c940 0x7fb9b01f6bf0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0007d60 tx=0x7fb9a0001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb99effd700 1 -- 172.21.15.7:0/1427455081 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0004d60 con 0x7fb9b015c940 2024-09-13T18:54:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb99effd700 1 -- 172.21.15.7:0/1427455081 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a00070d0 con 0x7fb9b015c940 2024-09-13T18:54:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9af59e700 1 -- 172.21.15.7:0/1427455081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b014f790 con 0x7fb9b015c940 2024-09-13T18:54:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb99effd700 1 -- 172.21.15.7:0/1427455081 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0007480 con 0x7fb9b015c940 2024-09-13T18:54:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9af59e700 1 -- 172.21.15.7:0/1427455081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b01435c0 con 0x7fb9b015c940 2024-09-13T18:54:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.750+0000 7fb9af59e700 1 -- 172.21.15.7:0/1427455081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b014ff90 con 0x7fb9b015c940 2024-09-13T18:54:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.754+0000 7fb99effd700 1 -- 172.21.15.7:0/1427455081 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb9a00075e0 con 0x7fb9b015c940 2024-09-13T18:54:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.754+0000 7fb99effd700 1 --2- 172.21.15.7:0/1427455081 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb990066760 0x7fb990068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.754+0000 7fb99effd700 1 -- 172.21.15.7:0/1427455081 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb9a00851a0 con 0x7fb9b015c940 2024-09-13T18:54:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.754+0000 7fb9ae59c700 1 --2- 172.21.15.7:0/1427455081 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb990066760 0x7fb990068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.754+0000 7fb9ae59c700 1 --2- 172.21.15.7:0/1427455081 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb990066760 0x7fb990068c20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009bf0 tx=0x7fb9a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:42.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.762+0000 7fb99effd700 1 -- 172.21.15.7:0/1427455081 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb9b014ff90 con 0x7fb9b015c940 2024-09-13T18:54:42.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.970+0000 7fb9af59e700 1 -- 172.21.15.7:0/1427455081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9b014ff90 con 0x7fb9b015c940 2024-09-13T18:54:42.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.970+0000 7fb99effd700 1 -- 172.21.15.7:0/1427455081 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4735 (secure 0 0 0) 0x7fb9b014ff90 con 0x7fb9b015c940 2024-09-13T18:54:42.976 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:42.976 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1829974483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1829974483},{"type":"v1","addr":"172.21.15.171:6835","nonce":1829974483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1948505526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1948505526},{"type":"v1","addr":"172.21.15.171:6837","nonce":1948505526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3617964311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3617964311},{"type":"v1","addr":"172.21.15.7:6833","nonce":3617964311}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:21.419627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/3768750832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3768750832},{"type":"v1","addr":"172.21.15.7:6835","nonce":3768750832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:42.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.974+0000 7fb9af59e700 1 -- 172.21.15.7:0/1427455081 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb990066760 msgr2=0x7fb990068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:42.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.974+0000 7fb9af59e700 1 --2- 172.21.15.7:0/1427455081 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb990066760 0x7fb990068c20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4009bf0 tx=0x7fb9a4008040 comp rx=0 tx=0).stop 2024-09-13T18:54:42.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.974+0000 7fb9af59e700 1 -- 172.21.15.7:0/1427455081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9b015c940 msgr2=0x7fb9b01f6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:42.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.974+0000 7fb9af59e700 1 --2- 172.21.15.7:0/1427455081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9b015c940 0x7fb9b01f6bf0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0007d60 tx=0x7fb9a0001a00 comp rx=0 tx=0).stop 2024-09-13T18:54:42.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.974+0000 7fb9af59e700 1 -- 172.21.15.7:0/1427455081 shutdown_connections 2024-09-13T18:54:42.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.974+0000 7fb9af59e700 1 --2- 172.21.15.7:0/1427455081 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb990066760 0x7fb990068c20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:42.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.974+0000 7fb9af59e700 1 --2- 172.21.15.7:0/1427455081 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb9b0143140 0x7fb9b00ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:42.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.974+0000 7fb9af59e700 1 --2- 172.21.15.7:0/1427455081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9b015c940 0x7fb9b01f6bf0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:42.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.974+0000 7fb9af59e700 1 --2- 172.21.15.7:0/1427455081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb998006a80 0x7fb9b015a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:42.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.974+0000 7fb9af59e700 1 -- 172.21.15.7:0/1427455081 >> 172.21.15.7:0/1427455081 conn(0x7fb9b00b0570 msgr2=0x7fb9b00b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:42.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.974+0000 7fb9af59e700 1 -- 172.21.15.7:0/1427455081 shutdown_connections 2024-09-13T18:54:42.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:42.974+0000 7fb9af59e700 1 -- 172.21.15.7:0/1427455081 wait complete. 2024-09-13T18:54:42.980 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-09-13T18:54:42.991 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T18:54:43.110 INFO:teuthology.orchestra.run.smithi007.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":73,"msgr_send_messages":45,"msgr_recv_bytes":123512,"msgr_send_bytes":22337,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.015828407,"msgr_running_send_time":0.004160231,"msgr_running_recv_time":0.012238824,"msgr_running_fast_dispatch_time":0.002507779,"msgr_send_messages_queue_lat":{"avgcount":45,"sum":0.005228923,"avgtime":0.000116198},"msgr_handle_ack_lat":{"avgcount":36,"sum":0.000012160,"avgtime":0.000000337},"msgr_recv_encrypted_bytes":105824,"msgr_send_encrypted_bytes":8848},"AsyncMessenger::Worker-1":{"msgr_recv_messages":17,"msgr_send_messages":25,"msgr_recv_bytes":10129,"msgr_send_bytes":102344,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.004830116,"msgr_running_send_time":0.001236327,"msgr_running_recv_time":0.002775052,"msgr_running_fast_dispatch_time":0.000501745,"msgr_send_messages_queue_lat":{"avgcount":23,"sum":0.003592534,"avgtime":0.000156197},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":96080},"AsyncMessenger::Worker-2":{"msgr_recv_messages":28,"msgr_send_messages":30,"msgr_recv_bytes":15583,"msgr_send_bytes":29672,"msgr_created_connections":7,"msgr_active_connections":6,"msgr_running_total_time":0.006112097,"msgr_running_send_time":0.001208172,"msgr_running_recv_time":0.003388071,"msgr_running_fast_dispatch_time":0.001076368,"msgr_send_messages_queue_lat":{"avgcount":29,"sum":0.007211756,"avgtime":0.000248681},"msgr_handle_ack_lat":{"avgcount":2,"sum":0.000000328,"avgtime":0.000000164},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":59,"sum":0.007548210,"avgtime":0.000127935}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000190271,"avgtime":0.000047567}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005761441,"avgtime":0.002880720},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":7,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":2,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":5,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4201668,"rdpos":4195196,"jlat":{"avgcount":5,"sum":0.019245253,"avgtime":0.003849050},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":40836,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":1,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005288191,"avgtime":0.005288191},"req_getattr_latency":{"avgcount":1,"sum":0.000473250,"avgtime":0.000473250},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":792,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":245753,"buffer_anon_items":73,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5544,"osdmap_items":90,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":56951,"mds_co_items":240,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":57,"op_send_bytes":7406,"op_resend":0,"op_reply":57,"op_latency":{"avgcount":57,"sum":0.091997851,"avgtime":0.001613997},"op_inflight":0,"oplen_avg":{"avgcount":57,"sum":112},"op":57,"op_r":36,"op_w":21,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":5,"osdop_writefull":2,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":56,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":13,"omap_rd":48,"omap_del":4},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":1,"omap_total_updates":2,"omap_total_removes":1},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":115,"get_sum":136373,"get_or_fail_fail":0,"get_or_fail_success":115,"take":0,"take_sum":0,"put":115,"put_sum":136373,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":57,"take_sum":8476,"put":57,"put_sum":8476,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":57,"take_sum":57,"put":57,"put_sum":57,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x560106079c20":{"val":0,"max":3758096384,"get_started":0,"get":5,"get_sum":6030,"get_or_fail_fail":0,"get_or_fail_success":5,"take":0,"take_sum":0,"put":5,"put_sum":6030,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T18:54:43.112 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 36, "avgtime": 3.37e-07, "sum": 1.216e-05 }, "msgr_recv_bytes": 123512, "msgr_recv_encrypted_bytes": 105824, "msgr_recv_messages": 73, "msgr_running_fast_dispatch_time": 0.002507779, "msgr_running_recv_time": 0.012238824, "msgr_running_send_time": 0.004160231, "msgr_running_total_time": 0.015828407, "msgr_send_bytes": 22337, "msgr_send_encrypted_bytes": 8848, "msgr_send_messages": 45, "msgr_send_messages_queue_lat": { "avgcount": 45, "avgtime": 0.000116198, "sum": 0.005228923 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 10129, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 17, "msgr_running_fast_dispatch_time": 0.000501745, "msgr_running_recv_time": 0.002775052, "msgr_running_send_time": 0.001236327, "msgr_running_total_time": 0.004830116, "msgr_send_bytes": 102344, "msgr_send_encrypted_bytes": 96080, "msgr_send_messages": 25, "msgr_send_messages_queue_lat": { "avgcount": 23, "avgtime": 0.000156197, "sum": 0.003592534 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 6, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 2, "avgtime": 1.64e-07, "sum": 3.28e-07 }, "msgr_recv_bytes": 15583, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 28, "msgr_running_fast_dispatch_time": 0.001076368, "msgr_running_recv_time": 0.003388071, "msgr_running_send_time": 0.001208172, "msgr_running_total_time": 0.006112097, "msgr_send_bytes": 29672, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 30, "msgr_send_messages_queue_lat": { "avgcount": 29, "avgtime": 0.000248681, "sum": 0.007211756 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 59, "avgtime": 0.000127935, "sum": 0.00754821 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 4.7567e-05, "sum": 0.000190271 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 2, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00288072, "sum": 0.005761441 }, "request": 4, "root_rbytes": 7, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 5, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.00384905, "sum": 0.019245253 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4201668 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 40836 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 1, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005288191, "sum": 0.005288191 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.00047325, "sum": 0.00047325 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 792, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 245753, "buffer_anon_items": 73, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 56951, "mds_co_items": 240, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5544, "osdmap_items": 90, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 56, "map_full": 0, "map_inc": 1, "omap_del": 4, "omap_rd": 48, "omap_wr": 13, "op": 57, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 57, "avgtime": 0.001613997, "sum": 0.091997851 }, "op_pg": 0, "op_r": 36, "op_reply": 57, "op_resend": 0, "op_rmw": 0, "op_send": 57, "op_send_bytes": 7406, "op_w": 21, "oplen_avg": { "avgcount": 57, "sum": 112 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 2, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 1, "omap_total_objs": 1, "omap_total_removes": 1, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 115, "get_or_fail_fail": 0, "get_or_fail_success": 115, "get_started": 0, "get_sum": 136373, "max": 104857600, "put": 115, "put_sum": 136373, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 57, "put_sum": 8476, "take": 57, "take_sum": 8476, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 57, "put_sum": 57, "take": 57, "take_sum": 57, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x560106079c20": { "get": 5, "get_or_fail_fail": 0, "get_or_fail_success": 5, "get_started": 0, "get_sum": 6030, "max": 3758096384, "put": 5, "put_sum": 6030, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T18:54:43.112 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-09-13T18:54:43.112 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:43.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7f562700 1 Processor -- start 2024-09-13T18:54:43.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7f562700 1 -- start start 2024-09-13T18:54:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7f562700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f4e78070080 0x7f4e780cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7f562700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e780cce40 con 0x7f4e780cc980 2024-09-13T18:54:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7f562700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e780cd120 con 0x7f4e780b2d00 2024-09-13T18:54:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7f562700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e78071ce0 con 0x7f4e78070080 2024-09-13T18:54:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7cafd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4e780cc980 0x7f4e78070460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51436/0 (socket says 172.21.15.7:51436) 2024-09-13T18:54:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7cafd700 1 -- 172.21.15.7:0/2164512276 learned_addr learned my addr 172.21.15.7:0/2164512276 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7d2fe700 1 --2- 172.21.15.7:0/2164512276 >> v2:172.21.15.171:3300/0 conn(0x7f4e78070080 0x7f4e780cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e77fff700 1 -- 172.21.15.7:0/2164512276 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537491638 0 0) 0x7f4e780cce40 con 0x7f4e780cc980 2024-09-13T18:54:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e77fff700 1 -- 172.21.15.7:0/2164512276 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e600035d0 con 0x7f4e780cc980 2024-09-13T18:54:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e77fff700 1 -- 172.21.15.7:0/2164512276 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 63182826 0 0) 0x7f4e600035d0 con 0x7f4e780cc980 2024-09-13T18:54:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e77fff700 1 -- 172.21.15.7:0/2164512276 >> v2:172.21.15.171:3300/0 conn(0x7f4e78070080 msgr2=0x7f4e780cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e77fff700 1 --2- 172.21.15.7:0/2164512276 >> v2:172.21.15.171:3300/0 conn(0x7f4e78070080 0x7f4e780cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e77fff700 1 -- 172.21.15.7:0/2164512276 >> v1:172.21.15.171:6789/0 conn(0x7f4e780b2d00 legacy=0x7f4e7806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e77fff700 1 -- 172.21.15.7:0/2164512276 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e780cd400 con 0x7f4e780cc980 2024-09-13T18:54:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e77fff700 1 -- 172.21.15.7:0/2164512276 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7f4e6c001670 con 0x7f4e780cc980 2024-09-13T18:54:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e77fff700 1 -- 172.21.15.7:0/2164512276 >> v1:172.21.15.7:6789/0 conn(0x7f4e780cc980 legacy=0x7f4e78070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e77fff700 1 --2- 172.21.15.7:0/2164512276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e60003a90 0x7f4e60003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e77fff700 1 --2- 172.21.15.7:0/2164512276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60004870 0x7f4e60004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e77fff700 1 --2- 172.21.15.7:0/2164512276 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4e60005230 0x7f4e6000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e77fff700 1 -- 172.21.15.7:0/2164512276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6000c100 con 0x7f4e60004870 2024-09-13T18:54:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e77fff700 1 -- 172.21.15.7:0/2164512276 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6000c240 con 0x7f4e60003a90 2024-09-13T18:54:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e77fff700 1 -- 172.21.15.7:0/2164512276 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6000c400 con 0x7f4e60005230 2024-09-13T18:54:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7cafd700 1 --2- 172.21.15.7:0/2164512276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60004870 0x7f4e60004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7d2fe700 1 --2- 172.21.15.7:0/2164512276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e60003a90 0x7f4e60003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7daff700 1 --2- 172.21.15.7:0/2164512276 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4e60005230 0x7f4e6000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7cafd700 1 -- 172.21.15.7:0/2164512276 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4e60005230 msgr2=0x7f4e6000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7cafd700 1 --2- 172.21.15.7:0/2164512276 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4e60005230 0x7f4e6000ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7cafd700 1 -- 172.21.15.7:0/2164512276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e60003a90 msgr2=0x7f4e60003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7cafd700 1 --2- 172.21.15.7:0/2164512276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e60003a90 0x7f4e60003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7cafd700 1 -- 172.21.15.7:0/2164512276 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e600035d0 con 0x7f4e60004870 2024-09-13T18:54:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7daff700 1 --2- 172.21.15.7:0/2164512276 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4e60005230 0x7f4e6000ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7daff700 1 -- 172.21.15.7:0/2164512276 reap_dead start 2024-09-13T18:54:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e7cafd700 1 --2- 172.21.15.7:0/2164512276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60004870 0x7f4e60004cf0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00c2c0 tx=0x7f4e600058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91742e32ff3bd47c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e77fff700 1 -- 172.21.15.7:0/2164512276 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c007b90 con 0x7f4e60004870 2024-09-13T18:54:43.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e77fff700 1 -- 172.21.15.7:0/2164512276 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6c00ce20 con 0x7f4e60004870 2024-09-13T18:54:43.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.222+0000 7f4e77fff700 1 -- 172.21.15.7:0/2164512276 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c005bc0 con 0x7f4e60004870 2024-09-13T18:54:43.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7f562700 1 -- 172.21.15.7:0/2164512276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60004870 msgr2=0x7f4e60004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:43.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7f562700 1 --2- 172.21.15.7:0/2164512276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60004870 0x7f4e60004cf0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00c2c0 tx=0x7f4e600058a0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7f562700 1 -- 172.21.15.7:0/2164512276 shutdown_connections 2024-09-13T18:54:43.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7f562700 1 --2- 172.21.15.7:0/2164512276 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60004870 0x7f4e60004cf0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7f562700 1 -- 172.21.15.7:0/2164512276 >> 172.21.15.7:0/2164512276 conn(0x7f4e780b0830 msgr2=0x7f4e780b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:43.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7f562700 1 -- 172.21.15.7:0/2164512276 shutdown_connections 2024-09-13T18:54:43.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7f562700 1 -- 172.21.15.7:0/2164512276 wait complete. 2024-09-13T18:54:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7f562700 1 Processor -- start 2024-09-13T18:54:43.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7f562700 1 -- start start 2024-09-13T18:54:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7f562700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60004870 0x7f4e781e0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7f562700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e781e2f20 0x7f4e781df880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7f562700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4e781ec1f0 0x7f4e781ec670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7f562700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e781ecbb0 con 0x7f4e60004870 2024-09-13T18:54:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7f562700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e781e33a0 con 0x7f4e781e2f20 2024-09-13T18:54:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7f562700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e781e1e80 con 0x7f4e781ec1f0 2024-09-13T18:54:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7d2fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60004870 0x7f4e781e0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7d2fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60004870 0x7f4e781e0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40174/0 (socket says 172.21.15.7:40174) 2024-09-13T18:54:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7d2fe700 1 -- 172.21.15.7:0/361749535 learned_addr learned my addr 172.21.15.7:0/361749535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7cafd700 1 --2- 172.21.15.7:0/361749535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e781e2f20 0x7f4e781df880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.226+0000 7f4e7daff700 1 --2- 172.21.15.7:0/361749535 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4e781ec1f0 0x7f4e781ec670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:43.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.230+0000 7f4e7d2fe700 1 -- 172.21.15.7:0/361749535 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4e781ec1f0 msgr2=0x7f4e781ec670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.230+0000 7f4e7d2fe700 1 --2- 172.21.15.7:0/361749535 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4e781ec1f0 0x7f4e781ec670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.230+0000 7f4e7d2fe700 1 -- 172.21.15.7:0/361749535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e781e2f20 msgr2=0x7f4e781df880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.230+0000 7f4e7d2fe700 1 --2- 172.21.15.7:0/361749535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e781e2f20 0x7f4e781df880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.230+0000 7f4e7d2fe700 1 -- 172.21.15.7:0/361749535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e781e2160 con 0x7f4e60004870 2024-09-13T18:54:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.230+0000 7f4e7cafd700 1 --2- 172.21.15.7:0/361749535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e781e2f20 0x7f4e781df880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.230+0000 7f4e7d2fe700 1 --2- 172.21.15.7:0/361749535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60004870 0x7f4e781e0a40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4e68011aa0 tx=0x7f4e68011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.230+0000 7f4e75ffb700 1 -- 172.21.15.7:0/361749535 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6800f940 con 0x7f4e60004870 2024-09-13T18:54:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.230+0000 7f4e75ffb700 1 -- 172.21.15.7:0/361749535 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6800faa0 con 0x7f4e60004870 2024-09-13T18:54:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.230+0000 7f4e7f562700 1 -- 172.21.15.7:0/361749535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e781e2440 con 0x7f4e60004870 2024-09-13T18:54:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.230+0000 7f4e75ffb700 1 -- 172.21.15.7:0/361749535 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6800fe50 con 0x7f4e60004870 2024-09-13T18:54:43.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.230+0000 7f4e7f562700 1 -- 172.21.15.7:0/361749535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e781e2a00 con 0x7f4e60004870 2024-09-13T18:54:43.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.230+0000 7f4e7f562700 1 -- 172.21.15.7:0/361749535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e781e2bc0 con 0x7f4e60004870 2024-09-13T18:54:43.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.230+0000 7f4e75ffb700 1 -- 172.21.15.7:0/361749535 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4e68014440 con 0x7f4e60004870 2024-09-13T18:54:43.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.230+0000 7f4e75ffb700 1 --2- 172.21.15.7:0/361749535 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4e64066760 0x7f4e64068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:43.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.234+0000 7f4e75ffb700 1 -- 172.21.15.7:0/361749535 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4e680848e0 con 0x7f4e60004870 2024-09-13T18:54:43.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.234+0000 7f4e7cafd700 1 --2- 172.21.15.7:0/361749535 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4e64066760 0x7f4e64068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:43.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.234+0000 7f4e7cafd700 1 --2- 172.21.15.7:0/361749535 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4e64066760 0x7f4e64068c20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4e781e19c0 tx=0x7f4e6c007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:43.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.242+0000 7f4e75ffb700 1 -- 172.21.15.7:0/361749535 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4e68051b50 con 0x7f4e60004870 2024-09-13T18:54:43.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.450+0000 7f4e7f562700 1 -- 172.21.15.7:0/361749535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e780cdee0 con 0x7f4e60004870 2024-09-13T18:54:43.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.454+0000 7f4e75ffb700 1 -- 172.21.15.7:0/361749535 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f4e68055170 con 0x7f4e60004870 2024-09-13T18:54:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:43.459 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1829974483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1829974483},{"type":"v1","addr":"172.21.15.171:6835","nonce":1829974483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1948505526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1948505526},{"type":"v1","addr":"172.21.15.171:6837","nonce":1948505526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3617964311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3617964311},{"type":"v1","addr":"172.21.15.7:6833","nonce":3617964311}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:21.419627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/3768750832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3768750832},{"type":"v1","addr":"172.21.15.7:6835","nonce":3768750832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:43.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.454+0000 7f4e7f562700 1 -- 172.21.15.7:0/361749535 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4e64066760 msgr2=0x7f4e64068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:43.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.454+0000 7f4e7f562700 1 --2- 172.21.15.7:0/361749535 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4e64066760 0x7f4e64068c20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4e781e19c0 tx=0x7f4e6c007000 comp rx=0 tx=0).stop 2024-09-13T18:54:43.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.454+0000 7f4e7f562700 1 -- 172.21.15.7:0/361749535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60004870 msgr2=0x7f4e781e0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:43.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.454+0000 7f4e7f562700 1 --2- 172.21.15.7:0/361749535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60004870 0x7f4e781e0a40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4e68011aa0 tx=0x7f4e68011db0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.454+0000 7f4e7f562700 1 -- 172.21.15.7:0/361749535 shutdown_connections 2024-09-13T18:54:43.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.454+0000 7f4e7f562700 1 --2- 172.21.15.7:0/361749535 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4e64066760 0x7f4e64068c20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.454+0000 7f4e7f562700 1 --2- 172.21.15.7:0/361749535 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4e781ec1f0 0x7f4e781ec670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.454+0000 7f4e7f562700 1 --2- 172.21.15.7:0/361749535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e781e2f20 0x7f4e781df880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.454+0000 7f4e7f562700 1 --2- 172.21.15.7:0/361749535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60004870 0x7f4e781e0a40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.454+0000 7f4e7f562700 1 -- 172.21.15.7:0/361749535 >> 172.21.15.7:0/361749535 conn(0x7f4e780b0830 msgr2=0x7f4e781e9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:43.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.454+0000 7f4e7f562700 1 -- 172.21.15.7:0/361749535 shutdown_connections 2024-09-13T18:54:43.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.454+0000 7f4e7f562700 1 -- 172.21.15.7:0/361749535 wait complete. 2024-09-13T18:54:43.463 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-09-13T18:54:43.474 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T18:54:43.599 INFO:teuthology.orchestra.run.smithi007.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":74,"msgr_send_messages":45,"msgr_recv_bytes":123930,"msgr_send_bytes":22337,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.016076729,"msgr_running_send_time":0.004256436,"msgr_running_recv_time":0.012288264,"msgr_running_fast_dispatch_time":0.002606590,"msgr_send_messages_queue_lat":{"avgcount":45,"sum":0.005228923,"avgtime":0.000116198},"msgr_handle_ack_lat":{"avgcount":37,"sum":0.000012750,"avgtime":0.000000344},"msgr_recv_encrypted_bytes":105824,"msgr_send_encrypted_bytes":8848},"AsyncMessenger::Worker-1":{"msgr_recv_messages":17,"msgr_send_messages":25,"msgr_recv_bytes":10129,"msgr_send_bytes":102344,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.004830116,"msgr_running_send_time":0.001236327,"msgr_running_recv_time":0.002775052,"msgr_running_fast_dispatch_time":0.000501745,"msgr_send_messages_queue_lat":{"avgcount":23,"sum":0.003592534,"avgtime":0.000156197},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":96080},"AsyncMessenger::Worker-2":{"msgr_recv_messages":28,"msgr_send_messages":30,"msgr_recv_bytes":15583,"msgr_send_bytes":29672,"msgr_created_connections":7,"msgr_active_connections":6,"msgr_running_total_time":0.006112097,"msgr_running_send_time":0.001208172,"msgr_running_recv_time":0.003388071,"msgr_running_fast_dispatch_time":0.001076368,"msgr_send_messages_queue_lat":{"avgcount":29,"sum":0.007211756,"avgtime":0.000248681},"msgr_handle_ack_lat":{"avgcount":2,"sum":0.000000328,"avgtime":0.000000164},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":59,"sum":0.007548210,"avgtime":0.000127935}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000190271,"avgtime":0.000047567}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005761441,"avgtime":0.002880720},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":7,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":2,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":5,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4201668,"rdpos":4195196,"jlat":{"avgcount":5,"sum":0.019245253,"avgtime":0.003849050},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":40836,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":1,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005288191,"avgtime":0.005288191},"req_getattr_latency":{"avgcount":1,"sum":0.000473250,"avgtime":0.000473250},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":792,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":245753,"buffer_anon_items":73,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5544,"osdmap_items":90,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":56951,"mds_co_items":240,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":57,"op_send_bytes":7406,"op_resend":0,"op_reply":57,"op_latency":{"avgcount":57,"sum":0.091997851,"avgtime":0.001613997},"op_inflight":0,"oplen_avg":{"avgcount":57,"sum":112},"op":57,"op_r":36,"op_w":21,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":5,"osdop_writefull":2,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":56,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":13,"omap_rd":48,"omap_del":4},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":1,"omap_total_updates":2,"omap_total_removes":1},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":116,"get_sum":136701,"get_or_fail_fail":0,"get_or_fail_success":116,"take":0,"take_sum":0,"put":116,"put_sum":136701,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":57,"take_sum":8476,"put":57,"put_sum":8476,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":57,"take_sum":57,"put":57,"put_sum":57,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x560106079c20":{"val":0,"max":3758096384,"get_started":0,"get":5,"get_sum":6030,"get_or_fail_fail":0,"get_or_fail_success":5,"take":0,"take_sum":0,"put":5,"put_sum":6030,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T18:54:43.600 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 37, "avgtime": 3.44e-07, "sum": 1.275e-05 }, "msgr_recv_bytes": 123930, "msgr_recv_encrypted_bytes": 105824, "msgr_recv_messages": 74, "msgr_running_fast_dispatch_time": 0.00260659, "msgr_running_recv_time": 0.012288264, "msgr_running_send_time": 0.004256436, "msgr_running_total_time": 0.016076729, "msgr_send_bytes": 22337, "msgr_send_encrypted_bytes": 8848, "msgr_send_messages": 45, "msgr_send_messages_queue_lat": { "avgcount": 45, "avgtime": 0.000116198, "sum": 0.005228923 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 10129, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 17, "msgr_running_fast_dispatch_time": 0.000501745, "msgr_running_recv_time": 0.002775052, "msgr_running_send_time": 0.001236327, "msgr_running_total_time": 0.004830116, "msgr_send_bytes": 102344, "msgr_send_encrypted_bytes": 96080, "msgr_send_messages": 25, "msgr_send_messages_queue_lat": { "avgcount": 23, "avgtime": 0.000156197, "sum": 0.003592534 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 6, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 2, "avgtime": 1.64e-07, "sum": 3.28e-07 }, "msgr_recv_bytes": 15583, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 28, "msgr_running_fast_dispatch_time": 0.001076368, "msgr_running_recv_time": 0.003388071, "msgr_running_send_time": 0.001208172, "msgr_running_total_time": 0.006112097, "msgr_send_bytes": 29672, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 30, "msgr_send_messages_queue_lat": { "avgcount": 29, "avgtime": 0.000248681, "sum": 0.007211756 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 59, "avgtime": 0.000127935, "sum": 0.00754821 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 4.7567e-05, "sum": 0.000190271 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 2, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00288072, "sum": 0.005761441 }, "request": 4, "root_rbytes": 7, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 5, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.00384905, "sum": 0.019245253 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4201668 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 40836 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 1, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005288191, "sum": 0.005288191 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.00047325, "sum": 0.00047325 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 792, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 245753, "buffer_anon_items": 73, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 56951, "mds_co_items": 240, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5544, "osdmap_items": 90, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 56, "map_full": 0, "map_inc": 1, "omap_del": 4, "omap_rd": 48, "omap_wr": 13, "op": 57, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 57, "avgtime": 0.001613997, "sum": 0.091997851 }, "op_pg": 0, "op_r": 36, "op_reply": 57, "op_resend": 0, "op_rmw": 0, "op_send": 57, "op_send_bytes": 7406, "op_w": 21, "oplen_avg": { "avgcount": 57, "sum": 112 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 2, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 1, "omap_total_objs": 1, "omap_total_removes": 1, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 116, "get_or_fail_fail": 0, "get_or_fail_success": 116, "get_started": 0, "get_sum": 136701, "max": 104857600, "put": 116, "put_sum": 136701, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 57, "put_sum": 8476, "take": 57, "take_sum": 8476, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 57, "put_sum": 57, "take": 57, "take_sum": 57, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x560106079c20": { "get": 5, "get_or_fail_fail": 0, "get_or_fail_success": 5, "get_started": 0, "get_sum": 6030, "max": 3758096384, "put": 5, "put_sum": 6030, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T18:54:43.600 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-13T18:54:43.601 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:43.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.706+0000 7fdaacb22700 1 Processor -- start 2024-09-13T18:54:43.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.706+0000 7fdaacb22700 1 -- start start 2024-09-13T18:54:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.706+0000 7fdaacb22700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fdaa81606a0 0x7fdaa80d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.706+0000 7fdaacb22700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdaa80d17a0 0x7fdaa80ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.706+0000 7fdaacb22700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdaa8160b10 0x7fdaa8160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.706+0000 7fdaacb22700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa81614e0 con 0x7fdaa80d17a0 2024-09-13T18:54:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.706+0000 7fdaacb22700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa80ced20 con 0x7fdaa81606a0 2024-09-13T18:54:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.706+0000 7fdaacb22700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa80d1ca0 con 0x7fdaa8160b10 2024-09-13T18:54:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.706+0000 7fdaa6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdaa80d17a0 0x7fdaa80ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.706+0000 7fdaa6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdaa80d17a0 0x7fdaa80ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40186/0 (socket says 172.21.15.7:40186) 2024-09-13T18:54:43.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.706+0000 7fdaa6ffd700 1 -- 172.21.15.7:0/642978200 learned_addr learned my addr 172.21.15.7:0/642978200 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:43.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.706+0000 7fdaa77fe700 1 --2- 172.21.15.7:0/642978200 >> v2:172.21.15.171:3300/0 conn(0x7fdaa81606a0 0x7fdaa80d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:43.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.706+0000 7fdaa7fff700 1 --2- 172.21.15.7:0/642978200 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdaa8160b10 0x7fdaa8160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.706+0000 7fdaa6ffd700 1 -- 172.21.15.7:0/642978200 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdaa8160b10 msgr2=0x7fdaa8160fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.706+0000 7fdaa6ffd700 1 --2- 172.21.15.7:0/642978200 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdaa8160b10 0x7fdaa8160fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.706+0000 7fdaa6ffd700 1 -- 172.21.15.7:0/642978200 >> v2:172.21.15.171:3300/0 conn(0x7fdaa81606a0 msgr2=0x7fdaa80d3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.706+0000 7fdaa6ffd700 1 --2- 172.21.15.7:0/642978200 >> v2:172.21.15.171:3300/0 conn(0x7fdaa81606a0 0x7fdaa80d3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.706+0000 7fdaa6ffd700 1 -- 172.21.15.7:0/642978200 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa80cdd00 con 0x7fdaa80d17a0 2024-09-13T18:54:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.710+0000 7fdaa6ffd700 1 --2- 172.21.15.7:0/642978200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa80d17a0 0x7fdaa80ce7e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fda90009910 tx=0x7fda90009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54845319a4c76f29 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.710+0000 7fdaa67fc700 1 -- 172.21.15.7:0/642978200 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90011830 con 0x7fdaa80d17a0 2024-09-13T18:54:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.710+0000 7fdaa67fc700 1 -- 172.21.15.7:0/642978200 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda90011990 con 0x7fdaa80d17a0 2024-09-13T18:54:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.710+0000 7fdaa67fc700 1 -- 172.21.15.7:0/642978200 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90011d40 con 0x7fdaa80d17a0 2024-09-13T18:54:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.710+0000 7fdaacb22700 1 -- 172.21.15.7:0/642978200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa80d17a0 msgr2=0x7fdaa80ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.710+0000 7fdaacb22700 1 --2- 172.21.15.7:0/642978200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa80d17a0 0x7fdaa80ce7e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fda90009910 tx=0x7fda90009c20 comp rx=0 tx=0).stop 2024-09-13T18:54:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.710+0000 7fdaacb22700 1 -- 172.21.15.7:0/642978200 shutdown_connections 2024-09-13T18:54:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.710+0000 7fdaacb22700 1 --2- 172.21.15.7:0/642978200 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdaa8160b10 0x7fdaa8160fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.710+0000 7fdaacb22700 1 --2- 172.21.15.7:0/642978200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa80d17a0 0x7fdaa80ce7e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.710+0000 7fdaacb22700 1 --2- 172.21.15.7:0/642978200 >> v2:172.21.15.171:3300/0 conn(0x7fdaa81606a0 0x7fdaa80d3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.710+0000 7fdaacb22700 1 -- 172.21.15.7:0/642978200 >> 172.21.15.7:0/642978200 conn(0x7fdaa80d2d10 msgr2=0x7fdaa80d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:43.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.710+0000 7fdaacb22700 1 -- 172.21.15.7:0/642978200 shutdown_connections 2024-09-13T18:54:43.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.710+0000 7fdaacb22700 1 -- 172.21.15.7:0/642978200 wait complete. 2024-09-13T18:54:43.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.710+0000 7fdaacb22700 1 Processor -- start 2024-09-13T18:54:43.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.710+0000 7fdaacb22700 1 -- start start 2024-09-13T18:54:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaacb22700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdaa80d17a0 0x7fdaa80cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaacb22700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa81606a0 0x7fdaa815a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaacb22700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdaa8160b10 0x7fdaa8154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaacb22700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa80c9de0 con 0x7fdaa81606a0 2024-09-13T18:54:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaacb22700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa81f6300 con 0x7fdaa80d17a0 2024-09-13T18:54:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaacb22700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa81f65e0 con 0x7fdaa8160b10 2024-09-13T18:54:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaa6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa81606a0 0x7fdaa815a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaa77fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdaa80d17a0 0x7fdaa80cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaa7fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdaa8160b10 0x7fdaa8154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaa6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa81606a0 0x7fdaa815a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40202/0 (socket says 172.21.15.7:40202) 2024-09-13T18:54:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaa6ffd700 1 -- 172.21.15.7:0/2999837100 learned_addr learned my addr 172.21.15.7:0/2999837100 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaa6ffd700 1 -- 172.21.15.7:0/2999837100 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdaa8160b10 msgr2=0x7fdaa8154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaa6ffd700 1 --2- 172.21.15.7:0/2999837100 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdaa8160b10 0x7fdaa8154ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaa6ffd700 1 -- 172.21.15.7:0/2999837100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdaa80d17a0 msgr2=0x7fdaa80cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaa6ffd700 1 --2- 172.21.15.7:0/2999837100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdaa80d17a0 0x7fdaa80cc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaa6ffd700 1 -- 172.21.15.7:0/2999837100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda90009570 con 0x7fdaa81606a0 2024-09-13T18:54:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaa6ffd700 1 --2- 172.21.15.7:0/2999837100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa81606a0 0x7fdaa815a450 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fda90009860 tx=0x7fda90004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fda9ffff700 1 -- 172.21.15.7:0/2999837100 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90004450 con 0x7fdaa81606a0 2024-09-13T18:54:43.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fda9ffff700 1 -- 172.21.15.7:0/2999837100 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9001e070 con 0x7fdaa81606a0 2024-09-13T18:54:43.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaacb22700 1 -- 172.21.15.7:0/2999837100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa81f68c0 con 0x7fdaa81606a0 2024-09-13T18:54:43.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fda9ffff700 1 -- 172.21.15.7:0/2999837100 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90003900 con 0x7fdaa81606a0 2024-09-13T18:54:43.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaacb22700 1 -- 172.21.15.7:0/2999837100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa80b7790 con 0x7fdaa81606a0 2024-09-13T18:54:43.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.714+0000 7fdaacb22700 1 -- 172.21.15.7:0/2999837100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa80c9f50 con 0x7fdaa81606a0 2024-09-13T18:54:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.718+0000 7fda9ffff700 1 -- 172.21.15.7:0/2999837100 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fda90003a60 con 0x7fdaa81606a0 2024-09-13T18:54:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.718+0000 7fda9ffff700 1 --2- 172.21.15.7:0/2999837100 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fda88066730 0x7fda88068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.718+0000 7fda9ffff700 1 -- 172.21.15.7:0/2999837100 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fda90084ec0 con 0x7fdaa81606a0 2024-09-13T18:54:43.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.718+0000 7fdaa77fe700 1 --2- 172.21.15.7:0/2999837100 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fda88066730 0x7fda88068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:43.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.718+0000 7fdaa77fe700 1 --2- 172.21.15.7:0/2999837100 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fda88066730 0x7fda88068bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0011200 tx=0x7fdaa000d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.726+0000 7fda9ffff700 1 -- 172.21.15.7:0/2999837100 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fda90052320 con 0x7fdaa81606a0 2024-09-13T18:54:43.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.934+0000 7fdaacb22700 1 -- 172.21.15.7:0/2999837100 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdaa80ce7e0 con 0x7fdaa81606a0 2024-09-13T18:54:43.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.934+0000 7fda9ffff700 1 -- 172.21.15.7:0/2999837100 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4735 (secure 0 0 0) 0x7fda90055940 con 0x7fdaa81606a0 2024-09-13T18:54:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:43.940 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1829974483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1829974483},{"type":"v1","addr":"172.21.15.171:6835","nonce":1829974483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1948505526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1948505526},{"type":"v1","addr":"172.21.15.171:6837","nonce":1948505526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3617964311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3617964311},{"type":"v1","addr":"172.21.15.7:6833","nonce":3617964311}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:21.419627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/3768750832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3768750832},{"type":"v1","addr":"172.21.15.7:6835","nonce":3768750832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.934+0000 7fdaacb22700 1 -- 172.21.15.7:0/2999837100 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fda88066730 msgr2=0x7fda88068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.938+0000 7fdaacb22700 1 --2- 172.21.15.7:0/2999837100 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fda88066730 0x7fda88068bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0011200 tx=0x7fdaa000d040 comp rx=0 tx=0).stop 2024-09-13T18:54:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.938+0000 7fdaacb22700 1 -- 172.21.15.7:0/2999837100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa81606a0 msgr2=0x7fdaa815a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.938+0000 7fdaacb22700 1 --2- 172.21.15.7:0/2999837100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa81606a0 0x7fdaa815a450 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fda90009860 tx=0x7fda90004770 comp rx=0 tx=0).stop 2024-09-13T18:54:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.938+0000 7fdaacb22700 1 -- 172.21.15.7:0/2999837100 shutdown_connections 2024-09-13T18:54:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.938+0000 7fdaacb22700 1 --2- 172.21.15.7:0/2999837100 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fda88066730 0x7fda88068bf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.938+0000 7fdaacb22700 1 --2- 172.21.15.7:0/2999837100 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdaa8160b10 0x7fdaa8154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.938+0000 7fdaacb22700 1 --2- 172.21.15.7:0/2999837100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdaa81606a0 0x7fdaa815a450 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.938+0000 7fdaacb22700 1 --2- 172.21.15.7:0/2999837100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdaa80d17a0 0x7fdaa80cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.938+0000 7fdaacb22700 1 -- 172.21.15.7:0/2999837100 >> 172.21.15.7:0/2999837100 conn(0x7fdaa80d2d10 msgr2=0x7fdaa815b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.938+0000 7fdaacb22700 1 -- 172.21.15.7:0/2999837100 shutdown_connections 2024-09-13T18:54:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:43.938+0000 7fdaacb22700 1 -- 172.21.15.7:0/2999837100 wait complete. 2024-09-13T18:54:43.944 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-09-13T18:54:43.955 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json perf dump 2024-09-13T18:54:44.070 INFO:teuthology.orchestra.run.smithi007.stdout:{"AsyncMessenger::Worker-0":{"msgr_recv_messages":74,"msgr_send_messages":45,"msgr_recv_bytes":123930,"msgr_send_bytes":22337,"msgr_created_connections":6,"msgr_active_connections":2,"msgr_running_total_time":0.016076729,"msgr_running_send_time":0.004256436,"msgr_running_recv_time":0.012288264,"msgr_running_fast_dispatch_time":0.002606590,"msgr_send_messages_queue_lat":{"avgcount":45,"sum":0.005228923,"avgtime":0.000116198},"msgr_handle_ack_lat":{"avgcount":37,"sum":0.000012750,"avgtime":0.000000344},"msgr_recv_encrypted_bytes":105824,"msgr_send_encrypted_bytes":8848},"AsyncMessenger::Worker-1":{"msgr_recv_messages":17,"msgr_send_messages":25,"msgr_recv_bytes":10129,"msgr_send_bytes":102344,"msgr_created_connections":6,"msgr_active_connections":5,"msgr_running_total_time":0.004830116,"msgr_running_send_time":0.001236327,"msgr_running_recv_time":0.002775052,"msgr_running_fast_dispatch_time":0.000501745,"msgr_send_messages_queue_lat":{"avgcount":23,"sum":0.003592534,"avgtime":0.000156197},"msgr_handle_ack_lat":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"msgr_recv_encrypted_bytes":160,"msgr_send_encrypted_bytes":96080},"AsyncMessenger::Worker-2":{"msgr_recv_messages":28,"msgr_send_messages":30,"msgr_recv_bytes":15583,"msgr_send_bytes":29672,"msgr_created_connections":7,"msgr_active_connections":6,"msgr_running_total_time":0.006112097,"msgr_running_send_time":0.001208172,"msgr_running_recv_time":0.003388071,"msgr_running_fast_dispatch_time":0.001076368,"msgr_send_messages_queue_lat":{"avgcount":29,"sum":0.007211756,"avgtime":0.000248681},"msgr_handle_ack_lat":{"avgcount":2,"sum":0.000000328,"avgtime":0.000000164},"msgr_recv_encrypted_bytes":0,"msgr_send_encrypted_bytes":0},"cct":{"total_workers":1,"unhealthy_workers":0},"finisher-MDSRank":{"queue_len":0,"complete_latency":{"avgcount":59,"sum":0.007548210,"avgtime":0.000127935}},"finisher-PurgeQueue":{"queue_len":0,"complete_latency":{"avgcount":4,"sum":0.000190271,"avgtime":0.000047567}},"mds":{"request":4,"reply":2,"reply_latency":{"avgcount":2,"sum":0.005761441,"avgtime":0.002880720},"slow_reply":0,"forward":0,"dir_fetch_complete":12,"dir_fetch_keys":10,"dir_commit":0,"dir_split":0,"dir_merge":0,"inodes":11,"inodes_top":0,"inodes_bottom":11,"inodes_pin_tail":0,"inodes_pinned":11,"inodes_expired":0,"inodes_with_caps":2,"caps":2,"subtrees":2,"traverse":4,"traverse_hit":4,"traverse_forward":0,"traverse_discover":0,"traverse_dir_fetch":0,"traverse_remote_ino":0,"traverse_lock":0,"load_cent":0,"q":0,"exported":0,"exported_inodes":0,"imported":0,"imported_inodes":0,"openino_dir_fetch":0,"openino_backtrace_fetch":0,"openino_peer_discover":0,"root_rfiles":1,"root_rbytes":7,"root_rsnaps":0,"scrub_backtrace_fetch":0,"scrub_set_tag":0,"scrub_backtrace_repaired":0,"scrub_inotable_repaired":0,"scrub_dir_inodes":0,"scrub_dir_base_inodes":0,"scrub_dirfrag_rstats":0,"scrub_file_inodes":0,"handle_inode_file_caps":0,"ceph_cap_op_revoke":0,"ceph_cap_op_grant":0,"ceph_cap_op_trunc":0,"ceph_cap_op_flushsnap_ack":0,"ceph_cap_op_flush_ack":0,"handle_client_caps":2,"handle_client_caps_dirty":1,"handle_client_cap_release":0,"process_request_cap_release":1},"mds_cache":{"dir_update":0,"dir_update_receipt":0,"dir_try_discover":0,"dir_send_discover":0,"dir_handle_discover":0,"num_strays":0,"num_strays_delayed":0,"num_strays_enqueuing":0,"strays_created":0,"strays_enqueued":0,"strays_reintegrated":0,"strays_migrated":0,"num_recovering_processing":0,"num_recovering_enqueued":0,"num_recovering_prioritized":0,"recovery_started":0,"recovery_completed":0,"ireq_enqueue_scrub":0,"ireq_exportdir":0,"ireq_flush":0,"ireq_fragmentdir":0,"ireq_fragstats":0,"ireq_inodestats":0},"mds_log":{"evadd":5,"evex":0,"evtrm":0,"ev":7,"evexg":0,"evexd":0,"segadd":0,"segex":0,"segtrm":0,"seg":1,"segexg":0,"segexd":0,"expos":4194304,"wrpos":4201668,"rdpos":4195196,"jlat":{"avgcount":5,"sum":0.019245253,"avgtime":0.003849050},"replayed":2},"mds_mem":{"ino":14,"ino+":14,"ino-":0,"dir":12,"dir+":12,"dir-":0,"dn":11,"dn+":11,"dn-":0,"cap":2,"cap+":2,"cap-":0,"rss":40836,"heap":223608},"mds_server":{"dispatch_client_request":3,"dispatch_server_request":0,"handle_client_request":4,"handle_client_session":1,"handle_peer_request":0,"req_create_latency":{"avgcount":1,"sum":0.005288191,"avgtime":0.005288191},"req_getattr_latency":{"avgcount":1,"sum":0.000473250,"avgtime":0.000473250},"req_getfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_link_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookup_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookuphash_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupino_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupname_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupparent_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lookupsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_lssnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mkdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mknod_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_mksnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_open_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_readdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rename_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_renamesnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_snapdiff_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmdir_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmsnap_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_rmxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setdirlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setfilelock_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setlayout_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_setxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_symlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"req_unlink_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000},"cap_revoke_eviction":0,"cap_acquisition_throttle":0,"req_getvxattr_latency":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"mds_sessions":{"session_count":1,"session_add":1,"session_remove":0,"sessions_open":1,"sessions_stale":0,"total_load":1,"average_load":1,"avg_session_uptime":792,"mdthresh_evicted":0},"mempool":{"bloom_filter_bytes":0,"bloom_filter_items":0,"bluestore_alloc_bytes":0,"bluestore_alloc_items":0,"bluestore_cache_data_bytes":0,"bluestore_cache_data_items":0,"bluestore_cache_onode_bytes":0,"bluestore_cache_onode_items":0,"bluestore_cache_meta_bytes":0,"bluestore_cache_meta_items":0,"bluestore_cache_other_bytes":0,"bluestore_cache_other_items":0,"bluestore_cache_buffer_bytes":0,"bluestore_cache_buffer_items":0,"bluestore_extent_bytes":0,"bluestore_extent_items":0,"bluestore_blob_bytes":0,"bluestore_blob_items":0,"bluestore_shared_blob_bytes":0,"bluestore_shared_blob_items":0,"bluestore_inline_bl_bytes":0,"bluestore_inline_bl_items":0,"bluestore_fsck_bytes":0,"bluestore_fsck_items":0,"bluestore_txc_bytes":0,"bluestore_txc_items":0,"bluestore_writing_deferred_bytes":0,"bluestore_writing_deferred_items":0,"bluestore_writing_bytes":0,"bluestore_writing_items":0,"bluefs_bytes":0,"bluefs_items":0,"bluefs_file_reader_bytes":0,"bluefs_file_reader_items":0,"bluefs_file_writer_bytes":0,"bluefs_file_writer_items":0,"buffer_anon_bytes":245753,"buffer_anon_items":73,"buffer_meta_bytes":0,"buffer_meta_items":0,"osd_bytes":0,"osd_items":0,"osd_mapbl_bytes":0,"osd_mapbl_items":0,"osd_pglog_bytes":0,"osd_pglog_items":0,"osdmap_bytes":5544,"osdmap_items":90,"osdmap_mapping_bytes":0,"osdmap_mapping_items":0,"pgmap_bytes":0,"pgmap_items":0,"mds_co_bytes":56951,"mds_co_items":240,"unittest_1_bytes":0,"unittest_1_items":0,"unittest_2_bytes":0,"unittest_2_items":0},"objecter":{"op_active":0,"op_laggy":0,"op_send":57,"op_send_bytes":7406,"op_resend":0,"op_reply":57,"op_latency":{"avgcount":57,"sum":0.091997851,"avgtime":0.001613997},"op_inflight":0,"oplen_avg":{"avgcount":57,"sum":112},"op":57,"op_r":36,"op_w":21,"op_rmw":0,"op_pg":0,"osdop_stat":2,"osdop_create":0,"osdop_read":8,"osdop_write":5,"osdop_writefull":2,"osdop_writesame":0,"osdop_append":0,"osdop_zero":1,"osdop_truncate":0,"osdop_delete":5,"osdop_mapext":0,"osdop_sparse_read":0,"osdop_clonerange":0,"osdop_getxattr":24,"osdop_setxattr":0,"osdop_cmpxattr":0,"osdop_rmxattr":0,"osdop_resetxattrs":0,"osdop_call":0,"osdop_watch":0,"osdop_notify":0,"osdop_src_cmpxattr":0,"osdop_pgls":0,"osdop_pgls_filter":0,"osdop_other":0,"linger_active":0,"linger_send":0,"linger_resend":0,"linger_ping":0,"poolop_active":0,"poolop_send":0,"poolop_resend":0,"poolstat_active":0,"poolstat_send":0,"poolstat_resend":0,"statfs_active":0,"statfs_send":0,"statfs_resend":0,"command_active":0,"command_send":0,"command_resend":0,"map_epoch":56,"map_full":0,"map_inc":1,"osd_sessions":7,"osd_session_open":7,"osd_session_close":0,"osd_laggy":0,"omap_wr":13,"omap_rd":48,"omap_del":4},"oft":{"omap_total_objs":1,"omap_total_kv_pairs":1,"omap_total_updates":2,"omap_total_removes":1},"osd-slow-ops":{"slow_ops_count":0},"purge_queue":{"pq_executing_ops":0,"pq_executing_ops_high_water":0,"pq_executing":0,"pq_executing_high_water":0,"pq_executed_ops":0,"pq_executed":0,"pq_item_in_journal":0},"throttle-msgr_dispatch_throttler-mds":{"val":0,"max":104857600,"get_started":0,"get":116,"get_sum":136701,"get_or_fail_fail":0,"get_or_fail_success":116,"take":0,"take_sum":0,"put":116,"put_sum":136701,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_bytes":{"val":0,"max":104857600,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":57,"take_sum":8476,"put":57,"put_sum":8476,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-objecter_ops":{"val":0,"max":1024,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":57,"take_sum":57,"put":57,"put_sum":57,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle":{"val":0,"max":3758096384,"get_started":0,"get":0,"get_sum":0,"get_or_fail_fail":0,"get_or_fail_success":0,"take":0,"take_sum":0,"put":0,"put_sum":0,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}},"throttle-write_buf_throttle-0x560106079c20":{"val":0,"max":3758096384,"get_started":0,"get":5,"get_sum":6030,"get_or_fail_fail":0,"get_or_fail_success":5,"take":0,"take_sum":0,"put":5,"put_sum":6030,"wait":{"avgcount":0,"sum":0.000000000,"avgtime":0.000000000}}} 2024-09-13T18:54:44.072 DEBUG:tasks.cephfs.filesystem:_json_asok output { "AsyncMessenger::Worker-0": { "msgr_active_connections": 2, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 37, "avgtime": 3.44e-07, "sum": 1.275e-05 }, "msgr_recv_bytes": 123930, "msgr_recv_encrypted_bytes": 105824, "msgr_recv_messages": 74, "msgr_running_fast_dispatch_time": 0.00260659, "msgr_running_recv_time": 0.012288264, "msgr_running_send_time": 0.004256436, "msgr_running_total_time": 0.016076729, "msgr_send_bytes": 22337, "msgr_send_encrypted_bytes": 8848, "msgr_send_messages": 45, "msgr_send_messages_queue_lat": { "avgcount": 45, "avgtime": 0.000116198, "sum": 0.005228923 } }, "AsyncMessenger::Worker-1": { "msgr_active_connections": 5, "msgr_created_connections": 6, "msgr_handle_ack_lat": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "msgr_recv_bytes": 10129, "msgr_recv_encrypted_bytes": 160, "msgr_recv_messages": 17, "msgr_running_fast_dispatch_time": 0.000501745, "msgr_running_recv_time": 0.002775052, "msgr_running_send_time": 0.001236327, "msgr_running_total_time": 0.004830116, "msgr_send_bytes": 102344, "msgr_send_encrypted_bytes": 96080, "msgr_send_messages": 25, "msgr_send_messages_queue_lat": { "avgcount": 23, "avgtime": 0.000156197, "sum": 0.003592534 } }, "AsyncMessenger::Worker-2": { "msgr_active_connections": 6, "msgr_created_connections": 7, "msgr_handle_ack_lat": { "avgcount": 2, "avgtime": 1.64e-07, "sum": 3.28e-07 }, "msgr_recv_bytes": 15583, "msgr_recv_encrypted_bytes": 0, "msgr_recv_messages": 28, "msgr_running_fast_dispatch_time": 0.001076368, "msgr_running_recv_time": 0.003388071, "msgr_running_send_time": 0.001208172, "msgr_running_total_time": 0.006112097, "msgr_send_bytes": 29672, "msgr_send_encrypted_bytes": 0, "msgr_send_messages": 30, "msgr_send_messages_queue_lat": { "avgcount": 29, "avgtime": 0.000248681, "sum": 0.007211756 } }, "cct": { "total_workers": 1, "unhealthy_workers": 0 }, "finisher-MDSRank": { "complete_latency": { "avgcount": 59, "avgtime": 0.000127935, "sum": 0.00754821 }, "queue_len": 0 }, "finisher-PurgeQueue": { "complete_latency": { "avgcount": 4, "avgtime": 4.7567e-05, "sum": 0.000190271 }, "queue_len": 0 }, "mds": { "caps": 2, "ceph_cap_op_flush_ack": 0, "ceph_cap_op_flushsnap_ack": 0, "ceph_cap_op_grant": 0, "ceph_cap_op_revoke": 0, "ceph_cap_op_trunc": 0, "dir_commit": 0, "dir_fetch_complete": 12, "dir_fetch_keys": 10, "dir_merge": 0, "dir_split": 0, "exported": 0, "exported_inodes": 0, "forward": 0, "handle_client_cap_release": 0, "handle_client_caps": 2, "handle_client_caps_dirty": 1, "handle_inode_file_caps": 0, "imported": 0, "imported_inodes": 0, "inodes": 11, "inodes_bottom": 11, "inodes_expired": 0, "inodes_pin_tail": 0, "inodes_pinned": 11, "inodes_top": 0, "inodes_with_caps": 2, "load_cent": 0, "openino_backtrace_fetch": 0, "openino_dir_fetch": 0, "openino_peer_discover": 0, "process_request_cap_release": 1, "q": 0, "reply": 2, "reply_latency": { "avgcount": 2, "avgtime": 0.00288072, "sum": 0.005761441 }, "request": 4, "root_rbytes": 7, "root_rfiles": 1, "root_rsnaps": 0, "scrub_backtrace_fetch": 0, "scrub_backtrace_repaired": 0, "scrub_dir_base_inodes": 0, "scrub_dir_inodes": 0, "scrub_dirfrag_rstats": 0, "scrub_file_inodes": 0, "scrub_inotable_repaired": 0, "scrub_set_tag": 0, "slow_reply": 0, "subtrees": 2, "traverse": 4, "traverse_dir_fetch": 0, "traverse_discover": 0, "traverse_forward": 0, "traverse_hit": 4, "traverse_lock": 0, "traverse_remote_ino": 0 }, "mds_cache": { "dir_handle_discover": 0, "dir_send_discover": 0, "dir_try_discover": 0, "dir_update": 0, "dir_update_receipt": 0, "ireq_enqueue_scrub": 0, "ireq_exportdir": 0, "ireq_flush": 0, "ireq_fragmentdir": 0, "ireq_fragstats": 0, "ireq_inodestats": 0, "num_recovering_enqueued": 0, "num_recovering_prioritized": 0, "num_recovering_processing": 0, "num_strays": 0, "num_strays_delayed": 0, "num_strays_enqueuing": 0, "recovery_completed": 0, "recovery_started": 0, "strays_created": 0, "strays_enqueued": 0, "strays_migrated": 0, "strays_reintegrated": 0 }, "mds_log": { "ev": 7, "evadd": 5, "evex": 0, "evexd": 0, "evexg": 0, "evtrm": 0, "expos": 4194304, "jlat": { "avgcount": 5, "avgtime": 0.00384905, "sum": 0.019245253 }, "rdpos": 4195196, "replayed": 2, "seg": 1, "segadd": 0, "segex": 0, "segexd": 0, "segexg": 0, "segtrm": 0, "wrpos": 4201668 }, "mds_mem": { "cap": 2, "cap+": 2, "cap-": 0, "dir": 12, "dir+": 12, "dir-": 0, "dn": 11, "dn+": 11, "dn-": 0, "heap": 223608, "ino": 14, "ino+": 14, "ino-": 0, "rss": 40836 }, "mds_server": { "cap_acquisition_throttle": 0, "cap_revoke_eviction": 0, "dispatch_client_request": 3, "dispatch_server_request": 0, "handle_client_request": 4, "handle_client_session": 1, "handle_peer_request": 0, "req_create_latency": { "avgcount": 1, "avgtime": 0.005288191, "sum": 0.005288191 }, "req_getattr_latency": { "avgcount": 1, "avgtime": 0.00047325, "sum": 0.00047325 }, "req_getfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_getvxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_link_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookup_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookuphash_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupino_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupname_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupparent_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lookupsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_lssnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mkdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mknod_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_mksnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_open_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_readdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rename_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_renamesnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmdir_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmsnap_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_rmxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setdirlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setfilelock_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setlayout_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_setxattr_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_snapdiff_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_symlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 }, "req_unlink_latency": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "mds_sessions": { "average_load": 1, "avg_session_uptime": 792, "mdthresh_evicted": 0, "session_add": 1, "session_count": 1, "session_remove": 0, "sessions_open": 1, "sessions_stale": 0, "total_load": 1 }, "mempool": { "bloom_filter_bytes": 0, "bloom_filter_items": 0, "bluefs_bytes": 0, "bluefs_file_reader_bytes": 0, "bluefs_file_reader_items": 0, "bluefs_file_writer_bytes": 0, "bluefs_file_writer_items": 0, "bluefs_items": 0, "bluestore_alloc_bytes": 0, "bluestore_alloc_items": 0, "bluestore_blob_bytes": 0, "bluestore_blob_items": 0, "bluestore_cache_buffer_bytes": 0, "bluestore_cache_buffer_items": 0, "bluestore_cache_data_bytes": 0, "bluestore_cache_data_items": 0, "bluestore_cache_meta_bytes": 0, "bluestore_cache_meta_items": 0, "bluestore_cache_onode_bytes": 0, "bluestore_cache_onode_items": 0, "bluestore_cache_other_bytes": 0, "bluestore_cache_other_items": 0, "bluestore_extent_bytes": 0, "bluestore_extent_items": 0, "bluestore_fsck_bytes": 0, "bluestore_fsck_items": 0, "bluestore_inline_bl_bytes": 0, "bluestore_inline_bl_items": 0, "bluestore_shared_blob_bytes": 0, "bluestore_shared_blob_items": 0, "bluestore_txc_bytes": 0, "bluestore_txc_items": 0, "bluestore_writing_bytes": 0, "bluestore_writing_deferred_bytes": 0, "bluestore_writing_deferred_items": 0, "bluestore_writing_items": 0, "buffer_anon_bytes": 245753, "buffer_anon_items": 73, "buffer_meta_bytes": 0, "buffer_meta_items": 0, "mds_co_bytes": 56951, "mds_co_items": 240, "osd_bytes": 0, "osd_items": 0, "osd_mapbl_bytes": 0, "osd_mapbl_items": 0, "osd_pglog_bytes": 0, "osd_pglog_items": 0, "osdmap_bytes": 5544, "osdmap_items": 90, "osdmap_mapping_bytes": 0, "osdmap_mapping_items": 0, "pgmap_bytes": 0, "pgmap_items": 0, "unittest_1_bytes": 0, "unittest_1_items": 0, "unittest_2_bytes": 0, "unittest_2_items": 0 }, "objecter": { "command_active": 0, "command_resend": 0, "command_send": 0, "linger_active": 0, "linger_ping": 0, "linger_resend": 0, "linger_send": 0, "map_epoch": 56, "map_full": 0, "map_inc": 1, "omap_del": 4, "omap_rd": 48, "omap_wr": 13, "op": 57, "op_Infinitylight": 0, "op_active": 0, "op_laggy": 0, "op_latency": { "avgcount": 57, "avgtime": 0.001613997, "sum": 0.091997851 }, "op_pg": 0, "op_r": 36, "op_reply": 57, "op_resend": 0, "op_rmw": 0, "op_send": 57, "op_send_bytes": 7406, "op_w": 21, "oplen_avg": { "avgcount": 57, "sum": 112 }, "osd_laggy": 0, "osd_session_close": 0, "osd_session_open": 7, "osd_sessions": 7, "osdop_append": 0, "osdop_call": 0, "osdop_clonerange": 0, "osdop_cmpxattr": 0, "osdop_create": 0, "osdop_delete": 5, "osdop_getxattr": 24, "osdop_mapext": 0, "osdop_notify": 0, "osdop_other": 0, "osdop_pgls": 0, "osdop_pgls_filter": 0, "osdop_read": 8, "osdop_resetxattrs": 0, "osdop_rmxattr": 0, "osdop_setxattr": 0, "osdop_sparse_read": 0, "osdop_src_cmpxattr": 0, "osdop_stat": 2, "osdop_truncate": 0, "osdop_watch": 0, "osdop_write": 5, "osdop_writefull": 2, "osdop_writesame": 0, "osdop_zero": 1, "poolop_active": 0, "poolop_resend": 0, "poolop_send": 0, "poolstat_active": 0, "poolstat_resend": 0, "poolstat_send": 0, "statfs_active": 0, "statfs_resend": 0, "statfs_send": 0 }, "oft": { "omap_total_kv_pairs": 1, "omap_total_objs": 1, "omap_total_removes": 1, "omap_total_updates": 2 }, "osd-slow-ops": { "slow_ops_count": 0 }, "purge_queue": { "pq_executed": 0, "pq_executed_ops": 0, "pq_executing": 0, "pq_executing_high_water": 0, "pq_executing_ops": 0, "pq_executing_ops_high_water": 0, "pq_item_in_journal": 0 }, "throttle-msgr_dispatch_throttler-mds": { "get": 116, "get_or_fail_fail": 0, "get_or_fail_success": 116, "get_started": 0, "get_sum": 136701, "max": 104857600, "put": 116, "put_sum": 136701, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_bytes": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 104857600, "put": 57, "put_sum": 8476, "take": 57, "take_sum": 8476, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-objecter_ops": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 1024, "put": 57, "put_sum": 57, "take": 57, "take_sum": 57, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle": { "get": 0, "get_or_fail_fail": 0, "get_or_fail_success": 0, "get_started": 0, "get_sum": 0, "max": 3758096384, "put": 0, "put_sum": 0, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } }, "throttle-write_buf_throttle-0x560106079c20": { "get": 5, "get_or_fail_fail": 0, "get_or_fail_success": 5, "get_started": 0, "get_sum": 6030, "max": 3758096384, "put": 5, "put_sum": 6030, "take": 0, "take_sum": 0, "val": 0, "wait": { "avgcount": 0, "avgtime": 0.0, "sum": 0.0 } } } 2024-09-13T18:54:44.072 INFO:tasks.cephfs.test_openfiletable:omap_total_removes:1 2024-09-13T18:54:44.072 INFO:tasks.cephfs.test_openfiletable:omap_total_kv_pairs:1 2024-09-13T18:54:44.073 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T18:54:44.073 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T18:54:44.092 DEBUG:teuthology.orchestra.run.smithi171:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T18:54:44.113 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T18:54:44.113 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:54:44.113 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:54:44.120 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:54:44.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.274+0000 7f3ae2207700 1 Processor -- start 2024-09-13T18:54:44.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.274+0000 7f3ae2207700 1 -- start start 2024-09-13T18:54:44.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3ae2207700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f3adc0cc980 0x7f3adc0cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:44.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3ae2207700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3adc0b2d00 0x7f3adc06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:44.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3ae2207700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3adc0cce40 con 0x7f3adc070080 2024-09-13T18:54:44.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3ae2207700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3adc071ce0 con 0x7f3adc0b2d00 2024-09-13T18:54:44.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3ae2207700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3adc0cd120 con 0x7f3adc0cc980 2024-09-13T18:54:44.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3adbfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3adc0b2d00 0x7f3adc06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:44.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3adbfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3adc0b2d00 0x7f3adc06bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40212/0 (socket says 172.21.15.7:40212) 2024-09-13T18:54:44.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3adbfff700 1 -- 172.21.15.7:0/1398331465 learned_addr learned my addr 172.21.15.7:0/1398331465 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:44.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3adaffd700 1 --2- 172.21.15.7:0/1398331465 >> v2:172.21.15.171:3300/0 conn(0x7f3adc0cc980 0x7f3adc0cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:44.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3adbfff700 1 -- 172.21.15.7:0/1398331465 >> v2:172.21.15.171:3300/0 conn(0x7f3adc0cc980 msgr2=0x7f3adc0cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:44.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3adbfff700 1 --2- 172.21.15.7:0/1398331465 >> v2:172.21.15.171:3300/0 conn(0x7f3adc0cc980 0x7f3adc0cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3adbfff700 1 -- 172.21.15.7:0/1398331465 >> v1:172.21.15.171:6789/0 conn(0x7f3adc070080 legacy=0x7f3adc070460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:44.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3adbfff700 1 -- 172.21.15.7:0/1398331465 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3adc0cd400 con 0x7f3adc0b2d00 2024-09-13T18:54:44.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3adbfff700 1 --2- 172.21.15.7:0/1398331465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3adc0b2d00 0x7f3adc06bbf0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3adc071c30 tx=0x7f3ad4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e555aebda33338a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:44.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3ada7fc700 1 -- 172.21.15.7:0/1398331465 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4011830 con 0x7f3adc0b2d00 2024-09-13T18:54:44.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3ada7fc700 1 -- 172.21.15.7:0/1398331465 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad4011990 con 0x7f3adc0b2d00 2024-09-13T18:54:44.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3ada7fc700 1 -- 172.21.15.7:0/1398331465 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4011d20 con 0x7f3adc0b2d00 2024-09-13T18:54:44.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3ae2207700 1 -- 172.21.15.7:0/1398331465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3adc0b2d00 msgr2=0x7f3adc06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:44.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3ae2207700 1 --2- 172.21.15.7:0/1398331465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3adc0b2d00 0x7f3adc06bbf0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3adc071c30 tx=0x7f3ad4009bf0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3ae2207700 1 -- 172.21.15.7:0/1398331465 shutdown_connections 2024-09-13T18:54:44.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3ae2207700 1 --2- 172.21.15.7:0/1398331465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3adc0b2d00 0x7f3adc06bbf0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3ae2207700 1 --2- 172.21.15.7:0/1398331465 >> v2:172.21.15.171:3300/0 conn(0x7f3adc0cc980 0x7f3adc0cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3ae2207700 1 -- 172.21.15.7:0/1398331465 >> 172.21.15.7:0/1398331465 conn(0x7f3adc0b0830 msgr2=0x7f3adc0b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:44.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3ae2207700 1 -- 172.21.15.7:0/1398331465 shutdown_connections 2024-09-13T18:54:44.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.278+0000 7f3ae2207700 1 -- 172.21.15.7:0/1398331465 wait complete. 2024-09-13T18:54:44.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.282+0000 7f3ae2207700 1 Processor -- start 2024-09-13T18:54:44.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.282+0000 7f3ae2207700 1 -- start start 2024-09-13T18:54:44.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3ae2207700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3adc070080 0x7f3adc1d5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:44.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3ae2207700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3adc0b2d00 0x7f3adc1d6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:44.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3ae2207700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3adc0cc980 0x7f3adc1ebab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:44.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3ae2207700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc1d6b30 con 0x7f3adc070080 2024-09-13T18:54:44.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3ae2207700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc1d5f20 con 0x7f3adc0cc980 2024-09-13T18:54:44.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3ae2207700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3adc1d6200 con 0x7f3adc0b2d00 2024-09-13T18:54:44.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3adb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3adc070080 0x7f3adc1d5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:44.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3adb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3adc070080 0x7f3adc1d5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40224/0 (socket says 172.21.15.7:40224) 2024-09-13T18:54:44.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3adb7fe700 1 -- 172.21.15.7:0/291671384 learned_addr learned my addr 172.21.15.7:0/291671384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:44.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3adbfff700 1 --2- 172.21.15.7:0/291671384 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3adc0cc980 0x7f3adc1ebab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:44.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3adaffd700 1 --2- 172.21.15.7:0/291671384 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3adc0b2d00 0x7f3adc1d6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:44.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3adb7fe700 1 -- 172.21.15.7:0/291671384 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3adc0b2d00 msgr2=0x7f3adc1d6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:44.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3adb7fe700 1 --2- 172.21.15.7:0/291671384 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3adc0b2d00 0x7f3adc1d6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3adb7fe700 1 -- 172.21.15.7:0/291671384 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3adc0cc980 msgr2=0x7f3adc1ebab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:44.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3adb7fe700 1 --2- 172.21.15.7:0/291671384 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3adc0cc980 0x7f3adc1ebab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3adb7fe700 1 -- 172.21.15.7:0/291671384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad4009570 con 0x7f3adc070080 2024-09-13T18:54:44.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3adb7fe700 1 --2- 172.21.15.7:0/291671384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3adc070080 0x7f3adc1d5820 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400c500 tx=0x7f3ac400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:44.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3ad3fff700 1 -- 172.21.15.7:0/291671384 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac40052d0 con 0x7f3adc070080 2024-09-13T18:54:44.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3ad3fff700 1 -- 172.21.15.7:0/291671384 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac4005430 con 0x7f3adc070080 2024-09-13T18:54:44.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3ae2207700 1 -- 172.21.15.7:0/291671384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3adc1d64e0 con 0x7f3adc070080 2024-09-13T18:54:44.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3ad3fff700 1 -- 172.21.15.7:0/291671384 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac40055a0 con 0x7f3adc070080 2024-09-13T18:54:44.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.286+0000 7f3ae2207700 1 -- 172.21.15.7:0/291671384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3adc1d7500 con 0x7f3adc070080 2024-09-13T18:54:44.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.290+0000 7f3ae2207700 1 -- 172.21.15.7:0/291671384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3adc0cdee0 con 0x7f3adc070080 2024-09-13T18:54:44.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.290+0000 7f3ad3fff700 1 -- 172.21.15.7:0/291671384 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3ac40038e0 con 0x7f3adc070080 2024-09-13T18:54:44.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.290+0000 7f3ad3fff700 1 --2- 172.21.15.7:0/291671384 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3abc066760 0x7f3abc068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:44.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.290+0000 7f3ad3fff700 1 -- 172.21.15.7:0/291671384 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3ac40835c0 con 0x7f3adc070080 2024-09-13T18:54:44.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.290+0000 7f3adaffd700 1 --2- 172.21.15.7:0/291671384 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3abc066760 0x7f3abc068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:44.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.290+0000 7f3adaffd700 1 --2- 172.21.15.7:0/291671384 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3abc066760 0x7f3abc068c20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3adc0aa1f0 tx=0x7f3acc00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.298+0000 7f3ad3fff700 1 -- 172.21.15.7:0/291671384 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3ac4050850 con 0x7f3adc070080 2024-09-13T18:54:44.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.474+0000 7f3ae2207700 1 -- 172.21.15.7:0/291671384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3adc0ae540 con 0x7f3adc070080 2024-09-13T18:54:44.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.478+0000 7f3ad3fff700 1 -- 172.21.15.7:0/291671384 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16592 (secure 0 0 0) 0x7f3ac4053e70 con 0x7f3adc070080 2024-09-13T18:54:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:44.485 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":56,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:54:17.402890+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:54:07.330160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T18:54:07.981407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/3168373873":"2024-09-14T18:54:17.388088+0000","172.21.15.7:6834/3168373873":"2024-09-14T18:54:17.388088+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:54:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.482+0000 7f3ae2207700 1 -- 172.21.15.7:0/291671384 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3abc066760 msgr2=0x7f3abc068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.482+0000 7f3ae2207700 1 --2- 172.21.15.7:0/291671384 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3abc066760 0x7f3abc068c20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3adc0aa1f0 tx=0x7f3acc00b040 comp rx=0 tx=0).stop 2024-09-13T18:54:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.482+0000 7f3ae2207700 1 -- 172.21.15.7:0/291671384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3adc070080 msgr2=0x7f3adc1d5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.482+0000 7f3ae2207700 1 --2- 172.21.15.7:0/291671384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3adc070080 0x7f3adc1d5820 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400c500 tx=0x7f3ac400c810 comp rx=0 tx=0).stop 2024-09-13T18:54:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.482+0000 7f3ae2207700 1 -- 172.21.15.7:0/291671384 shutdown_connections 2024-09-13T18:54:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.482+0000 7f3ae2207700 1 --2- 172.21.15.7:0/291671384 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3abc066760 0x7f3abc068c20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.482+0000 7f3ae2207700 1 --2- 172.21.15.7:0/291671384 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3adc0cc980 0x7f3adc1ebab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.482+0000 7f3ae2207700 1 --2- 172.21.15.7:0/291671384 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f3adc0b2d00 0x7f3adc1d6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.482+0000 7f3ae2207700 1 --2- 172.21.15.7:0/291671384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3adc070080 0x7f3adc1d5820 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.482+0000 7f3ae2207700 1 -- 172.21.15.7:0/291671384 >> 172.21.15.7:0/291671384 conn(0x7f3adc0b0830 msgr2=0x7f3adc147fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.482+0000 7f3ae2207700 1 -- 172.21.15.7:0/291671384 shutdown_connections 2024-09-13T18:54:44.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.482+0000 7f3ae2207700 1 -- 172.21.15.7:0/291671384 wait complete. 2024-09-13T18:54:44.500 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:54:07.330160+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:54:07.981407+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:54:44.500 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:54:44.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.610+0000 7f5126995700 1 Processor -- start 2024-09-13T18:54:44.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.610+0000 7f5126995700 1 -- start start 2024-09-13T18:54:44.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5126995700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f512015cd70 0x7f51200ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:44.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5126995700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f51200ceba0 0x7f512006c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:44.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5126995700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f512006ceb0 con 0x7f512006ca60 2024-09-13T18:54:44.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5126995700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512006de30 con 0x7f51200ceba0 2024-09-13T18:54:44.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5126995700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512006d240 con 0x7f512015cd70 2024-09-13T18:54:44.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5125192700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f51200ceba0 0x7f512006c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:44.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5126194700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f512006ca60 0x7f512006a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51448/0 (socket says 172.21.15.7:51448) 2024-09-13T18:54:44.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5126194700 1 -- 172.21.15.7:0/1976970115 learned_addr learned my addr 172.21.15.7:0/1976970115 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:44.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5125993700 1 --2- 172.21.15.7:0/1976970115 >> v2:172.21.15.171:3300/0 conn(0x7f512015cd70 0x7f51200ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:44.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5124991700 1 -- 172.21.15.7:0/1976970115 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 902650855 0 0) 0x7f5118001490 con 0x7f512006ca60 2024-09-13T18:54:44.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5124991700 1 -- 172.21.15.7:0/1976970115 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f510c0035d0 con 0x7f512006ca60 2024-09-13T18:54:44.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5125192700 1 -- 172.21.15.7:0/1976970115 >> v2:172.21.15.171:3300/0 conn(0x7f512015cd70 msgr2=0x7f51200ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:44.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5125192700 1 --2- 172.21.15.7:0/1976970115 >> v2:172.21.15.171:3300/0 conn(0x7f512015cd70 0x7f51200ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5125192700 1 -- 172.21.15.7:0/1976970115 >> v1:172.21.15.7:6789/0 conn(0x7f512006ca60 legacy=0x7f512006a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:44.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5125192700 1 -- 172.21.15.7:0/1976970115 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512006d500 con 0x7f51200ceba0 2024-09-13T18:54:44.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5125192700 1 --2- 172.21.15.7:0/1976970115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51200ceba0 0x7f512006c520 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5120067ab0 tx=0x7f511c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4890b4370611251 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:44.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5124991700 1 -- 172.21.15.7:0/1976970115 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c011830 con 0x7f51200ceba0 2024-09-13T18:54:44.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5124991700 1 -- 172.21.15.7:0/1976970115 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511c011990 con 0x7f51200ceba0 2024-09-13T18:54:44.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5124991700 1 -- 172.21.15.7:0/1976970115 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c011d20 con 0x7f51200ceba0 2024-09-13T18:54:44.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5126995700 1 -- 172.21.15.7:0/1976970115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51200ceba0 msgr2=0x7f512006c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:44.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5126995700 1 --2- 172.21.15.7:0/1976970115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51200ceba0 0x7f512006c520 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5120067ab0 tx=0x7f511c009b30 comp rx=0 tx=0).stop 2024-09-13T18:54:44.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5126995700 1 -- 172.21.15.7:0/1976970115 shutdown_connections 2024-09-13T18:54:44.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5126995700 1 --2- 172.21.15.7:0/1976970115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51200ceba0 0x7f512006c520 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5126995700 1 --2- 172.21.15.7:0/1976970115 >> v2:172.21.15.171:3300/0 conn(0x7f512015cd70 0x7f51200ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5126995700 1 -- 172.21.15.7:0/1976970115 >> 172.21.15.7:0/1976970115 conn(0x7f51200cf120 msgr2=0x7f51200cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:44.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5126995700 1 -- 172.21.15.7:0/1976970115 shutdown_connections 2024-09-13T18:54:44.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.614+0000 7f5126995700 1 -- 172.21.15.7:0/1976970115 wait complete. 2024-09-13T18:54:44.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.618+0000 7f5126995700 1 Processor -- start 2024-09-13T18:54:44.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.618+0000 7f5126995700 1 -- start start 2024-09-13T18:54:44.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5126995700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f512006ca60 0x7f5120153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:44.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5126995700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51200ceba0 0x7f51200719b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:44.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5126995700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f512015cd70 0x7f5120071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:44.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5126995700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512009a430 con 0x7f51200ceba0 2024-09-13T18:54:44.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5126995700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51201f6de0 con 0x7f512015cd70 2024-09-13T18:54:44.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5126995700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5120099a90 con 0x7f512006ca60 2024-09-13T18:54:44.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5125192700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51200ceba0 0x7f51200719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:44.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5125192700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51200ceba0 0x7f51200719b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40240/0 (socket says 172.21.15.7:40240) 2024-09-13T18:54:44.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5125192700 1 -- 172.21.15.7:0/3327384205 learned_addr learned my addr 172.21.15.7:0/3327384205 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:44.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5126194700 1 --2- 172.21.15.7:0/3327384205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f512015cd70 0x7f5120071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:44.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5125993700 1 --2- 172.21.15.7:0/3327384205 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f512006ca60 0x7f5120153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:44.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5125192700 1 -- 172.21.15.7:0/3327384205 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f512006ca60 msgr2=0x7f5120153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:44.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5125192700 1 --2- 172.21.15.7:0/3327384205 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f512006ca60 0x7f5120153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5125192700 1 -- 172.21.15.7:0/3327384205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f512015cd70 msgr2=0x7f5120071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:44.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5125192700 1 --2- 172.21.15.7:0/3327384205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f512015cd70 0x7f5120071ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5125192700 1 -- 172.21.15.7:0/3327384205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c0094b0 con 0x7f51200ceba0 2024-09-13T18:54:44.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5125993700 1 --2- 172.21.15.7:0/3327384205 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f512006ca60 0x7f5120153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:44.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5125192700 1 --2- 172.21.15.7:0/3327384205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51200ceba0 0x7f51200719b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f511c014040 tx=0x7f511c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:44.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f51127fc700 1 -- 172.21.15.7:0/3327384205 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c004590 con 0x7f51200ceba0 2024-09-13T18:54:44.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f51127fc700 1 -- 172.21.15.7:0/3327384205 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511c01e070 con 0x7f51200ceba0 2024-09-13T18:54:44.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5126995700 1 -- 172.21.15.7:0/3327384205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5120099d70 con 0x7f51200ceba0 2024-09-13T18:54:44.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f51127fc700 1 -- 172.21.15.7:0/3327384205 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c0038e0 con 0x7f51200ceba0 2024-09-13T18:54:44.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f5126995700 1 -- 172.21.15.7:0/3327384205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5120153bb0 con 0x7f51200ceba0 2024-09-13T18:54:44.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.622+0000 7f51127fc700 1 -- 172.21.15.7:0/3327384205 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f511c003a40 con 0x7f51200ceba0 2024-09-13T18:54:44.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.626+0000 7f5126995700 1 -- 172.21.15.7:0/3327384205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5120067830 con 0x7f51200ceba0 2024-09-13T18:54:44.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.626+0000 7f51127fc700 1 --2- 172.21.15.7:0/3327384205 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5104066760 0x7f5104068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:44.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.626+0000 7f51127fc700 1 -- 172.21.15.7:0/3327384205 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f511c084ed0 con 0x7f51200ceba0 2024-09-13T18:54:44.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.626+0000 7f5125993700 1 --2- 172.21.15.7:0/3327384205 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5104066760 0x7f5104068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:44.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.626+0000 7f5125993700 1 --2- 172.21.15.7:0/3327384205 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5104066760 0x7f5104068c20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5114009ba0 tx=0x7f5114008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:44.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.634+0000 7f51127fc700 1 -- 172.21.15.7:0/3327384205 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f511c052270 con 0x7f51200ceba0 2024-09-13T18:54:44.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.802+0000 7f5126995700 1 -- 172.21.15.7:0/3327384205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f512006b600 con 0x7f51200ceba0 2024-09-13T18:54:44.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.802+0000 7f51127fc700 1 -- 172.21.15.7:0/3327384205 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f512006b600 con 0x7f51200ceba0 2024-09-13T18:54:44.810 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:54:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.806+0000 7f5126995700 1 -- 172.21.15.7:0/3327384205 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5104066760 msgr2=0x7f5104068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.806+0000 7f5126995700 1 --2- 172.21.15.7:0/3327384205 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5104066760 0x7f5104068c20 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5114009ba0 tx=0x7f5114008040 comp rx=0 tx=0).stop 2024-09-13T18:54:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.806+0000 7f5126995700 1 -- 172.21.15.7:0/3327384205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51200ceba0 msgr2=0x7f51200719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.806+0000 7f5126995700 1 --2- 172.21.15.7:0/3327384205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51200ceba0 0x7f51200719b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f511c014040 tx=0x7f511c004770 comp rx=0 tx=0).stop 2024-09-13T18:54:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.806+0000 7f5126995700 1 -- 172.21.15.7:0/3327384205 shutdown_connections 2024-09-13T18:54:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.806+0000 7f5126995700 1 --2- 172.21.15.7:0/3327384205 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5104066760 0x7f5104068c20 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.806+0000 7f5126995700 1 --2- 172.21.15.7:0/3327384205 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f512015cd70 0x7f5120071ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.806+0000 7f5126995700 1 --2- 172.21.15.7:0/3327384205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51200ceba0 0x7f51200719b0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.806+0000 7f5126995700 1 --2- 172.21.15.7:0/3327384205 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f512006ca60 0x7f5120153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.806+0000 7f5126995700 1 -- 172.21.15.7:0/3327384205 >> 172.21.15.7:0/3327384205 conn(0x7f51200cf120 msgr2=0x7f51200aebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:44.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.806+0000 7f5126995700 1 -- 172.21.15.7:0/3327384205 shutdown_connections 2024-09-13T18:54:44.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.806+0000 7f5126995700 1 -- 172.21.15.7:0/3327384205 wait complete. 2024-09-13T18:54:44.826 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:54:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.942+0000 7f4935769700 1 Processor -- start 2024-09-13T18:54:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.942+0000 7f4935769700 1 -- start start 2024-09-13T18:54:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f4935769700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f49301606a0 0x7f49300d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f4935769700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f4930160b10 0x7f4930160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f4935769700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49300d1ca0 con 0x7f49300d17a0 2024-09-13T18:54:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f4935769700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49301614e0 con 0x7f49301606a0 2024-09-13T18:54:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f4935769700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49300cdd00 con 0x7f4930160b10 2024-09-13T18:54:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f492ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f49301606a0 0x7f49300d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f492ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f49301606a0 0x7f49300d3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40244/0 (socket says 172.21.15.7:40244) 2024-09-13T18:54:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f492ffff700 1 -- 172.21.15.7:0/3513679590 learned_addr learned my addr 172.21.15.7:0/3513679590 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f4934f68700 1 --2- 172.21.15.7:0/3513679590 >> v2:172.21.15.171:3300/0 conn(0x7f4930160b10 0x7f4930160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f492ffff700 1 -- 172.21.15.7:0/3513679590 >> v2:172.21.15.171:3300/0 conn(0x7f4930160b10 msgr2=0x7f4930160fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f492ffff700 1 --2- 172.21.15.7:0/3513679590 >> v2:172.21.15.171:3300/0 conn(0x7f4930160b10 0x7f4930160fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f492ffff700 1 -- 172.21.15.7:0/3513679590 >> v1:172.21.15.171:6789/0 conn(0x7f49300d17a0 legacy=0x7f49300ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f492ffff700 1 -- 172.21.15.7:0/3513679590 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49300cdfe0 con 0x7f49301606a0 2024-09-13T18:54:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f492ffff700 1 --2- 172.21.15.7:0/3513679590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49301606a0 0x7f49300d3800 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4920009910 tx=0x7f4920009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8da04aeff71498 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f492effd700 1 -- 172.21.15.7:0/3513679590 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920011830 con 0x7f49301606a0 2024-09-13T18:54:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f492effd700 1 -- 172.21.15.7:0/3513679590 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4920011990 con 0x7f49301606a0 2024-09-13T18:54:44.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f492effd700 1 -- 172.21.15.7:0/3513679590 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920011d40 con 0x7f49301606a0 2024-09-13T18:54:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f4935769700 1 -- 172.21.15.7:0/3513679590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49301606a0 msgr2=0x7f49300d3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f4935769700 1 --2- 172.21.15.7:0/3513679590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49301606a0 0x7f49300d3800 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4920009910 tx=0x7f4920009c20 comp rx=0 tx=0).stop 2024-09-13T18:54:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f4935769700 1 -- 172.21.15.7:0/3513679590 shutdown_connections 2024-09-13T18:54:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f4935769700 1 --2- 172.21.15.7:0/3513679590 >> v2:172.21.15.171:3300/0 conn(0x7f4930160b10 0x7f4930160fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f4935769700 1 --2- 172.21.15.7:0/3513679590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49301606a0 0x7f49300d3800 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f4935769700 1 -- 172.21.15.7:0/3513679590 >> 172.21.15.7:0/3513679590 conn(0x7f49300d2d10 msgr2=0x7f49300d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f4935769700 1 -- 172.21.15.7:0/3513679590 shutdown_connections 2024-09-13T18:54:44.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.946+0000 7f4935769700 1 -- 172.21.15.7:0/3513679590 wait complete. 2024-09-13T18:54:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.950+0000 7f4935769700 1 Processor -- start 2024-09-13T18:54:44.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.950+0000 7f4935769700 1 -- start start 2024-09-13T18:54:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.950+0000 7f4935769700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49300d17a0 0x7f49301525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.950+0000 7f4935769700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f49301606a0 0x7f49301e7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.950+0000 7f4935769700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4930160b10 0x7f49301e8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:44.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.950+0000 7f4935769700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4930152b00 con 0x7f4930160b10 2024-09-13T18:54:44.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.950+0000 7f4935769700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49301580d0 con 0x7f49300d17a0 2024-09-13T18:54:44.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.950+0000 7f4935769700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f493015f550 con 0x7f49301606a0 2024-09-13T18:54:44.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f4934f68700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4930160b10 0x7f49301e8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:44.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f4934f68700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4930160b10 0x7f49301e8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40252/0 (socket says 172.21.15.7:40252) 2024-09-13T18:54:44.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f4934f68700 1 -- 172.21.15.7:0/3012543741 learned_addr learned my addr 172.21.15.7:0/3012543741 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:44.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f492ffff700 1 --2- 172.21.15.7:0/3012543741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49300d17a0 0x7f49301525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:44.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f492f7fe700 1 --2- 172.21.15.7:0/3012543741 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f49301606a0 0x7f49301e7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:44.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f4934f68700 1 -- 172.21.15.7:0/3012543741 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f49301606a0 msgr2=0x7f49301e7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:44.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f4934f68700 1 --2- 172.21.15.7:0/3012543741 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f49301606a0 0x7f49301e7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f4934f68700 1 -- 172.21.15.7:0/3012543741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49300d17a0 msgr2=0x7f49301525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:44.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f4934f68700 1 --2- 172.21.15.7:0/3012543741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49300d17a0 0x7f49301525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:44.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f4934f68700 1 -- 172.21.15.7:0/3012543741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4920009570 con 0x7f4930160b10 2024-09-13T18:54:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f4934f68700 1 --2- 172.21.15.7:0/3012543741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4930160b10 0x7f49301e8420 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f492800efc0 tx=0x7f492800c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f492cff9700 1 -- 172.21.15.7:0/3012543741 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492800cad0 con 0x7f4930160b10 2024-09-13T18:54:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f492cff9700 1 -- 172.21.15.7:0/3012543741 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492800f040 con 0x7f4930160b10 2024-09-13T18:54:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f4935769700 1 -- 172.21.15.7:0/3012543741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493015f830 con 0x7f4930160b10 2024-09-13T18:54:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f492cff9700 1 -- 172.21.15.7:0/3012543741 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492800ce70 con 0x7f4930160b10 2024-09-13T18:54:44.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f4935769700 1 -- 172.21.15.7:0/3012543741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4930157d60 con 0x7f4930160b10 2024-09-13T18:54:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f4935769700 1 -- 172.21.15.7:0/3012543741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4930158210 con 0x7f4930160b10 2024-09-13T18:54:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f492cff9700 1 -- 172.21.15.7:0/3012543741 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4928014480 con 0x7f4930160b10 2024-09-13T18:54:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.954+0000 7f492cff9700 1 --2- 172.21.15.7:0/3012543741 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4910066760 0x7f4910068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.958+0000 7f492cff9700 1 -- 172.21.15.7:0/3012543741 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4928084c10 con 0x7f4930160b10 2024-09-13T18:54:44.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.958+0000 7f492ffff700 1 --2- 172.21.15.7:0/3012543741 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4910066760 0x7f4910068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:44.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.958+0000 7f492ffff700 1 --2- 172.21.15.7:0/3012543741 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4910066760 0x7f4910068c20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4920014040 tx=0x7f4920008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:44.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:44.962+0000 7f492cff9700 1 -- 172.21.15.7:0/3012543741 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4928051ea0 con 0x7f4930160b10 2024-09-13T18:54:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.138+0000 7f4935769700 1 -- 172.21.15.7:0/3012543741 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49300cf500 con 0x7f4930160b10 2024-09-13T18:54:45.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.138+0000 7f492cff9700 1 -- 172.21.15.7:0/3012543741 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7f49280554c0 con 0x7f4930160b10 2024-09-13T18:54:45.143 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:54:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.138+0000 7f4935769700 1 -- 172.21.15.7:0/3012543741 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4910066760 msgr2=0x7f4910068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.138+0000 7f4935769700 1 --2- 172.21.15.7:0/3012543741 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4910066760 0x7f4910068c20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4920014040 tx=0x7f4920008030 comp rx=0 tx=0).stop 2024-09-13T18:54:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.138+0000 7f4935769700 1 -- 172.21.15.7:0/3012543741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4930160b10 msgr2=0x7f49301e8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.138+0000 7f4935769700 1 --2- 172.21.15.7:0/3012543741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4930160b10 0x7f49301e8420 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f492800efc0 tx=0x7f492800c0c0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.138+0000 7f4935769700 1 -- 172.21.15.7:0/3012543741 shutdown_connections 2024-09-13T18:54:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.138+0000 7f4935769700 1 --2- 172.21.15.7:0/3012543741 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4910066760 0x7f4910068c20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.138+0000 7f4935769700 1 --2- 172.21.15.7:0/3012543741 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4930160b10 0x7f49301e8420 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.138+0000 7f4935769700 1 --2- 172.21.15.7:0/3012543741 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f49301606a0 0x7f49301e7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.138+0000 7f4935769700 1 --2- 172.21.15.7:0/3012543741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f49300d17a0 0x7f49301525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.138+0000 7f4935769700 1 -- 172.21.15.7:0/3012543741 >> 172.21.15.7:0/3012543741 conn(0x7f49300d2d10 msgr2=0x7f49300b5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.138+0000 7f4935769700 1 -- 172.21.15.7:0/3012543741 shutdown_connections 2024-09-13T18:54:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.138+0000 7f4935769700 1 -- 172.21.15.7:0/3012543741 wait complete. 2024-09-13T18:54:45.158 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:54:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b4d209700 1 Processor -- start 2024-09-13T18:54:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b4d209700 1 -- start start 2024-09-13T18:54:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b4d209700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5b481606a0 0x7f5b480d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b4d209700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f5b480d17a0 0x7f5b480ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b4d209700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b480ced20 con 0x7f5b48160b10 2024-09-13T18:54:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b4d209700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b480d1ca0 con 0x7f5b481606a0 2024-09-13T18:54:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b4d209700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b480cdd00 con 0x7f5b480d17a0 2024-09-13T18:54:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b47fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5b481606a0 0x7f5b480d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b47fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5b481606a0 0x7f5b480d3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40260/0 (socket says 172.21.15.7:40260) 2024-09-13T18:54:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b47fff700 1 -- 172.21.15.7:0/3355622029 learned_addr learned my addr 172.21.15.7:0/3355622029 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b4ca08700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5b48160b10 0x7f5b480b5b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51458/0 (socket says 172.21.15.7:51458) 2024-09-13T18:54:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b477fe700 1 --2- 172.21.15.7:0/3355622029 >> v2:172.21.15.171:3300/0 conn(0x7f5b480d17a0 0x7f5b480ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b47fff700 1 -- 172.21.15.7:0/3355622029 >> v2:172.21.15.171:3300/0 conn(0x7f5b480d17a0 msgr2=0x7f5b480ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b47fff700 1 --2- 172.21.15.7:0/3355622029 >> v2:172.21.15.171:3300/0 conn(0x7f5b480d17a0 0x7f5b480ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b47fff700 1 -- 172.21.15.7:0/3355622029 >> v1:172.21.15.7:6789/0 conn(0x7f5b48160b10 legacy=0x7f5b480b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b47fff700 1 -- 172.21.15.7:0/3355622029 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b480cdfe0 con 0x7f5b481606a0 2024-09-13T18:54:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b477fe700 1 --2- 172.21.15.7:0/3355622029 >> v2:172.21.15.171:3300/0 conn(0x7f5b480d17a0 0x7f5b480ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b47fff700 1 --2- 172.21.15.7:0/3355622029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b481606a0 0x7f5b480d3800 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009910 tx=0x7f5b38009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d8acb4dd0550fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b46ffd700 1 -- 172.21.15.7:0/3355622029 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3800ba20 con 0x7f5b481606a0 2024-09-13T18:54:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b46ffd700 1 -- 172.21.15.7:0/3355622029 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3800bb80 con 0x7f5b481606a0 2024-09-13T18:54:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.270+0000 7f5b46ffd700 1 -- 172.21.15.7:0/3355622029 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b38009e60 con 0x7f5b481606a0 2024-09-13T18:54:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4d209700 1 -- 172.21.15.7:0/3355622029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b481606a0 msgr2=0x7f5b480d3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4d209700 1 --2- 172.21.15.7:0/3355622029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b481606a0 0x7f5b480d3800 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009910 tx=0x7f5b38009c20 comp rx=0 tx=0).stop 2024-09-13T18:54:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4d209700 1 -- 172.21.15.7:0/3355622029 shutdown_connections 2024-09-13T18:54:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4d209700 1 --2- 172.21.15.7:0/3355622029 >> v2:172.21.15.171:3300/0 conn(0x7f5b480d17a0 0x7f5b480ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4d209700 1 --2- 172.21.15.7:0/3355622029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b481606a0 0x7f5b480d3800 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4d209700 1 -- 172.21.15.7:0/3355622029 >> 172.21.15.7:0/3355622029 conn(0x7f5b480d2d10 msgr2=0x7f5b480d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4d209700 1 -- 172.21.15.7:0/3355622029 shutdown_connections 2024-09-13T18:54:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4d209700 1 -- 172.21.15.7:0/3355622029 wait complete. 2024-09-13T18:54:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4d209700 1 Processor -- start 2024-09-13T18:54:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4d209700 1 -- start start 2024-09-13T18:54:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4d209700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b480d17a0 0x7f5b4814c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4d209700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5b481606a0 0x7f5b480c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4d209700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b48160b10 0x7f5b4815cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4d209700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b480cb680 con 0x7f5b48160b10 2024-09-13T18:54:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4d209700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b480c9200 con 0x7f5b480d17a0 2024-09-13T18:54:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4d209700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b480cace0 con 0x7f5b481606a0 2024-09-13T18:54:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4ca08700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b48160b10 0x7f5b4815cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4ca08700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b48160b10 0x7f5b4815cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40262/0 (socket says 172.21.15.7:40262) 2024-09-13T18:54:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4ca08700 1 -- 172.21.15.7:0/639114707 learned_addr learned my addr 172.21.15.7:0/639114707 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b47fff700 1 --2- 172.21.15.7:0/639114707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b480d17a0 0x7f5b4814c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b477fe700 1 --2- 172.21.15.7:0/639114707 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5b481606a0 0x7f5b480c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4ca08700 1 -- 172.21.15.7:0/639114707 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5b481606a0 msgr2=0x7f5b480c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4ca08700 1 --2- 172.21.15.7:0/639114707 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5b481606a0 0x7f5b480c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4ca08700 1 -- 172.21.15.7:0/639114707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b480d17a0 msgr2=0x7f5b4814c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4ca08700 1 --2- 172.21.15.7:0/639114707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b480d17a0 0x7f5b4814c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.274+0000 7f5b4ca08700 1 -- 172.21.15.7:0/639114707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b38009570 con 0x7f5b48160b10 2024-09-13T18:54:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.278+0000 7f5b4ca08700 1 --2- 172.21.15.7:0/639114707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b48160b10 0x7f5b4815cbb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000cbc0 tx=0x7f5b4000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.278+0000 7f5b44ff9700 1 -- 172.21.15.7:0/639114707 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4000aaa0 con 0x7f5b48160b10 2024-09-13T18:54:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.278+0000 7f5b44ff9700 1 -- 172.21.15.7:0/639114707 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4000ac00 con 0x7f5b48160b10 2024-09-13T18:54:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.278+0000 7f5b4d209700 1 -- 172.21.15.7:0/639114707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b480cafc0 con 0x7f5b48160b10 2024-09-13T18:54:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.278+0000 7f5b44ff9700 1 -- 172.21.15.7:0/639114707 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40007070 con 0x7f5b48160b10 2024-09-13T18:54:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.278+0000 7f5b4d209700 1 -- 172.21.15.7:0/639114707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4814c550 con 0x7f5b48160b10 2024-09-13T18:54:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.278+0000 7f5b4d209700 1 -- 172.21.15.7:0/639114707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b480cf290 con 0x7f5b48160b10 2024-09-13T18:54:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.278+0000 7f5b44ff9700 1 -- 172.21.15.7:0/639114707 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f5b400071d0 con 0x7f5b48160b10 2024-09-13T18:54:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.278+0000 7f5b44ff9700 1 --2- 172.21.15.7:0/639114707 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5b34066730 0x7f5b34068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.278+0000 7f5b44ff9700 1 -- 172.21.15.7:0/639114707 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5b40083ec0 con 0x7f5b48160b10 2024-09-13T18:54:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.282+0000 7f5b47fff700 1 --2- 172.21.15.7:0/639114707 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5b34066730 0x7f5b34068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.282+0000 7f5b47fff700 1 --2- 172.21.15.7:0/639114707 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5b34066730 0x7f5b34068bf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5b38000c00 tx=0x7f5b38008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:45.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.290+0000 7f5b44ff9700 1 -- 172.21.15.7:0/639114707 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5b400511f0 con 0x7f5b48160b10 2024-09-13T18:54:45.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.462+0000 7f5b4d209700 1 -- 172.21.15.7:0/639114707 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5b480cb7c0 con 0x7f5b48160b10 2024-09-13T18:54:45.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.462+0000 7f5b44ff9700 1 -- 172.21.15.7:0/639114707 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5b480cb7c0 con 0x7f5b48160b10 2024-09-13T18:54:45.469 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:54:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.466+0000 7f5b4d209700 1 -- 172.21.15.7:0/639114707 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5b34066730 msgr2=0x7f5b34068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.466+0000 7f5b4d209700 1 --2- 172.21.15.7:0/639114707 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5b34066730 0x7f5b34068bf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5b38000c00 tx=0x7f5b38008030 comp rx=0 tx=0).stop 2024-09-13T18:54:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.466+0000 7f5b4d209700 1 -- 172.21.15.7:0/639114707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b48160b10 msgr2=0x7f5b4815cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.466+0000 7f5b4d209700 1 --2- 172.21.15.7:0/639114707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b48160b10 0x7f5b4815cbb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000cbc0 tx=0x7f5b4000ced0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.466+0000 7f5b4d209700 1 -- 172.21.15.7:0/639114707 shutdown_connections 2024-09-13T18:54:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.466+0000 7f5b4d209700 1 --2- 172.21.15.7:0/639114707 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5b34066730 0x7f5b34068bf0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.466+0000 7f5b4d209700 1 --2- 172.21.15.7:0/639114707 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b48160b10 0x7f5b4815cbb0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.466+0000 7f5b4d209700 1 --2- 172.21.15.7:0/639114707 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f5b481606a0 0x7f5b480c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.466+0000 7f5b4d209700 1 --2- 172.21.15.7:0/639114707 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b480d17a0 0x7f5b4814c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.466+0000 7f5b4d209700 1 -- 172.21.15.7:0/639114707 >> 172.21.15.7:0/639114707 conn(0x7f5b480d2d10 msgr2=0x7f5b4806c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.466+0000 7f5b4d209700 1 -- 172.21.15.7:0/639114707 shutdown_connections 2024-09-13T18:54:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.466+0000 7f5b4d209700 1 -- 172.21.15.7:0/639114707 wait complete. 2024-09-13T18:54:45.484 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T18:54:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.594+0000 7fad97fff700 1 Processor -- start 2024-09-13T18:54:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.594+0000 7fad97fff700 1 -- start start 2024-09-13T18:54:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad97fff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fad980d17a0 0x7fad980d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad97fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad980d1ca0 con 0x7fad981606a0 2024-09-13T18:54:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad97fff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad980cdd00 con 0x7fad98160b10 2024-09-13T18:54:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad97fff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad980cdfe0 con 0x7fad980d17a0 2024-09-13T18:54:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad96ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fad981606a0 0x7fad980ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51464/0 (socket says 172.21.15.7:51464) 2024-09-13T18:54:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad96ffd700 1 -- 172.21.15.7:0/2216635489 learned_addr learned my addr 172.21.15.7:0/2216635489 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad8ffff700 1 --2- 172.21.15.7:0/2216635489 >> v2:172.21.15.171:3300/0 conn(0x7fad980d17a0 0x7fad980d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad967fc700 1 -- 172.21.15.7:0/2216635489 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1813422454 0 0) 0x7fad980cdd00 con 0x7fad98160b10 2024-09-13T18:54:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad967fc700 1 -- 172.21.15.7:0/2216635489 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad840035d0 con 0x7fad98160b10 2024-09-13T18:54:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad967fc700 1 -- 172.21.15.7:0/2216635489 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915753643 0 0) 0x7fad980d1ca0 con 0x7fad981606a0 2024-09-13T18:54:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad8ffff700 1 -- 172.21.15.7:0/2216635489 >> v1:172.21.15.171:6789/0 conn(0x7fad98160b10 legacy=0x7fad980b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad8ffff700 1 -- 172.21.15.7:0/2216635489 >> v1:172.21.15.7:6789/0 conn(0x7fad981606a0 legacy=0x7fad980ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad8ffff700 1 -- 172.21.15.7:0/2216635489 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad980ce2c0 con 0x7fad980d17a0 2024-09-13T18:54:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad8ffff700 1 --2- 172.21.15.7:0/2216635489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad980d17a0 0x7fad980d3800 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fad80009880 tx=0x7fad80009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f21b81fa0f9b5f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad967fc700 1 -- 172.21.15.7:0/2216635489 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80011830 con 0x7fad980d17a0 2024-09-13T18:54:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad967fc700 1 -- 172.21.15.7:0/2216635489 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad80011990 con 0x7fad980d17a0 2024-09-13T18:54:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad967fc700 1 -- 172.21.15.7:0/2216635489 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad80011d20 con 0x7fad980d17a0 2024-09-13T18:54:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad97fff700 1 -- 172.21.15.7:0/2216635489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad980d17a0 msgr2=0x7fad980d3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad97fff700 1 --2- 172.21.15.7:0/2216635489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad980d17a0 0x7fad980d3800 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fad80009880 tx=0x7fad80009b90 comp rx=0 tx=0).stop 2024-09-13T18:54:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.598+0000 7fad97fff700 1 -- 172.21.15.7:0/2216635489 shutdown_connections 2024-09-13T18:54:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.602+0000 7fad97fff700 1 --2- 172.21.15.7:0/2216635489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad980d17a0 0x7fad980d3800 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.602+0000 7fad97fff700 1 -- 172.21.15.7:0/2216635489 >> 172.21.15.7:0/2216635489 conn(0x7fad980d2d10 msgr2=0x7fad980d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.602+0000 7fad97fff700 1 -- 172.21.15.7:0/2216635489 shutdown_connections 2024-09-13T18:54:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.602+0000 7fad97fff700 1 -- 172.21.15.7:0/2216635489 wait complete. 2024-09-13T18:54:45.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.602+0000 7fad97fff700 1 Processor -- start 2024-09-13T18:54:45.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.602+0000 7fad97fff700 1 -- start start 2024-09-13T18:54:45.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad97fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad980d17a0 0x7fad9814f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:45.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad97fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fad981606a0 0x7fad9809c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:45.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad97fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad98160b10 0x7fad9809ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:45.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad97fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9809f950 con 0x7fad980d17a0 2024-09-13T18:54:45.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad97fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9809fc30 con 0x7fad98160b10 2024-09-13T18:54:45.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad97fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9809ff10 con 0x7fad981606a0 2024-09-13T18:54:45.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad96ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad980d17a0 0x7fad9814f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad96ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad980d17a0 0x7fad9814f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40266/0 (socket says 172.21.15.7:40266) 2024-09-13T18:54:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad96ffd700 1 -- 172.21.15.7:0/2837165413 learned_addr learned my addr 172.21.15.7:0/2837165413 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad8ffff700 1 --2- 172.21.15.7:0/2837165413 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fad981606a0 0x7fad9809c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad977fe700 1 --2- 172.21.15.7:0/2837165413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad98160b10 0x7fad9809ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad96ffd700 1 -- 172.21.15.7:0/2837165413 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fad981606a0 msgr2=0x7fad9809c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad96ffd700 1 --2- 172.21.15.7:0/2837165413 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fad981606a0 0x7fad9809c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad96ffd700 1 -- 172.21.15.7:0/2837165413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad98160b10 msgr2=0x7fad9809ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad96ffd700 1 --2- 172.21.15.7:0/2837165413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad98160b10 0x7fad9809ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad96ffd700 1 -- 172.21.15.7:0/2837165413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad800094b0 con 0x7fad980d17a0 2024-09-13T18:54:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad8ffff700 1 --2- 172.21.15.7:0/2837165413 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fad981606a0 0x7fad9809c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad977fe700 1 --2- 172.21.15.7:0/2837165413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad98160b10 0x7fad9809ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:45.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad96ffd700 1 --2- 172.21.15.7:0/2837165413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad980d17a0 0x7fad9814f0e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fad8800c750 tx=0x7fad8800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad8f7fe700 1 -- 172.21.15.7:0/2837165413 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad88004ca0 con 0x7fad980d17a0 2024-09-13T18:54:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad8f7fe700 1 -- 172.21.15.7:0/2837165413 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad88004e20 con 0x7fad980d17a0 2024-09-13T18:54:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad97fff700 1 -- 172.21.15.7:0/2837165413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad980a01f0 con 0x7fad980d17a0 2024-09-13T18:54:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad8f7fe700 1 -- 172.21.15.7:0/2837165413 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad88007310 con 0x7fad980d17a0 2024-09-13T18:54:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad97fff700 1 -- 172.21.15.7:0/2837165413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9809ec20 con 0x7fad980d17a0 2024-09-13T18:54:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.606+0000 7fad97fff700 1 -- 172.21.15.7:0/2837165413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad980a03e0 con 0x7fad980d17a0 2024-09-13T18:54:45.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.610+0000 7fad8f7fe700 1 -- 172.21.15.7:0/2837165413 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fad88007470 con 0x7fad980d17a0 2024-09-13T18:54:45.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.610+0000 7fad8f7fe700 1 --2- 172.21.15.7:0/2837165413 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fad78066730 0x7fad78068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:45.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.610+0000 7fad8f7fe700 1 -- 172.21.15.7:0/2837165413 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fad88083f30 con 0x7fad980d17a0 2024-09-13T18:54:45.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.610+0000 7fad8ffff700 1 --2- 172.21.15.7:0/2837165413 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fad78066730 0x7fad78068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:45.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.610+0000 7fad8ffff700 1 --2- 172.21.15.7:0/2837165413 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fad78066730 0x7fad78068bf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fad980d1bd0 tx=0x7fad80008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:45.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.618+0000 7fad8f7fe700 1 -- 172.21.15.7:0/2837165413 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fad980a03e0 con 0x7fad980d17a0 2024-09-13T18:54:45.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.798+0000 7fad97fff700 1 -- 172.21.15.7:0/2837165413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fad980a03e0 con 0x7fad980d17a0 2024-09-13T18:54:45.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.802+0000 7fad8f7fe700 1 -- 172.21.15.7:0/2837165413 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v56) v1 ==== 80+0+120 (secure 0 0 0) 0x7fad980a03e0 con 0x7fad980d17a0 2024-09-13T18:54:45.806 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6835/3168373873 2024-09-14T18:54:17.388088+0000 2024-09-13T18:54:45.807 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6834/3168373873 2024-09-14T18:54:17.388088+0000 2024-09-13T18:54:45.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.802+0000 7fad97fff700 1 -- 172.21.15.7:0/2837165413 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fad78066730 msgr2=0x7fad78068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:45.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.802+0000 7fad97fff700 1 --2- 172.21.15.7:0/2837165413 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fad78066730 0x7fad78068bf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fad980d1bd0 tx=0x7fad80008030 comp rx=0 tx=0).stop 2024-09-13T18:54:45.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.802+0000 7fad97fff700 1 -- 172.21.15.7:0/2837165413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad980d17a0 msgr2=0x7fad9814f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:45.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.802+0000 7fad97fff700 1 --2- 172.21.15.7:0/2837165413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad980d17a0 0x7fad9814f0e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fad8800c750 tx=0x7fad8800cb10 comp rx=0 tx=0).stop 2024-09-13T18:54:45.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.802+0000 7fad97fff700 1 -- 172.21.15.7:0/2837165413 shutdown_connections 2024-09-13T18:54:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.802+0000 7fad97fff700 1 --2- 172.21.15.7:0/2837165413 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fad78066730 0x7fad78068bf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.802+0000 7fad97fff700 1 --2- 172.21.15.7:0/2837165413 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fad98160b10 0x7fad9809ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.802+0000 7fad97fff700 1 --2- 172.21.15.7:0/2837165413 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fad981606a0 0x7fad9809c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.802+0000 7fad97fff700 1 --2- 172.21.15.7:0/2837165413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad980d17a0 0x7fad9814f0e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.802+0000 7fad97fff700 1 -- 172.21.15.7:0/2837165413 >> 172.21.15.7:0/2837165413 conn(0x7fad980d2d10 msgr2=0x7fad9806b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.802+0000 7fad97fff700 1 -- 172.21.15.7:0/2837165413 shutdown_connections 2024-09-13T18:54:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:45.802+0000 7fad97fff700 1 -- 172.21.15.7:0/2837165413 wait complete. 2024-09-13T18:54:45.810 INFO:teuthology.orchestra.run.smithi007.stderr:listed 2 entries 2024-09-13T18:54:45.826 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi007.front.sepia.ceph.com... 2024-09-13T18:54:45.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:45.826 DEBUG:teuthology.orchestra.run.smithi007:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T18:54:45.838 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[37393]: fuse finished with error 0 and tester_r 0 2024-09-13T18:54:45.840 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-13T18:54:45.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:45.840 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T18:54:45.965 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:54:45.965 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:54:45.972 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346336k,nr_inodes=4086584,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18533 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T18:54:45.973 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T18:54:45.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:45.973 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-13T18:54:45.979 INFO:teuthology.orchestra.run.smithi171.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-09-13T18:54:45.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:54:45.980 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:54:45.980 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:54:45.988 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16349432k,nr_inodes=4087358,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276116k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276112k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276116k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T18:54:45.988 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-13T18:54:45.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:45.988 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-13T18:54:46.036 INFO:teuthology.orchestra.run.smithi007.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-13T18:54:46.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:54:46.037 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:54:46.037 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:54:46.044 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346336k,nr_inodes=4086584,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18533 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T18:54:46.044 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T18:54:46.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:46.045 DEBUG:teuthology.orchestra.run.smithi171:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-13T18:54:46.051 INFO:teuthology.orchestra.run.smithi171.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-13T18:54:46.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:54:46.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:46.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.166+0000 7f14e1ced700 1 Processor -- start 2024-09-13T18:54:46.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.166+0000 7f14e1ced700 1 -- start start 2024-09-13T18:54:46.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e1ced700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f14dc1599b0 0x7f14dc0ada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:46.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e1ced700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f14dc070e50 0x7f14dc0b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:46.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e1ced700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14dc0b1aa0 con 0x7f14dc0cabf0 2024-09-13T18:54:46.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e1ced700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14dc06c3c0 con 0x7f14dc070e50 2024-09-13T18:54:46.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e1ced700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14dc06be40 con 0x7f14dc1599b0 2024-09-13T18:54:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14dbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f14dc0cabf0 0x7f14dc070740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51474/0 (socket says 172.21.15.7:51474) 2024-09-13T18:54:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14dbfff700 1 -- 172.21.15.7:0/1402614057 learned_addr learned my addr 172.21.15.7:0/1402614057 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e14ec700 1 --2- 172.21.15.7:0/1402614057 >> v2:172.21.15.7:3300/0 conn(0x7f14dc070e50 0x7f14dc0b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e0ceb700 1 --2- 172.21.15.7:0/1402614057 >> v2:172.21.15.171:3300/0 conn(0x7f14dc1599b0 0x7f14dc0ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14db7fe700 1 -- 172.21.15.7:0/1402614057 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528870309 0 0) 0x7f14dc0b1aa0 con 0x7f14dc0cabf0 2024-09-13T18:54:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14db7fe700 1 -- 172.21.15.7:0/1402614057 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14c80035d0 con 0x7f14dc0cabf0 2024-09-13T18:54:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e14ec700 1 -- 172.21.15.7:0/1402614057 >> v2:172.21.15.171:3300/0 conn(0x7f14dc1599b0 msgr2=0x7f14dc0ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e14ec700 1 --2- 172.21.15.7:0/1402614057 >> v2:172.21.15.171:3300/0 conn(0x7f14dc1599b0 0x7f14dc0ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e14ec700 1 -- 172.21.15.7:0/1402614057 >> v1:172.21.15.7:6789/0 conn(0x7f14dc0cabf0 legacy=0x7f14dc070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14db7fe700 1 -- 172.21.15.7:0/1402614057 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 485942593 0 0) 0x7f14c80035d0 con 0x7f14dc0cabf0 2024-09-13T18:54:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e14ec700 1 -- 172.21.15.7:0/1402614057 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc0cb140 con 0x7f14dc070e50 2024-09-13T18:54:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e0ceb700 1 --2- 172.21.15.7:0/1402614057 >> v2:172.21.15.171:3300/0 conn(0x7f14dc1599b0 0x7f14dc0ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:46.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e14ec700 1 --2- 172.21.15.7:0/1402614057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc070e50 0x7f14dc0b1f30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f14dc06b300 tx=0x7f14d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b18bb9f3d55b61 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:46.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14db7fe700 1 -- 172.21.15.7:0/1402614057 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d400e070 con 0x7f14dc070e50 2024-09-13T18:54:46.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14db7fe700 1 -- 172.21.15.7:0/1402614057 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d4003a40 con 0x7f14dc070e50 2024-09-13T18:54:46.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14db7fe700 1 -- 172.21.15.7:0/1402614057 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc001670 con 0x7f14dc070e50 2024-09-13T18:54:46.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e1ced700 1 -- 172.21.15.7:0/1402614057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc070e50 msgr2=0x7f14dc0b1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:46.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e1ced700 1 --2- 172.21.15.7:0/1402614057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc070e50 0x7f14dc0b1f30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f14dc06b300 tx=0x7f14d400b330 comp rx=0 tx=0).stop 2024-09-13T18:54:46.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e1ced700 1 -- 172.21.15.7:0/1402614057 shutdown_connections 2024-09-13T18:54:46.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e1ced700 1 --2- 172.21.15.7:0/1402614057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc070e50 0x7f14dc0b1f30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e1ced700 1 --2- 172.21.15.7:0/1402614057 >> v2:172.21.15.171:3300/0 conn(0x7f14dc1599b0 0x7f14dc0ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e1ced700 1 -- 172.21.15.7:0/1402614057 >> 172.21.15.7:0/1402614057 conn(0x7f14dc0b0570 msgr2=0x7f14dc0b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:46.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e1ced700 1 -- 172.21.15.7:0/1402614057 shutdown_connections 2024-09-13T18:54:46.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.170+0000 7f14e1ced700 1 -- 172.21.15.7:0/1402614057 wait complete. 2024-09-13T18:54:46.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.174+0000 7f14e1ced700 1 Processor -- start 2024-09-13T18:54:46.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.174+0000 7f14e1ced700 1 -- start start 2024-09-13T18:54:46.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.174+0000 7f14e1ced700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc070e50 0x7f14dc14c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:46.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.174+0000 7f14e1ced700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14dc0cabf0 0x7f14dc14ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:46.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14e1ced700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14dc1599b0 0x7f14dc1499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:46.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14e1ced700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc0a9100 con 0x7f14dc070e50 2024-09-13T18:54:46.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14e1ced700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc0a84f0 con 0x7f14dc0cabf0 2024-09-13T18:54:46.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14e1ced700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc0a87d0 con 0x7f14dc1599b0 2024-09-13T18:54:46.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14e0ceb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc070e50 0x7f14dc14c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:46.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14e0ceb700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc070e50 0x7f14dc14c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40286/0 (socket says 172.21.15.7:40286) 2024-09-13T18:54:46.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14e0ceb700 1 -- 172.21.15.7:0/1929974647 learned_addr learned my addr 172.21.15.7:0/1929974647 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:46.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14e14ec700 1 --2- 172.21.15.7:0/1929974647 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14dc1599b0 0x7f14dc1499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:46.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14dbfff700 1 --2- 172.21.15.7:0/1929974647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14dc0cabf0 0x7f14dc14ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:46.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14e0ceb700 1 -- 172.21.15.7:0/1929974647 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14dc1599b0 msgr2=0x7f14dc1499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:46.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14e0ceb700 1 --2- 172.21.15.7:0/1929974647 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14dc1599b0 0x7f14dc1499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14e0ceb700 1 -- 172.21.15.7:0/1929974647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14dc0cabf0 msgr2=0x7f14dc14ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14e0ceb700 1 --2- 172.21.15.7:0/1929974647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14dc0cabf0 0x7f14dc14ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14e0ceb700 1 -- 172.21.15.7:0/1929974647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d4009a70 con 0x7f14dc070e50 2024-09-13T18:54:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14dbfff700 1 --2- 172.21.15.7:0/1929974647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14dc0cabf0 0x7f14dc14ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14e0ceb700 1 --2- 172.21.15.7:0/1929974647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc070e50 0x7f14dc14c4e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f14c400e810 tx=0x7f14c400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14d97fa700 1 -- 172.21.15.7:0/1929974647 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c4013610 con 0x7f14dc070e50 2024-09-13T18:54:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14d97fa700 1 -- 172.21.15.7:0/1929974647 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c4013770 con 0x7f14dc070e50 2024-09-13T18:54:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14e1ced700 1 -- 172.21.15.7:0/1929974647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14dc0a8ab0 con 0x7f14dc070e50 2024-09-13T18:54:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14d97fa700 1 -- 172.21.15.7:0/1929974647 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c4013b00 con 0x7f14dc070e50 2024-09-13T18:54:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.178+0000 7f14e1ced700 1 -- 172.21.15.7:0/1929974647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14dc0ad630 con 0x7f14dc070e50 2024-09-13T18:54:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.182+0000 7f14d97fa700 1 -- 172.21.15.7:0/1929974647 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f14c4013c60 con 0x7f14dc070e50 2024-09-13T18:54:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.182+0000 7f14e1ced700 1 -- 172.21.15.7:0/1929974647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14dc070700 con 0x7f14dc070e50 2024-09-13T18:54:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.182+0000 7f14d97fa700 1 --2- 172.21.15.7:0/1929974647 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f14bc066730 0x7f14bc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.182+0000 7f14d97fa700 1 -- 172.21.15.7:0/1929974647 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f14c4083e60 con 0x7f14dc070e50 2024-09-13T18:54:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.182+0000 7f14dbfff700 1 --2- 172.21.15.7:0/1929974647 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f14bc066730 0x7f14bc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.182+0000 7f14dbfff700 1 --2- 172.21.15.7:0/1929974647 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f14bc066730 0x7f14bc068bf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f14cc001eb0 tx=0x7f14cc001d00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.190+0000 7f14d97fa700 1 -- 172.21.15.7:0/1929974647 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f14c4051180 con 0x7f14dc070e50 2024-09-13T18:54:46.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.398+0000 7f14e1ced700 1 -- 172.21.15.7:0/1929974647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14dc0ada80 con 0x7f14dc070e50 2024-09-13T18:54:46.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.398+0000 7f14d97fa700 1 -- 172.21.15.7:0/1929974647 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f14c40547a0 con 0x7f14dc070e50 2024-09-13T18:54:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:46.406 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1829974483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1829974483},{"type":"v1","addr":"172.21.15.171:6835","nonce":1829974483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1948505526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1948505526},{"type":"v1","addr":"172.21.15.171:6837","nonce":1948505526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3617964311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3617964311},{"type":"v1","addr":"172.21.15.7:6833","nonce":3617964311}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:21.419627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/3768750832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3768750832},{"type":"v1","addr":"172.21.15.7:6835","nonce":3768750832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.402+0000 7f14e1ced700 1 -- 172.21.15.7:0/1929974647 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f14bc066730 msgr2=0x7f14bc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.402+0000 7f14e1ced700 1 --2- 172.21.15.7:0/1929974647 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f14bc066730 0x7f14bc068bf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f14cc001eb0 tx=0x7f14cc001d00 comp rx=0 tx=0).stop 2024-09-13T18:54:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.402+0000 7f14e1ced700 1 -- 172.21.15.7:0/1929974647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc070e50 msgr2=0x7f14dc14c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.402+0000 7f14e1ced700 1 --2- 172.21.15.7:0/1929974647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc070e50 0x7f14dc14c4e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f14c400e810 tx=0x7f14c400ebd0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.402+0000 7f14e1ced700 1 -- 172.21.15.7:0/1929974647 shutdown_connections 2024-09-13T18:54:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.402+0000 7f14e1ced700 1 --2- 172.21.15.7:0/1929974647 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f14bc066730 0x7f14bc068bf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.402+0000 7f14e1ced700 1 --2- 172.21.15.7:0/1929974647 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f14dc1599b0 0x7f14dc1499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.402+0000 7f14e1ced700 1 --2- 172.21.15.7:0/1929974647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14dc0cabf0 0x7f14dc14ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.402+0000 7f14e1ced700 1 --2- 172.21.15.7:0/1929974647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14dc070e50 0x7f14dc14c4e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.402+0000 7f14e1ced700 1 -- 172.21.15.7:0/1929974647 >> 172.21.15.7:0/1929974647 conn(0x7f14dc0b0570 msgr2=0x7f14dc0af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.402+0000 7f14e1ced700 1 -- 172.21.15.7:0/1929974647 shutdown_connections 2024-09-13T18:54:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.402+0000 7f14e1ced700 1 -- 172.21.15.7:0/1929974647 wait complete. 2024-09-13T18:54:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-09-13T18:54:46.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:54:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624b17d700 1 Processor -- start 2024-09-13T18:54:46.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624b17d700 1 -- start start 2024-09-13T18:54:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624b17d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62441606a0 0x7f62440d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624b17d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62440d1ca0 con 0x7f62440d17a0 2024-09-13T18:54:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624b17d700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62440cdd00 con 0x7f6244160b10 2024-09-13T18:54:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624b17d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62440cdfe0 con 0x7f62441606a0 2024-09-13T18:54:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624a17b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62441606a0 0x7f62440d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624997a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f62440d17a0 0x7f62440ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51482/0 (socket says 172.21.15.7:51482) 2024-09-13T18:54:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624a17b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62441606a0 0x7f62440d3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40292/0 (socket says 172.21.15.7:40292) 2024-09-13T18:54:46.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624997a700 1 -- 172.21.15.7:0/3220837696 learned_addr learned my addr 172.21.15.7:0/3220837696 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624a17b700 1 -- 172.21.15.7:0/3220837696 >> v1:172.21.15.171:6789/0 conn(0x7f6244160b10 legacy=0x7f62440b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624a17b700 1 -- 172.21.15.7:0/3220837696 >> v1:172.21.15.7:6789/0 conn(0x7f62440d17a0 legacy=0x7f62440ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624a17b700 1 -- 172.21.15.7:0/3220837696 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62440ce2c0 con 0x7f62441606a0 2024-09-13T18:54:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624a17b700 1 --2- 172.21.15.7:0/3220837696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62441606a0 0x7f62440d3800 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6240009880 tx=0x7f6240009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c66645b1b27bf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f6249179700 1 -- 172.21.15.7:0/3220837696 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240011830 con 0x7f62441606a0 2024-09-13T18:54:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f6249179700 1 -- 172.21.15.7:0/3220837696 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6240011990 con 0x7f62441606a0 2024-09-13T18:54:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f6249179700 1 -- 172.21.15.7:0/3220837696 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240011d20 con 0x7f62441606a0 2024-09-13T18:54:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624b17d700 1 -- 172.21.15.7:0/3220837696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62441606a0 msgr2=0x7f62440d3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624b17d700 1 --2- 172.21.15.7:0/3220837696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62441606a0 0x7f62440d3800 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6240009880 tx=0x7f6240009b90 comp rx=0 tx=0).stop 2024-09-13T18:54:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624b17d700 1 -- 172.21.15.7:0/3220837696 shutdown_connections 2024-09-13T18:54:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624b17d700 1 --2- 172.21.15.7:0/3220837696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62441606a0 0x7f62440d3800 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624b17d700 1 -- 172.21.15.7:0/3220837696 >> 172.21.15.7:0/3220837696 conn(0x7f62440d2d10 msgr2=0x7f62440d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624b17d700 1 -- 172.21.15.7:0/3220837696 shutdown_connections 2024-09-13T18:54:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.530+0000 7f624b17d700 1 -- 172.21.15.7:0/3220837696 wait complete. 2024-09-13T18:54:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624b17d700 1 Processor -- start 2024-09-13T18:54:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624b17d700 1 -- start start 2024-09-13T18:54:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624b17d700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f62440d17a0 0x7f62441f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624b17d700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62441606a0 0x7f6244153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624b17d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6244160b10 0x7f62440b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624b17d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62441599d0 con 0x7f6244160b10 2024-09-13T18:54:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624b17d700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6244159f90 con 0x7f62441606a0 2024-09-13T18:54:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624b17d700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62441589a0 con 0x7f62440d17a0 2024-09-13T18:54:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624a97c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6244160b10 0x7f62440b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624a97c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6244160b10 0x7f62440b7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40298/0 (socket says 172.21.15.7:40298) 2024-09-13T18:54:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624a97c700 1 -- 172.21.15.7:0/1577866259 learned_addr learned my addr 172.21.15.7:0/1577866259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624a17b700 1 --2- 172.21.15.7:0/1577866259 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f62440d17a0 0x7f62441f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624997a700 1 --2- 172.21.15.7:0/1577866259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62441606a0 0x7f6244153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624a97c700 1 -- 172.21.15.7:0/1577866259 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f62440d17a0 msgr2=0x7f62441f2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624a97c700 1 --2- 172.21.15.7:0/1577866259 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f62440d17a0 0x7f62441f2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624a97c700 1 -- 172.21.15.7:0/1577866259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62441606a0 msgr2=0x7f6244153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624a97c700 1 --2- 172.21.15.7:0/1577866259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62441606a0 0x7f6244153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624a97c700 1 -- 172.21.15.7:0/1577866259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62400094b0 con 0x7f6244160b10 2024-09-13T18:54:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624997a700 1 --2- 172.21.15.7:0/1577866259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62441606a0 0x7f6244153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624a97c700 1 --2- 172.21.15.7:0/1577866259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6244160b10 0x7f62440b7140 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f623c00cb50 tx=0x7f623c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f623affd700 1 -- 172.21.15.7:0/1577866259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c003c20 con 0x7f6244160b10 2024-09-13T18:54:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f623affd700 1 -- 172.21.15.7:0/1577866259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623c003d80 con 0x7f6244160b10 2024-09-13T18:54:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624b17d700 1 -- 172.21.15.7:0/1577866259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6244158c80 con 0x7f6244160b10 2024-09-13T18:54:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f623affd700 1 -- 172.21.15.7:0/1577866259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c00aac0 con 0x7f6244160b10 2024-09-13T18:54:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624b17d700 1 -- 172.21.15.7:0/1577866259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6244153880 con 0x7f6244160b10 2024-09-13T18:54:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.534+0000 7f624b17d700 1 -- 172.21.15.7:0/1577866259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f624415a100 con 0x7f6244160b10 2024-09-13T18:54:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.538+0000 7f623affd700 1 -- 172.21.15.7:0/1577866259 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f623c00ac20 con 0x7f6244160b10 2024-09-13T18:54:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.538+0000 7f623affd700 1 --2- 172.21.15.7:0/1577866259 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6228066760 0x7f6228068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.538+0000 7f623affd700 1 -- 172.21.15.7:0/1577866259 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f623c084b00 con 0x7f6244160b10 2024-09-13T18:54:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.538+0000 7f624a17b700 1 --2- 172.21.15.7:0/1577866259 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6228066760 0x7f6228068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.538+0000 7f624a17b700 1 --2- 172.21.15.7:0/1577866259 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6228066760 0x7f6228068c20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6240000c00 tx=0x7f6240008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:46.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.546+0000 7f623affd700 1 -- 172.21.15.7:0/1577866259 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f623c051f80 con 0x7f6244160b10 2024-09-13T18:54:46.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.718+0000 7f624b17d700 1 -- 172.21.15.7:0/1577866259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6244161360 con 0x7f6244160b10 2024-09-13T18:54:46.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.722+0000 7f623affd700 1 -- 172.21.15.7:0/1577866259 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16592 (secure 0 0 0) 0x7f623c0555a0 con 0x7f6244160b10 2024-09-13T18:54:46.727 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:46.727 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":56,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:54:17.402890+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:54:07.330160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T18:54:07.981407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/3168373873":"2024-09-14T18:54:17.388088+0000","172.21.15.7:6834/3168373873":"2024-09-14T18:54:17.388088+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:54:46.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.722+0000 7f624b17d700 1 -- 172.21.15.7:0/1577866259 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6228066760 msgr2=0x7f6228068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:46.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.722+0000 7f624b17d700 1 --2- 172.21.15.7:0/1577866259 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6228066760 0x7f6228068c20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6240000c00 tx=0x7f6240008030 comp rx=0 tx=0).stop 2024-09-13T18:54:46.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.722+0000 7f624b17d700 1 -- 172.21.15.7:0/1577866259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6244160b10 msgr2=0x7f62440b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:46.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.722+0000 7f624b17d700 1 --2- 172.21.15.7:0/1577866259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6244160b10 0x7f62440b7140 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f623c00cb50 tx=0x7f623c00ce60 comp rx=0 tx=0).stop 2024-09-13T18:54:46.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.722+0000 7f624b17d700 1 -- 172.21.15.7:0/1577866259 shutdown_connections 2024-09-13T18:54:46.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.722+0000 7f624b17d700 1 --2- 172.21.15.7:0/1577866259 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6228066760 0x7f6228068c20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.722+0000 7f624b17d700 1 --2- 172.21.15.7:0/1577866259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6244160b10 0x7f62440b7140 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.722+0000 7f624b17d700 1 --2- 172.21.15.7:0/1577866259 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62441606a0 0x7f6244153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.722+0000 7f624b17d700 1 --2- 172.21.15.7:0/1577866259 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f62440d17a0 0x7f62441f2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.722+0000 7f624b17d700 1 -- 172.21.15.7:0/1577866259 >> 172.21.15.7:0/1577866259 conn(0x7f62440d2d10 msgr2=0x7f62440b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:46.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.722+0000 7f624b17d700 1 -- 172.21.15.7:0/1577866259 shutdown_connections 2024-09-13T18:54:46.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.722+0000 7f624b17d700 1 -- 172.21.15.7:0/1577866259 wait complete. 2024-09-13T18:54:46.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T18:52:12.272418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T18:54:07.330160+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T18:54:07.981407+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T18:54:46.743 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T18:54:46.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.850+0000 7f17470aa700 1 Processor -- start 2024-09-13T18:54:46.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17470aa700 1 -- start start 2024-09-13T18:54:46.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17470aa700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f174015abb0 0x7f17400cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:46.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17470aa700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17400cdb70 0x7f17400cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:46.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17470aa700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17400ad860 con 0x7f17400afaa0 2024-09-13T18:54:46.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17470aa700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17400b01a0 con 0x7f17400cdb70 2024-09-13T18:54:46.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17470aa700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17400cfa00 con 0x7f174015abb0 2024-09-13T18:54:46.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17468a9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17400cdb70 0x7f17400cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:46.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17468a9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f17400cdb70 0x7f17400cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40302/0 (socket says 172.21.15.7:40302) 2024-09-13T18:54:46.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17468a9700 1 -- 172.21.15.7:0/605226697 learned_addr learned my addr 172.21.15.7:0/605226697 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:46.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17460a8700 1 --2- 172.21.15.7:0/605226697 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f174015abb0 0x7f17400cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:46.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17468a9700 1 -- 172.21.15.7:0/605226697 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f174015abb0 msgr2=0x7f17400cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:46.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17468a9700 1 --2- 172.21.15.7:0/605226697 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f174015abb0 0x7f17400cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17468a9700 1 -- 172.21.15.7:0/605226697 >> v1:172.21.15.171:6789/0 conn(0x7f17400afaa0 legacy=0x7f17400cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:46.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17468a9700 1 -- 172.21.15.7:0/605226697 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17400cb540 con 0x7f17400cdb70 2024-09-13T18:54:46.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17468a9700 1 --2- 172.21.15.7:0/605226697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17400cdb70 0x7f17400cb970 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1738009e50 tx=0x7f173800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d5585dc9e56128 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:46.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17450a6700 1 -- 172.21.15.7:0/605226697 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173800e070 con 0x7f17400cdb70 2024-09-13T18:54:46.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17450a6700 1 -- 172.21.15.7:0/605226697 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173800b9c0 con 0x7f17400cdb70 2024-09-13T18:54:46.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17450a6700 1 -- 172.21.15.7:0/605226697 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173800bd70 con 0x7f17400cdb70 2024-09-13T18:54:46.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17470aa700 1 -- 172.21.15.7:0/605226697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17400cdb70 msgr2=0x7f17400cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:46.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17470aa700 1 --2- 172.21.15.7:0/605226697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17400cdb70 0x7f17400cb970 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1738009e50 tx=0x7f173800b330 comp rx=0 tx=0).stop 2024-09-13T18:54:46.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17470aa700 1 -- 172.21.15.7:0/605226697 shutdown_connections 2024-09-13T18:54:46.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17470aa700 1 --2- 172.21.15.7:0/605226697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17400cdb70 0x7f17400cb970 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17470aa700 1 --2- 172.21.15.7:0/605226697 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f174015abb0 0x7f17400cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17470aa700 1 -- 172.21.15.7:0/605226697 >> 172.21.15.7:0/605226697 conn(0x7f17400b2580 msgr2=0x7f17400b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:46.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17470aa700 1 -- 172.21.15.7:0/605226697 shutdown_connections 2024-09-13T18:54:46.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.854+0000 7f17470aa700 1 -- 172.21.15.7:0/605226697 wait complete. 2024-09-13T18:54:46.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.858+0000 7f17470aa700 1 Processor -- start 2024-09-13T18:54:46.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.858+0000 7f17470aa700 1 -- start start 2024-09-13T18:54:46.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.858+0000 7f17470aa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17400afaa0 0x7f174014c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:46.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.858+0000 7f17470aa700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17400cdb70 0x7f17400b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:46.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.858+0000 7f17470aa700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f174015abb0 0x7f1740153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:46.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.858+0000 7f17470aa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17401517d0 con 0x7f17400afaa0 2024-09-13T18:54:46.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.858+0000 7f17470aa700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17400b3b90 con 0x7f174015abb0 2024-09-13T18:54:46.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.858+0000 7f17470aa700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17400b3e70 con 0x7f17400cdb70 2024-09-13T18:54:46.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.858+0000 7f17460a8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17400afaa0 0x7f174014c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:46.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.862+0000 7f17458a7700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17400cdb70 0x7f17400b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:46.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.862+0000 7f17460a8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17400afaa0 0x7f174014c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40318/0 (socket says 172.21.15.7:40318) 2024-09-13T18:54:46.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.862+0000 7f17460a8700 1 -- 172.21.15.7:0/3456791855 learned_addr learned my addr 172.21.15.7:0/3456791855 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:46.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.862+0000 7f17468a9700 1 --2- 172.21.15.7:0/3456791855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f174015abb0 0x7f1740153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.862+0000 7f17458a7700 1 -- 172.21.15.7:0/3456791855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f174015abb0 msgr2=0x7f1740153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.862+0000 7f17458a7700 1 --2- 172.21.15.7:0/3456791855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f174015abb0 0x7f1740153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.862+0000 7f17458a7700 1 -- 172.21.15.7:0/3456791855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17400afaa0 msgr2=0x7f174014c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.862+0000 7f17458a7700 1 --2- 172.21.15.7:0/3456791855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17400afaa0 0x7f174014c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.862+0000 7f17458a7700 1 -- 172.21.15.7:0/3456791855 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1738009b30 con 0x7f17400cdb70 2024-09-13T18:54:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.862+0000 7f17468a9700 1 --2- 172.21.15.7:0/3456791855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f174015abb0 0x7f1740153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:46.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.862+0000 7f17460a8700 1 --2- 172.21.15.7:0/3456791855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17400afaa0 0x7f174014c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:46.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.862+0000 7f17458a7700 1 --2- 172.21.15.7:0/3456791855 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17400cdb70 0x7f17400b5bf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f173000c390 tx=0x7f173000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:46.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.862+0000 7f1736ffd700 1 -- 172.21.15.7:0/3456791855 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17300052d0 con 0x7f17400cdb70 2024-09-13T18:54:46.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.862+0000 7f1736ffd700 1 -- 172.21.15.7:0/3456791855 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1730005450 con 0x7f17400cdb70 2024-09-13T18:54:46.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.862+0000 7f17470aa700 1 -- 172.21.15.7:0/3456791855 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1740151460 con 0x7f17400cdb70 2024-09-13T18:54:46.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.862+0000 7f1736ffd700 1 -- 172.21.15.7:0/3456791855 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1730006ac0 con 0x7f17400cdb70 2024-09-13T18:54:46.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.862+0000 7f17470aa700 1 -- 172.21.15.7:0/3456791855 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174014cb00 con 0x7f17400cdb70 2024-09-13T18:54:46.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.862+0000 7f17470aa700 1 -- 172.21.15.7:0/3456791855 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1740151980 con 0x7f17400cdb70 2024-09-13T18:54:46.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.866+0000 7f1736ffd700 1 -- 172.21.15.7:0/3456791855 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f17300036a0 con 0x7f17400cdb70 2024-09-13T18:54:46.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.866+0000 7f1736ffd700 1 --2- 172.21.15.7:0/3456791855 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1724066730 0x7f1724068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.866+0000 7f1736ffd700 1 -- 172.21.15.7:0/3456791855 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1730083240 con 0x7f17400cdb70 2024-09-13T18:54:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.866+0000 7f17460a8700 1 --2- 172.21.15.7:0/3456791855 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1724066730 0x7f1724068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:46.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.866+0000 7f17460a8700 1 --2- 172.21.15.7:0/3456791855 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1724066730 0x7f1724068bf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f173c007f00 tx=0x7f173c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:46.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:46.878+0000 7f1736ffd700 1 -- 172.21.15.7:0/3456791855 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1740151980 con 0x7f17400cdb70 2024-09-13T18:54:47.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.046+0000 7f17470aa700 1 -- 172.21.15.7:0/3456791855 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f17400b3fb0 con 0x7f17400cdb70 2024-09-13T18:54:47.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.050+0000 7f1736ffd700 1 -- 172.21.15.7:0/3456791855 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17400b3fb0 con 0x7f17400cdb70 2024-09-13T18:54:47.055 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-13T18:54:47.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.054+0000 7f17470aa700 1 -- 172.21.15.7:0/3456791855 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1724066730 msgr2=0x7f1724068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.054+0000 7f17470aa700 1 --2- 172.21.15.7:0/3456791855 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1724066730 0x7f1724068bf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f173c007f00 tx=0x7f173c008040 comp rx=0 tx=0).stop 2024-09-13T18:54:47.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.054+0000 7f17470aa700 1 -- 172.21.15.7:0/3456791855 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17400cdb70 msgr2=0x7f17400b5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.054+0000 7f17470aa700 1 --2- 172.21.15.7:0/3456791855 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17400cdb70 0x7f17400b5bf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f173000c390 tx=0x7f173000c6a0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.054+0000 7f17470aa700 1 -- 172.21.15.7:0/3456791855 shutdown_connections 2024-09-13T18:54:47.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.054+0000 7f17470aa700 1 --2- 172.21.15.7:0/3456791855 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1724066730 0x7f1724068bf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.054+0000 7f17470aa700 1 --2- 172.21.15.7:0/3456791855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f174015abb0 0x7f1740153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.054+0000 7f17470aa700 1 --2- 172.21.15.7:0/3456791855 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f17400cdb70 0x7f17400b5bf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.054+0000 7f17470aa700 1 --2- 172.21.15.7:0/3456791855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17400afaa0 0x7f174014c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.054+0000 7f17470aa700 1 -- 172.21.15.7:0/3456791855 >> 172.21.15.7:0/3456791855 conn(0x7f17400b2580 msgr2=0x7f17400b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:47.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.054+0000 7f17470aa700 1 -- 172.21.15.7:0/3456791855 shutdown_connections 2024-09-13T18:54:47.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.054+0000 7f17470aa700 1 -- 172.21.15.7:0/3456791855 wait complete. 2024-09-13T18:54:47.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T18:54:47.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f6190cde700 1 Processor -- start 2024-09-13T18:54:47.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f6190cde700 1 -- start start 2024-09-13T18:54:47.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f6190cde700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f618c1606a0 0x7f618c0d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f6190cde700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f618c0d17a0 0x7f618c0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f6190cde700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618c0ced20 con 0x7f618c160b10 2024-09-13T18:54:47.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f6190cde700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618c0d1ca0 con 0x7f618c0d17a0 2024-09-13T18:54:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f6190cde700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c0cdd00 con 0x7f618c1606a0 2024-09-13T18:54:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f618affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f618c0d17a0 0x7f618c0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f618bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f618c160b10 0x7f618c0b5b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51492/0 (socket says 172.21.15.7:51492) 2024-09-13T18:54:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f618affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f618c0d17a0 0x7f618c0ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40320/0 (socket says 172.21.15.7:40320) 2024-09-13T18:54:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f618bfff700 1 -- 172.21.15.7:0/4236455006 learned_addr learned my addr 172.21.15.7:0/4236455006 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f618b7fe700 1 --2- 172.21.15.7:0/4236455006 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f618c1606a0 0x7f618c0d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f618affd700 1 -- 172.21.15.7:0/4236455006 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f618c1606a0 msgr2=0x7f618c0d3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f618affd700 1 --2- 172.21.15.7:0/4236455006 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f618c1606a0 0x7f618c0d3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f618affd700 1 -- 172.21.15.7:0/4236455006 >> v1:172.21.15.7:6789/0 conn(0x7f618c160b10 legacy=0x7f618c0b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f618affd700 1 -- 172.21.15.7:0/4236455006 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c0cdfe0 con 0x7f618c0d17a0 2024-09-13T18:54:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f618b7fe700 1 --2- 172.21.15.7:0/4236455006 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f618c1606a0 0x7f618c0d3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f618affd700 1 --2- 172.21.15.7:0/4236455006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c0d17a0 0x7f618c0ce7e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6180009850 tx=0x7f6180009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47eee8910c12bd4c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:47.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f618a7fc700 1 -- 172.21.15.7:0/4236455006 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180011830 con 0x7f618c0d17a0 2024-09-13T18:54:47.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f618a7fc700 1 -- 172.21.15.7:0/4236455006 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6180011990 con 0x7f618c0d17a0 2024-09-13T18:54:47.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.190+0000 7f618a7fc700 1 -- 172.21.15.7:0/4236455006 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180011d20 con 0x7f618c0d17a0 2024-09-13T18:54:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f6190cde700 1 -- 172.21.15.7:0/4236455006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c0d17a0 msgr2=0x7f618c0ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f6190cde700 1 --2- 172.21.15.7:0/4236455006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c0d17a0 0x7f618c0ce7e0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6180009850 tx=0x7f6180009b60 comp rx=0 tx=0).stop 2024-09-13T18:54:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f6190cde700 1 -- 172.21.15.7:0/4236455006 shutdown_connections 2024-09-13T18:54:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f6190cde700 1 --2- 172.21.15.7:0/4236455006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c0d17a0 0x7f618c0ce7e0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f6190cde700 1 --2- 172.21.15.7:0/4236455006 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f618c1606a0 0x7f618c0d3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f6190cde700 1 -- 172.21.15.7:0/4236455006 >> 172.21.15.7:0/4236455006 conn(0x7f618c0d2d10 msgr2=0x7f618c0d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f6190cde700 1 -- 172.21.15.7:0/4236455006 shutdown_connections 2024-09-13T18:54:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f6190cde700 1 -- 172.21.15.7:0/4236455006 wait complete. 2024-09-13T18:54:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f6190cde700 1 Processor -- start 2024-09-13T18:54:47.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f6190cde700 1 -- start start 2024-09-13T18:54:47.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f6190cde700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f618c0d17a0 0x7f618c159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f6190cde700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f618c1606a0 0x7f618c0cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f6190cde700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c160b10 0x7f618c15cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f6190cde700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c15a0e0 con 0x7f618c160b10 2024-09-13T18:54:47.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f6190cde700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c0c9850 con 0x7f618c1606a0 2024-09-13T18:54:47.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f6190cde700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c0c8f20 con 0x7f618c0d17a0 2024-09-13T18:54:47.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f618bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c160b10 0x7f618c15cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f618b7fe700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f618c0d17a0 0x7f618c159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f618bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c160b10 0x7f618c15cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40324/0 (socket says 172.21.15.7:40324) 2024-09-13T18:54:47.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f618b7fe700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f618c0d17a0 0x7f618c159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:50240/0 (socket says 172.21.15.7:50240) 2024-09-13T18:54:47.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f618bfff700 1 -- 172.21.15.7:0/3187196809 learned_addr learned my addr 172.21.15.7:0/3187196809 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:47.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f618affd700 1 --2- 172.21.15.7:0/3187196809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f618c1606a0 0x7f618c0cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f618b7fe700 1 -- 172.21.15.7:0/3187196809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f618c1606a0 msgr2=0x7f618c0cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f618b7fe700 1 --2- 172.21.15.7:0/3187196809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f618c1606a0 0x7f618c0cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f618b7fe700 1 -- 172.21.15.7:0/3187196809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c160b10 msgr2=0x7f618c15cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f618b7fe700 1 --2- 172.21.15.7:0/3187196809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c160b10 0x7f618c15cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f618b7fe700 1 -- 172.21.15.7:0/3187196809 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61800094b0 con 0x7f618c0d17a0 2024-09-13T18:54:47.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.194+0000 7f618bfff700 1 --2- 172.21.15.7:0/3187196809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c160b10 0x7f618c15cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:47.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.198+0000 7f618b7fe700 1 --2- 172.21.15.7:0/3187196809 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f618c0d17a0 0x7f618c159ba0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f617c00e810 tx=0x7f617c00eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:47.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.198+0000 7f6173fff700 1 -- 172.21.15.7:0/3187196809 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c004230 con 0x7f618c0d17a0 2024-09-13T18:54:47.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.198+0000 7f6190cde700 1 -- 172.21.15.7:0/3187196809 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618c0c9200 con 0x7f618c0d17a0 2024-09-13T18:54:47.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.198+0000 7f6173fff700 1 -- 172.21.15.7:0/3187196809 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617c004390 con 0x7f618c0d17a0 2024-09-13T18:54:47.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.198+0000 7f6173fff700 1 -- 172.21.15.7:0/3187196809 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c0036c0 con 0x7f618c0d17a0 2024-09-13T18:54:47.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.198+0000 7f6190cde700 1 -- 172.21.15.7:0/3187196809 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618c0c8b40 con 0x7f618c0d17a0 2024-09-13T18:54:47.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.198+0000 7f6190cde700 1 -- 172.21.15.7:0/3187196809 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618c0ce7e0 con 0x7f618c0d17a0 2024-09-13T18:54:47.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.198+0000 7f6173fff700 1 -- 172.21.15.7:0/3187196809 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f617c003820 con 0x7f618c0d17a0 2024-09-13T18:54:47.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.202+0000 7f6173fff700 1 --2- 172.21.15.7:0/3187196809 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6178066730 0x7f6178068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.202+0000 7f6173fff700 1 -- 172.21.15.7:0/3187196809 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f617c083af0 con 0x7f618c0d17a0 2024-09-13T18:54:47.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.202+0000 7f618affd700 1 --2- 172.21.15.7:0/3187196809 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6178066730 0x7f6178068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.202+0000 7f618affd700 1 --2- 172.21.15.7:0/3187196809 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6178066730 0x7f6178068bf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6180009850 tx=0x7f6180008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:47.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.210+0000 7f6173fff700 1 -- 172.21.15.7:0/3187196809 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f617c050d80 con 0x7f618c0d17a0 2024-09-13T18:54:47.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.378+0000 7f6190cde700 1 -- 172.21.15.7:0/3187196809 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f618c0c9060 con 0x7f618c0d17a0 2024-09-13T18:54:47.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.378+0000 7f6173fff700 1 -- 172.21.15.7:0/3187196809 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7f618c0c9060 con 0x7f618c0d17a0 2024-09-13T18:54:47.385 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-13T18:54:47.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.382+0000 7f6190cde700 1 -- 172.21.15.7:0/3187196809 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6178066730 msgr2=0x7f6178068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.382+0000 7f6190cde700 1 --2- 172.21.15.7:0/3187196809 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6178066730 0x7f6178068bf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6180009850 tx=0x7f6180008030 comp rx=0 tx=0).stop 2024-09-13T18:54:47.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.382+0000 7f6190cde700 1 -- 172.21.15.7:0/3187196809 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f618c0d17a0 msgr2=0x7f618c159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.382+0000 7f6190cde700 1 --2- 172.21.15.7:0/3187196809 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f618c0d17a0 0x7f618c159ba0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f617c00e810 tx=0x7f617c00eb20 comp rx=0 tx=0).stop 2024-09-13T18:54:47.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.382+0000 7f6190cde700 1 -- 172.21.15.7:0/3187196809 shutdown_connections 2024-09-13T18:54:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.382+0000 7f6190cde700 1 --2- 172.21.15.7:0/3187196809 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6178066730 0x7f6178068bf0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.382+0000 7f6190cde700 1 --2- 172.21.15.7:0/3187196809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f618c160b10 0x7f618c15cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.382+0000 7f6190cde700 1 --2- 172.21.15.7:0/3187196809 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f618c1606a0 0x7f618c0cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.382+0000 7f6190cde700 1 --2- 172.21.15.7:0/3187196809 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f618c0d17a0 0x7f618c159ba0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.382+0000 7f6190cde700 1 -- 172.21.15.7:0/3187196809 >> 172.21.15.7:0/3187196809 conn(0x7f618c0d2d10 msgr2=0x7f618c06bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.382+0000 7f6190cde700 1 -- 172.21.15.7:0/3187196809 shutdown_connections 2024-09-13T18:54:47.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.382+0000 7f6190cde700 1 -- 172.21.15.7:0/3187196809 wait complete. 2024-09-13T18:54:47.401 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T18:54:47.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5bdfaa700 1 Processor -- start 2024-09-13T18:54:47.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5bdfaa700 1 -- start start 2024-09-13T18:54:47.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5bdfaa700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5b8070e50 0x7fc5b80ada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5bdfaa700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b80b1aa0 con 0x7fc5b80cabf0 2024-09-13T18:54:47.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5bdfaa700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b806be40 con 0x7fc5b81599b0 2024-09-13T18:54:47.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5bdfaa700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b806c3c0 con 0x7fc5b8070e50 2024-09-13T18:54:47.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5bd7a9700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc5b80cabf0 0x7fc5b80b1f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51494/0 (socket says 172.21.15.7:51494) 2024-09-13T18:54:47.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5bd7a9700 1 -- 172.21.15.7:0/2930539915 learned_addr learned my addr 172.21.15.7:0/2930539915 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:47.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b7fff700 1 --2- 172.21.15.7:0/2930539915 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5b8070e50 0x7fc5b80ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b77fe700 1 -- 172.21.15.7:0/2930539915 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1127764547 0 0) 0x7fc5b80b1aa0 con 0x7fc5b80cabf0 2024-09-13T18:54:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b77fe700 1 -- 172.21.15.7:0/2930539915 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5a40035d0 con 0x7fc5b80cabf0 2024-09-13T18:54:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b77fe700 1 -- 172.21.15.7:0/2930539915 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3975250882 0 0) 0x7fc5a40035d0 con 0x7fc5b80cabf0 2024-09-13T18:54:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b77fe700 1 -- 172.21.15.7:0/2930539915 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5b8070e50 msgr2=0x7fc5b80ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b77fe700 1 --2- 172.21.15.7:0/2930539915 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5b8070e50 0x7fc5b80ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b77fe700 1 -- 172.21.15.7:0/2930539915 >> v1:172.21.15.171:6789/0 conn(0x7fc5b81599b0 legacy=0x7fc5b8070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b77fe700 1 -- 172.21.15.7:0/2930539915 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b80cb140 con 0x7fc5b80cabf0 2024-09-13T18:54:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b77fe700 1 -- 172.21.15.7:0/2930539915 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7fc5b0001670 con 0x7fc5b80cabf0 2024-09-13T18:54:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b77fe700 1 -- 172.21.15.7:0/2930539915 >> v1:172.21.15.7:6789/0 conn(0x7fc5b80cabf0 legacy=0x7fc5b80b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b77fe700 1 --2- 172.21.15.7:0/2930539915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5a4003a50 0x7fc5a4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b77fe700 1 --2- 172.21.15.7:0/2930539915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a4004870 0x7fc5a4004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b77fe700 1 --2- 172.21.15.7:0/2930539915 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5b8070e50 0x7fc5a4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b77fe700 1 -- 172.21.15.7:0/2930539915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b806c3c0 con 0x7fc5a4004870 2024-09-13T18:54:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b77fe700 1 -- 172.21.15.7:0/2930539915 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a400dd20 con 0x7fc5a4003a50 2024-09-13T18:54:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b77fe700 1 -- 172.21.15.7:0/2930539915 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a400dee0 con 0x7fc5b8070e50 2024-09-13T18:54:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5bd7a9700 1 --2- 172.21.15.7:0/2930539915 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5b8070e50 0x7fc5a4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b7fff700 1 --2- 172.21.15.7:0/2930539915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a4004870 0x7fc5a4004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5bcfa8700 1 --2- 172.21.15.7:0/2930539915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5a4003a50 0x7fc5a4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b7fff700 1 -- 172.21.15.7:0/2930539915 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5b8070e50 msgr2=0x7fc5a4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b7fff700 1 --2- 172.21.15.7:0/2930539915 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5b8070e50 0x7fc5a4005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b7fff700 1 -- 172.21.15.7:0/2930539915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5a4003a50 msgr2=0x7fc5a4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b7fff700 1 --2- 172.21.15.7:0/2930539915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5a4003a50 0x7fc5a4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b7fff700 1 -- 172.21.15.7:0/2930539915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a40035d0 con 0x7fc5a4004870 2024-09-13T18:54:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5bcfa8700 1 --2- 172.21.15.7:0/2930539915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5a4003a50 0x7fc5a4003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:47.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.518+0000 7fc5b7fff700 1 --2- 172.21.15.7:0/2930539915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a4004870 0x7fc5a4004cf0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc5b806dd10 tx=0x7fc5a40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ccfa133d3ab7b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:47.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.522+0000 7fc5b77fe700 1 -- 172.21.15.7:0/2930539915 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a000e690 con 0x7fc5a4004870 2024-09-13T18:54:47.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.522+0000 7fc5b77fe700 1 -- 172.21.15.7:0/2930539915 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a000e7f0 con 0x7fc5a4004870 2024-09-13T18:54:47.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.522+0000 7fc5b77fe700 1 -- 172.21.15.7:0/2930539915 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a000eb80 con 0x7fc5a4004870 2024-09-13T18:54:47.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.526+0000 7fc5bdfaa700 1 -- 172.21.15.7:0/2930539915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a4004870 msgr2=0x7fc5a4004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.526+0000 7fc5bdfaa700 1 --2- 172.21.15.7:0/2930539915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5a4004870 0x7fc5a4004cf0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc5b806dd10 tx=0x7fc5a40058a0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.526+0000 7fc5bd7a9700 1 -- 172.21.15.7:0/2930539915 reap_dead start 2024-09-13T18:54:47.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.526+0000 7fc5bdfaa700 1 -- 172.21.15.7:0/2930539915 shutdown_connections 2024-09-13T18:54:47.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.526+0000 7fc5bdfaa700 1 -- 172.21.15.7:0/2930539915 >> 172.21.15.7:0/2930539915 conn(0x7fc5b80b0570 msgr2=0x7fc5b80b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:47.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.526+0000 7fc5bdfaa700 1 -- 172.21.15.7:0/2930539915 shutdown_connections 2024-09-13T18:54:47.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.526+0000 7fc5bdfaa700 1 -- 172.21.15.7:0/2930539915 wait complete. 2024-09-13T18:54:47.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.526+0000 7fc5bdfaa700 1 Processor -- start 2024-09-13T18:54:47.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.526+0000 7fc5bdfaa700 1 -- start start 2024-09-13T18:54:47.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.526+0000 7fc5bdfaa700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5a4004870 0x7fc5b81f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.526+0000 7fc5bdfaa700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5b81509e0 0x7fc5b80cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.526+0000 7fc5bdfaa700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5b8145bc0 0x7fc5b8146040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.526+0000 7fc5bdfaa700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b8155a70 con 0x7fc5b81509e0 2024-09-13T18:54:47.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.526+0000 7fc5bdfaa700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b8146580 con 0x7fc5b8145bc0 2024-09-13T18:54:47.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.526+0000 7fc5bdfaa700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b81f73f0 con 0x7fc5a4004870 2024-09-13T18:54:47.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.530+0000 7fc5b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5b81509e0 0x7fc5b80cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.530+0000 7fc5bcfa8700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5a4004870 0x7fc5b81f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.530+0000 7fc5b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5b81509e0 0x7fc5b80cdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40340/0 (socket says 172.21.15.7:40340) 2024-09-13T18:54:47.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.530+0000 7fc5b7fff700 1 -- 172.21.15.7:0/312133218 learned_addr learned my addr 172.21.15.7:0/312133218 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:47.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.530+0000 7fc5bd7a9700 1 --2- 172.21.15.7:0/312133218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5b8145bc0 0x7fc5b8146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.530+0000 7fc5b7fff700 1 -- 172.21.15.7:0/312133218 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5a4004870 msgr2=0x7fc5b81f6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.530+0000 7fc5b7fff700 1 --2- 172.21.15.7:0/312133218 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5a4004870 0x7fc5b81f6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.530+0000 7fc5b7fff700 1 -- 172.21.15.7:0/312133218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5b8145bc0 msgr2=0x7fc5b8146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.530+0000 7fc5b7fff700 1 --2- 172.21.15.7:0/312133218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5b8145bc0 0x7fc5b8146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.530+0000 7fc5b7fff700 1 -- 172.21.15.7:0/312133218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b8150f80 con 0x7fc5b81509e0 2024-09-13T18:54:47.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.530+0000 7fc5b7fff700 1 --2- 172.21.15.7:0/312133218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5b81509e0 0x7fc5b80cdb70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80ce6e0 tx=0x7fc5a00047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:47.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.530+0000 7fc5b57fa700 1 -- 172.21.15.7:0/312133218 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a000e690 con 0x7fc5b81509e0 2024-09-13T18:54:47.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.530+0000 7fc5b57fa700 1 -- 172.21.15.7:0/312133218 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a0013c00 con 0x7fc5b81509e0 2024-09-13T18:54:47.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.530+0000 7fc5b57fa700 1 -- 172.21.15.7:0/312133218 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a001c450 con 0x7fc5b81509e0 2024-09-13T18:54:47.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.530+0000 7fc5bdfaa700 1 -- 172.21.15.7:0/312133218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b8155a70 con 0x7fc5b81509e0 2024-09-13T18:54:47.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.530+0000 7fc5bdfaa700 1 -- 172.21.15.7:0/312133218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b8151540 con 0x7fc5b81509e0 2024-09-13T18:54:47.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.530+0000 7fc5bdfaa700 1 -- 172.21.15.7:0/312133218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b80ada80 con 0x7fc5b81509e0 2024-09-13T18:54:47.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.534+0000 7fc5b57fa700 1 -- 172.21.15.7:0/312133218 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc5a001c5b0 con 0x7fc5b81509e0 2024-09-13T18:54:47.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.534+0000 7fc5b57fa700 1 --2- 172.21.15.7:0/312133218 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc598066730 0x7fc598068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.534+0000 7fc5b57fa700 1 -- 172.21.15.7:0/312133218 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc5a0085970 con 0x7fc5b81509e0 2024-09-13T18:54:47.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.534+0000 7fc5bcfa8700 1 --2- 172.21.15.7:0/312133218 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc598066730 0x7fc598068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.534+0000 7fc5bcfa8700 1 --2- 172.21.15.7:0/312133218 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc598066730 0x7fc598068bf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80021e0 tx=0x7fc5a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.538+0000 7fc5b57fa700 1 -- 172.21.15.7:0/312133218 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc5a0052e70 con 0x7fc5b81509e0 2024-09-13T18:54:47.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.718+0000 7fc5bdfaa700 1 -- 172.21.15.7:0/312133218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc5b80b1290 con 0x7fc5b81509e0 2024-09-13T18:54:47.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.718+0000 7fc5b57fa700 1 -- 172.21.15.7:0/312133218 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc5a0017020 con 0x7fc5b81509e0 2024-09-13T18:54:47.726 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-13T18:54:47.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.722+0000 7fc5bdfaa700 1 -- 172.21.15.7:0/312133218 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc598066730 msgr2=0x7fc598068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.722+0000 7fc5bdfaa700 1 --2- 172.21.15.7:0/312133218 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc598066730 0x7fc598068bf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80021e0 tx=0x7fc5a8009040 comp rx=0 tx=0).stop 2024-09-13T18:54:47.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.722+0000 7fc5bdfaa700 1 -- 172.21.15.7:0/312133218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5b81509e0 msgr2=0x7fc5b80cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.722+0000 7fc5bdfaa700 1 --2- 172.21.15.7:0/312133218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5b81509e0 0x7fc5b80cdb70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80ce6e0 tx=0x7fc5a00047e0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.722+0000 7fc5bdfaa700 1 -- 172.21.15.7:0/312133218 shutdown_connections 2024-09-13T18:54:47.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.722+0000 7fc5bdfaa700 1 --2- 172.21.15.7:0/312133218 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc598066730 0x7fc598068bf0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.722+0000 7fc5bdfaa700 1 --2- 172.21.15.7:0/312133218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc5b8145bc0 0x7fc5b8146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.722+0000 7fc5bdfaa700 1 --2- 172.21.15.7:0/312133218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5b81509e0 0x7fc5b80cdb70 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.722+0000 7fc5bdfaa700 1 --2- 172.21.15.7:0/312133218 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc5a4004870 0x7fc5b81f6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.722+0000 7fc5bdfaa700 1 -- 172.21.15.7:0/312133218 >> 172.21.15.7:0/312133218 conn(0x7fc5b80b0570 msgr2=0x7fc5b814dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:47.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.722+0000 7fc5bdfaa700 1 -- 172.21.15.7:0/312133218 shutdown_connections 2024-09-13T18:54:47.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.722+0000 7fc5bdfaa700 1 -- 172.21.15.7:0/312133218 wait complete. 2024-09-13T18:54:47.741 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:47.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.846+0000 7f642e4cf700 1 Processor -- start 2024-09-13T18:54:47.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.846+0000 7f642e4cf700 1 -- start start 2024-09-13T18:54:47.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.850+0000 7f642e4cf700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f642806a360 0x7f64280b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.850+0000 7f642e4cf700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642806e610 con 0x7f642806b750 2024-09-13T18:54:47.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.850+0000 7f642e4cf700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642806d5c0 con 0x7f64280b23b0 2024-09-13T18:54:47.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.850+0000 7f642e4cf700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64280b3470 con 0x7f642806a360 2024-09-13T18:54:47.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.850+0000 7f642dcce700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f642806b750 0x7f64280cc4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51496/0 (socket says 172.21.15.7:51496) 2024-09-13T18:54:47.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.850+0000 7f642dcce700 1 -- 172.21.15.7:0/3449496021 learned_addr learned my addr 172.21.15.7:0/3449496021 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:47.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.850+0000 7f642d4cd700 1 --2- 172.21.15.7:0/3449496021 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f642806a360 0x7f64280b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.850+0000 7f641ffff700 1 -- 172.21.15.7:0/3449496021 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820096554 0 0) 0x7f642806e610 con 0x7f642806b750 2024-09-13T18:54:47.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.850+0000 7f641ffff700 1 -- 172.21.15.7:0/3449496021 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64140035d0 con 0x7f642806b750 2024-09-13T18:54:47.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.850+0000 7f641ffff700 1 -- 172.21.15.7:0/3449496021 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147387339 0 0) 0x7f642806d5c0 con 0x7f64280b23b0 2024-09-13T18:54:47.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.850+0000 7f641ffff700 1 -- 172.21.15.7:0/3449496021 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f642806e610 con 0x7f64280b23b0 2024-09-13T18:54:47.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.850+0000 7f642d4cd700 1 -- 172.21.15.7:0/3449496021 >> v1:172.21.15.171:6789/0 conn(0x7f64280b23b0 legacy=0x7f64280cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.850+0000 7f642d4cd700 1 -- 172.21.15.7:0/3449496021 >> v1:172.21.15.7:6789/0 conn(0x7f642806b750 legacy=0x7f64280cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.850+0000 7f642d4cd700 1 -- 172.21.15.7:0/3449496021 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642806d850 con 0x7f642806a360 2024-09-13T18:54:47.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.850+0000 7f642d4cd700 1 --2- 172.21.15.7:0/3449496021 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f642806a360 0x7f64280b2e20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6424009880 tx=0x7f6424009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26fe3269d9473a4a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:47.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.850+0000 7f641ffff700 1 -- 172.21.15.7:0/3449496021 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424011830 con 0x7f642806a360 2024-09-13T18:54:47.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.850+0000 7f641ffff700 1 -- 172.21.15.7:0/3449496021 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6424011990 con 0x7f642806a360 2024-09-13T18:54:47.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.850+0000 7f641ffff700 1 -- 172.21.15.7:0/3449496021 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424011d20 con 0x7f642806a360 2024-09-13T18:54:47.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.854+0000 7f642e4cf700 1 -- 172.21.15.7:0/3449496021 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f642806a360 msgr2=0x7f64280b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.854+0000 7f642e4cf700 1 --2- 172.21.15.7:0/3449496021 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f642806a360 0x7f64280b2e20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6424009880 tx=0x7f6424009b90 comp rx=0 tx=0).stop 2024-09-13T18:54:47.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.854+0000 7f642e4cf700 1 -- 172.21.15.7:0/3449496021 shutdown_connections 2024-09-13T18:54:47.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.854+0000 7f642e4cf700 1 --2- 172.21.15.7:0/3449496021 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f642806a360 0x7f64280b2e20 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.854+0000 7f642e4cf700 1 -- 172.21.15.7:0/3449496021 >> 172.21.15.7:0/3449496021 conn(0x7f64280cf300 msgr2=0x7f64280cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:47.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.854+0000 7f642e4cf700 1 -- 172.21.15.7:0/3449496021 shutdown_connections 2024-09-13T18:54:47.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.854+0000 7f642e4cf700 1 -- 172.21.15.7:0/3449496021 wait complete. 2024-09-13T18:54:47.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642e4cf700 1 Processor -- start 2024-09-13T18:54:47.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642e4cf700 1 -- start start 2024-09-13T18:54:47.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642e4cf700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f642806a360 0x7f64281459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642e4cf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f642806b750 0x7f6428145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642e4cf700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64280b23b0 0x7f642814f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642e4cf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642815af00 con 0x7f642806b750 2024-09-13T18:54:47.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642e4cf700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642815aa70 con 0x7f64280b23b0 2024-09-13T18:54:47.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642e4cf700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642814fff0 con 0x7f642806a360 2024-09-13T18:54:47.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642cccc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f642806b750 0x7f6428145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642cccc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f642806b750 0x7f6428145ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40346/0 (socket says 172.21.15.7:40346) 2024-09-13T18:54:47.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642cccc700 1 -- 172.21.15.7:0/2724093648 learned_addr learned my addr 172.21.15.7:0/2724093648 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:47.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642d4cd700 1 --2- 172.21.15.7:0/2724093648 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f642806a360 0x7f64281459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642dcce700 1 --2- 172.21.15.7:0/2724093648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64280b23b0 0x7f642814f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642cccc700 1 -- 172.21.15.7:0/2724093648 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f642806a360 msgr2=0x7f64281459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642cccc700 1 --2- 172.21.15.7:0/2724093648 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f642806a360 0x7f64281459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642cccc700 1 -- 172.21.15.7:0/2724093648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64280b23b0 msgr2=0x7f642814f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:47.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642cccc700 1 --2- 172.21.15.7:0/2724093648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64280b23b0 0x7f642814f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:47.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642cccc700 1 -- 172.21.15.7:0/2724093648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64240094b0 con 0x7f642806b750 2024-09-13T18:54:47.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.858+0000 7f642cccc700 1 --2- 172.21.15.7:0/2724093648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f642806b750 0x7f6428145ef0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f641800c550 tx=0x7f641800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.862+0000 7f641dffb700 1 -- 172.21.15.7:0/2724093648 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64180052d0 con 0x7f642806b750 2024-09-13T18:54:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.862+0000 7f641dffb700 1 -- 172.21.15.7:0/2724093648 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6418005430 con 0x7f642806b750 2024-09-13T18:54:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.862+0000 7f641dffb700 1 -- 172.21.15.7:0/2724093648 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64180055a0 con 0x7f642806b750 2024-09-13T18:54:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.862+0000 7f642e4cf700 1 -- 172.21.15.7:0/2724093648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642815a140 con 0x7f642806b750 2024-09-13T18:54:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.862+0000 7f642e4cf700 1 -- 172.21.15.7:0/2724093648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642815a700 con 0x7f642806b750 2024-09-13T18:54:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.862+0000 7f641dffb700 1 -- 172.21.15.7:0/2724093648 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f64180038e0 con 0x7f642806b750 2024-09-13T18:54:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.862+0000 7f642e4cf700 1 -- 172.21.15.7:0/2724093648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6428159380 con 0x7f642806b750 2024-09-13T18:54:47.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.862+0000 7f641dffb700 1 --2- 172.21.15.7:0/2724093648 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f640c066760 0x7f640c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:47.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.862+0000 7f641dffb700 1 -- 172.21.15.7:0/2724093648 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6418083570 con 0x7f642806b750 2024-09-13T18:54:47.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.862+0000 7f642d4cd700 1 --2- 172.21.15.7:0/2724093648 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f640c066760 0x7f640c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:47.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.862+0000 7f642d4cd700 1 --2- 172.21.15.7:0/2724093648 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f640c066760 0x7f640c068c20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f64240090b0 tx=0x7f642401c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:47.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:47.874+0000 7f641dffb700 1 -- 172.21.15.7:0/2724093648 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6418050860 con 0x7f642806b750 2024-09-13T18:54:48.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.074+0000 7f642e4cf700 1 -- 172.21.15.7:0/2724093648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64280af590 con 0x7f642806b750 2024-09-13T18:54:48.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.074+0000 7f641dffb700 1 -- 172.21.15.7:0/2724093648 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f6418053e80 con 0x7f642806b750 2024-09-13T18:54:48.080 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:48.080 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1829974483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1829974483},{"type":"v1","addr":"172.21.15.171:6835","nonce":1829974483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1948505526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1948505526},{"type":"v1","addr":"172.21.15.171:6837","nonce":1948505526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3617964311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3617964311},{"type":"v1","addr":"172.21.15.7:6833","nonce":3617964311}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:21.419627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/3768750832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3768750832},{"type":"v1","addr":"172.21.15.7:6835","nonce":3768750832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.074+0000 7f642e4cf700 1 -- 172.21.15.7:0/2724093648 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f640c066760 msgr2=0x7f640c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.074+0000 7f642e4cf700 1 --2- 172.21.15.7:0/2724093648 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f640c066760 0x7f640c068c20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f64240090b0 tx=0x7f642401c040 comp rx=0 tx=0).stop 2024-09-13T18:54:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.074+0000 7f642e4cf700 1 -- 172.21.15.7:0/2724093648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f642806b750 msgr2=0x7f6428145ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.074+0000 7f642e4cf700 1 --2- 172.21.15.7:0/2724093648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f642806b750 0x7f6428145ef0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f641800c550 tx=0x7f641800c860 comp rx=0 tx=0).stop 2024-09-13T18:54:48.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.074+0000 7f642e4cf700 1 -- 172.21.15.7:0/2724093648 shutdown_connections 2024-09-13T18:54:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.078+0000 7f642e4cf700 1 --2- 172.21.15.7:0/2724093648 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f640c066760 0x7f640c068c20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.078+0000 7f642e4cf700 1 --2- 172.21.15.7:0/2724093648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f64280b23b0 0x7f642814f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.078+0000 7f642e4cf700 1 --2- 172.21.15.7:0/2724093648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f642806b750 0x7f6428145ef0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.078+0000 7f642e4cf700 1 --2- 172.21.15.7:0/2724093648 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f642806a360 0x7f64281459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.078+0000 7f642e4cf700 1 -- 172.21.15.7:0/2724093648 >> 172.21.15.7:0/2724093648 conn(0x7f64280cf300 msgr2=0x7f6428156d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.078+0000 7f642e4cf700 1 -- 172.21.15.7:0/2724093648 shutdown_connections 2024-09-13T18:54:48.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.078+0000 7f642e4cf700 1 -- 172.21.15.7:0/2724093648 wait complete. 2024-09-13T18:54:48.084 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-09-13T18:54:48.095 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:54:48.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f90a1ed8700 1 Processor -- start 2024-09-13T18:54:48.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f90a1ed8700 1 -- start start 2024-09-13T18:54:48.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f90a1ed8700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f909c0b3d10 0x7f909c0cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f90a1ed8700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f909c0d1810 0x7f909c1614c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f90a1ed8700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f909c0b2870 con 0x7f909c0d1430 2024-09-13T18:54:48.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f90a1ed8700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f909c0d3700 con 0x7f909c0d1810 2024-09-13T18:54:48.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f90a1ed8700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f909c0b1fd0 con 0x7f909c0b3d10 2024-09-13T18:54:48.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f90a16d7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f909c0d1810 0x7f909c1614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f90a16d7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f909c0d1810 0x7f909c1614c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40360/0 (socket says 172.21.15.7:40360) 2024-09-13T18:54:48.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f90a16d7700 1 -- 172.21.15.7:0/4027051005 learned_addr learned my addr 172.21.15.7:0/4027051005 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:48.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f909bfff700 1 --2- 172.21.15.7:0/4027051005 >> v2:172.21.15.171:3300/0 conn(0x7f909c0b3d10 0x7f909c0cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f909bfff700 1 -- 172.21.15.7:0/4027051005 >> v2:172.21.15.7:3300/0 conn(0x7f909c0d1810 msgr2=0x7f909c1614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:48.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f909bfff700 1 --2- 172.21.15.7:0/4027051005 >> v2:172.21.15.7:3300/0 conn(0x7f909c0d1810 0x7f909c1614c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f909bfff700 1 -- 172.21.15.7:0/4027051005 >> v1:172.21.15.171:6789/0 conn(0x7f909c0d1430 legacy=0x7f909c160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f909bfff700 1 -- 172.21.15.7:0/4027051005 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c0d3950 con 0x7f909c0b3d10 2024-09-13T18:54:48.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f909bfff700 1 --2- 172.21.15.7:0/4027051005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f909c0b3d10 0x7f909c0cfc20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f909c0d2b40 tx=0x7f909400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87748cb3cbb2dcdf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:48.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f909b7fe700 1 -- 172.21.15.7:0/4027051005 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400e070 con 0x7f909c0b3d10 2024-09-13T18:54:48.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f909b7fe700 1 -- 172.21.15.7:0/4027051005 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909400bb20 con 0x7f909c0b3d10 2024-09-13T18:54:48.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f909b7fe700 1 -- 172.21.15.7:0/4027051005 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094012430 con 0x7f909c0b3d10 2024-09-13T18:54:48.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f90a1ed8700 1 -- 172.21.15.7:0/4027051005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f909c0b3d10 msgr2=0x7f909c0cfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f90a1ed8700 1 --2- 172.21.15.7:0/4027051005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f909c0b3d10 0x7f909c0cfc20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f909c0d2b40 tx=0x7f909400b330 comp rx=0 tx=0).stop 2024-09-13T18:54:48.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f90a1ed8700 1 -- 172.21.15.7:0/4027051005 shutdown_connections 2024-09-13T18:54:48.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f90a1ed8700 1 --2- 172.21.15.7:0/4027051005 >> v2:172.21.15.7:3300/0 conn(0x7f909c0d1810 0x7f909c1614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f90a1ed8700 1 --2- 172.21.15.7:0/4027051005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f909c0b3d10 0x7f909c0cfc20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f90a1ed8700 1 -- 172.21.15.7:0/4027051005 >> 172.21.15.7:0/4027051005 conn(0x7f909c0d2be0 msgr2=0x7f909c0d2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:48.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f90a1ed8700 1 -- 172.21.15.7:0/4027051005 shutdown_connections 2024-09-13T18:54:48.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.206+0000 7f90a1ed8700 1 -- 172.21.15.7:0/4027051005 wait complete. 2024-09-13T18:54:48.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f90a1ed8700 1 Processor -- start 2024-09-13T18:54:48.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f90a1ed8700 1 -- start start 2024-09-13T18:54:48.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f90a1ed8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f909c0b3d10 0x7f909c155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f90a1ed8700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f909c0d1430 0x7f909c0b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f90a1ed8700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f909c0d1810 0x7f909c0b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f90a1ed8700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c0b5a30 con 0x7f909c0b3d10 2024-09-13T18:54:48.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f90a1ed8700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c0b5d10 con 0x7f909c0d1810 2024-09-13T18:54:48.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f90a1ed8700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909c0b5ff0 con 0x7f909c0d1430 2024-09-13T18:54:48.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f90a0ed6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f909c0b3d10 0x7f909c155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f90a0ed6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f909c0b3d10 0x7f909c155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40368/0 (socket says 172.21.15.7:40368) 2024-09-13T18:54:48.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f90a0ed6700 1 -- 172.21.15.7:0/3473869880 learned_addr learned my addr 172.21.15.7:0/3473869880 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:48.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f909bfff700 1 --2- 172.21.15.7:0/3473869880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f909c0d1430 0x7f909c0b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f90a16d7700 1 --2- 172.21.15.7:0/3473869880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f909c0d1810 0x7f909c0b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f90a0ed6700 1 -- 172.21.15.7:0/3473869880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f909c0d1430 msgr2=0x7f909c0b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f90a0ed6700 1 --2- 172.21.15.7:0/3473869880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f909c0d1430 0x7f909c0b6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f90a0ed6700 1 -- 172.21.15.7:0/3473869880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f909c0d1810 msgr2=0x7f909c0b7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f90a0ed6700 1 --2- 172.21.15.7:0/3473869880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f909c0d1810 0x7f909c0b7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f90a0ed6700 1 -- 172.21.15.7:0/3473869880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9094009a70 con 0x7f909c0b3d10 2024-09-13T18:54:48.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.210+0000 7f90a0ed6700 1 --2- 172.21.15.7:0/3473869880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f909c0b3d10 0x7f909c155a10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f908400c500 tx=0x7f908400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:48.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.214+0000 7f90997fa700 1 -- 172.21.15.7:0/3473869880 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90840052d0 con 0x7f909c0b3d10 2024-09-13T18:54:48.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.214+0000 7f90997fa700 1 -- 172.21.15.7:0/3473869880 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9084005430 con 0x7f909c0b3d10 2024-09-13T18:54:48.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.214+0000 7f90a1ed8700 1 -- 172.21.15.7:0/3473869880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f909c0b62d0 con 0x7f909c0b3d10 2024-09-13T18:54:48.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.214+0000 7f90997fa700 1 -- 172.21.15.7:0/3473869880 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90840055a0 con 0x7f909c0b3d10 2024-09-13T18:54:48.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.214+0000 7f90a1ed8700 1 -- 172.21.15.7:0/3473869880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909c0b6890 con 0x7f909c0b3d10 2024-09-13T18:54:48.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.214+0000 7f90a1ed8700 1 -- 172.21.15.7:0/3473869880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f909c0cfc20 con 0x7f909c0b3d10 2024-09-13T18:54:48.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.214+0000 7f90997fa700 1 -- 172.21.15.7:0/3473869880 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f90840038e0 con 0x7f909c0b3d10 2024-09-13T18:54:48.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.214+0000 7f90997fa700 1 --2- 172.21.15.7:0/3473869880 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f907c066730 0x7f907c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.214+0000 7f90997fa700 1 -- 172.21.15.7:0/3473869880 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9084083560 con 0x7f909c0b3d10 2024-09-13T18:54:48.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.214+0000 7f909bfff700 1 --2- 172.21.15.7:0/3473869880 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f907c066730 0x7f907c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.218+0000 7f909bfff700 1 --2- 172.21.15.7:0/3473869880 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f907c066730 0x7f907c068bf0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f909c0d2b40 tx=0x7f9094011000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:48.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.226+0000 7f90997fa700 1 -- 172.21.15.7:0/3473869880 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9084050870 con 0x7f909c0b3d10 2024-09-13T18:54:48.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.402+0000 7f90a1ed8700 1 -- 172.21.15.7:0/3473869880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f909c06b9a0 con 0x7f909c0b3d10 2024-09-13T18:54:48.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.406+0000 7f90997fa700 1 -- 172.21.15.7:0/3473869880 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16592 (secure 0 0 0) 0x7f9084053e90 con 0x7f909c0b3d10 2024-09-13T18:54:48.412 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:48.412 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":56,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:54:17.402890+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:54:07.330160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T18:54:07.981407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/3168373873":"2024-09-14T18:54:17.388088+0000","172.21.15.7:6834/3168373873":"2024-09-14T18:54:17.388088+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:54:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.406+0000 7f90a1ed8700 1 -- 172.21.15.7:0/3473869880 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f907c066730 msgr2=0x7f907c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.406+0000 7f90a1ed8700 1 --2- 172.21.15.7:0/3473869880 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f907c066730 0x7f907c068bf0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f909c0d2b40 tx=0x7f9094011000 comp rx=0 tx=0).stop 2024-09-13T18:54:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.406+0000 7f90a1ed8700 1 -- 172.21.15.7:0/3473869880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f909c0b3d10 msgr2=0x7f909c155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.406+0000 7f90a1ed8700 1 --2- 172.21.15.7:0/3473869880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f909c0b3d10 0x7f909c155a10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f908400c500 tx=0x7f908400c810 comp rx=0 tx=0).stop 2024-09-13T18:54:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.410+0000 7f90a1ed8700 1 -- 172.21.15.7:0/3473869880 shutdown_connections 2024-09-13T18:54:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.410+0000 7f90a1ed8700 1 --2- 172.21.15.7:0/3473869880 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f907c066730 0x7f907c068bf0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.410+0000 7f90a1ed8700 1 --2- 172.21.15.7:0/3473869880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f909c0d1810 0x7f909c0b7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.410+0000 7f90a1ed8700 1 --2- 172.21.15.7:0/3473869880 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f909c0d1430 0x7f909c0b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.410+0000 7f90a1ed8700 1 --2- 172.21.15.7:0/3473869880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f909c0b3d10 0x7f909c155a10 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.410+0000 7f90a1ed8700 1 -- 172.21.15.7:0/3473869880 >> 172.21.15.7:0/3473869880 conn(0x7f909c0d2be0 msgr2=0x7f909c0d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.410+0000 7f90a1ed8700 1 -- 172.21.15.7:0/3473869880 shutdown_connections 2024-09-13T18:54:48.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.410+0000 7f90a1ed8700 1 -- 172.21.15.7:0/3473869880 wait complete. 2024-09-13T18:54:48.428 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T18:54:48.429 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:48.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d6759e700 1 Processor -- start 2024-09-13T18:54:48.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d6759e700 1 -- start start 2024-09-13T18:54:48.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d6759e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f0d68161010 0x7f0d680b4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d6759e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d6806aa70 con 0x7f0d680cff40 2024-09-13T18:54:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d6759e700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d680b23b0 con 0x7f0d6806d960 2024-09-13T18:54:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d6759e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d680b2c10 con 0x7f0d68161010 2024-09-13T18:54:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d5ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0d680cff40 0x7f0d6806a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51506/0 (socket says 172.21.15.7:51506) 2024-09-13T18:54:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d5ffff700 1 -- 172.21.15.7:0/2305283744 learned_addr learned my addr 172.21.15.7:0/2305283744 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d6659c700 1 --2- 172.21.15.7:0/2305283744 >> v2:172.21.15.171:3300/0 conn(0x7f0d68161010 0x7f0d680b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d65d9b700 1 -- 172.21.15.7:0/2305283744 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716064050 0 0) 0x7f0d6806aa70 con 0x7f0d680cff40 2024-09-13T18:54:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d65d9b700 1 -- 172.21.15.7:0/2305283744 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d540035d0 con 0x7f0d680cff40 2024-09-13T18:54:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d65d9b700 1 -- 172.21.15.7:0/2305283744 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1666897622 0 0) 0x7f0d540035d0 con 0x7f0d680cff40 2024-09-13T18:54:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d65d9b700 1 -- 172.21.15.7:0/2305283744 >> v2:172.21.15.171:3300/0 conn(0x7f0d68161010 msgr2=0x7f0d680b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d65d9b700 1 --2- 172.21.15.7:0/2305283744 >> v2:172.21.15.171:3300/0 conn(0x7f0d68161010 0x7f0d680b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d65d9b700 1 -- 172.21.15.7:0/2305283744 >> v1:172.21.15.171:6789/0 conn(0x7f0d6806d960 legacy=0x7f0d68161a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d65d9b700 1 -- 172.21.15.7:0/2305283744 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d681658c0 con 0x7f0d680cff40 2024-09-13T18:54:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d65d9b700 1 -- 172.21.15.7:0/2305283744 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7f0d50001670 con 0x7f0d680cff40 2024-09-13T18:54:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d65d9b700 1 -- 172.21.15.7:0/2305283744 >> v1:172.21.15.7:6789/0 conn(0x7f0d680cff40 legacy=0x7f0d6806a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d65d9b700 1 --2- 172.21.15.7:0/2305283744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d54003a20 0x7f0d54003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d65d9b700 1 --2- 172.21.15.7:0/2305283744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d54004870 0x7f0d54004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d65d9b700 1 --2- 172.21.15.7:0/2305283744 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0d54005230 0x7f0d5400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d65d9b700 1 -- 172.21.15.7:0/2305283744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d680b2c10 con 0x7f0d54004870 2024-09-13T18:54:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d65d9b700 1 -- 172.21.15.7:0/2305283744 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5400c0d0 con 0x7f0d54003a20 2024-09-13T18:54:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d65d9b700 1 -- 172.21.15.7:0/2305283744 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5400c290 con 0x7f0d54005230 2024-09-13T18:54:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d6659c700 1 --2- 172.21.15.7:0/2305283744 >> v2:172.21.15.171:3300/0 conn(0x7f0d68161010 0x7f0d680b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d5ffff700 1 --2- 172.21.15.7:0/2305283744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d54004870 0x7f0d54004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d66d9d700 1 --2- 172.21.15.7:0/2305283744 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0d54005230 0x7f0d5400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d6659c700 1 --2- 172.21.15.7:0/2305283744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d54003a20 0x7f0d54003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d5ffff700 1 -- 172.21.15.7:0/2305283744 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0d54005230 msgr2=0x7f0d5400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d5ffff700 1 --2- 172.21.15.7:0/2305283744 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0d54005230 0x7f0d5400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d5ffff700 1 -- 172.21.15.7:0/2305283744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d54003a20 msgr2=0x7f0d54003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d5ffff700 1 --2- 172.21.15.7:0/2305283744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d54003a20 0x7f0d54003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d5ffff700 1 -- 172.21.15.7:0/2305283744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d540035d0 con 0x7f0d54004870 2024-09-13T18:54:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d66d9d700 1 --2- 172.21.15.7:0/2305283744 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0d54005230 0x7f0d5400ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d6659c700 1 --2- 172.21.15.7:0/2305283744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d54003a20 0x7f0d54003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d66d9d700 1 -- 172.21.15.7:0/2305283744 reap_dead start 2024-09-13T18:54:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d5ffff700 1 --2- 172.21.15.7:0/2305283744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d54004870 0x7f0d54004cf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000c320 tx=0x7f0d540058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7803f094b5730ba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d65d9b700 1 -- 172.21.15.7:0/2305283744 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50007c30 con 0x7f0d54004870 2024-09-13T18:54:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d65d9b700 1 -- 172.21.15.7:0/2305283744 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5000ce80 con 0x7f0d54004870 2024-09-13T18:54:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.538+0000 7f0d65d9b700 1 -- 172.21.15.7:0/2305283744 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50005bc0 con 0x7f0d54004870 2024-09-13T18:54:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.542+0000 7f0d6759e700 1 -- 172.21.15.7:0/2305283744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d54004870 msgr2=0x7f0d54004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.542+0000 7f0d6759e700 1 --2- 172.21.15.7:0/2305283744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d54004870 0x7f0d54004cf0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000c320 tx=0x7f0d540058a0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.542+0000 7f0d6759e700 1 -- 172.21.15.7:0/2305283744 shutdown_connections 2024-09-13T18:54:48.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.542+0000 7f0d6759e700 1 --2- 172.21.15.7:0/2305283744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d54004870 0x7f0d54004cf0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.542+0000 7f0d6759e700 1 -- 172.21.15.7:0/2305283744 >> 172.21.15.7:0/2305283744 conn(0x7f0d680cf300 msgr2=0x7f0d680cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:48.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.542+0000 7f0d6759e700 1 -- 172.21.15.7:0/2305283744 shutdown_connections 2024-09-13T18:54:48.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.542+0000 7f0d6759e700 1 -- 172.21.15.7:0/2305283744 wait complete. 2024-09-13T18:54:48.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.542+0000 7f0d6759e700 1 Processor -- start 2024-09-13T18:54:48.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.542+0000 7f0d6759e700 1 -- start start 2024-09-13T18:54:48.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.542+0000 7f0d6759e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0d54004870 0x7f0d681597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.542+0000 7f0d6759e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d680a9460 0x7f0d680adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.542+0000 7f0d6759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d680ae140 0x7f0d68163f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.542+0000 7f0d6759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d680a8450 con 0x7f0d680ae140 2024-09-13T18:54:48.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.542+0000 7f0d6759e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d68159ce0 con 0x7f0d680a9460 2024-09-13T18:54:48.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.542+0000 7f0d6759e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d680a86a0 con 0x7f0d54004870 2024-09-13T18:54:48.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.546+0000 7f0d66d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d680ae140 0x7f0d68163f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.546+0000 7f0d6659c700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0d54004870 0x7f0d681597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.546+0000 7f0d66d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d680ae140 0x7f0d68163f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40396/0 (socket says 172.21.15.7:40396) 2024-09-13T18:54:48.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.546+0000 7f0d66d9d700 1 -- 172.21.15.7:0/3489407212 learned_addr learned my addr 172.21.15.7:0/3489407212 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:48.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.546+0000 7f0d5ffff700 1 --2- 172.21.15.7:0/3489407212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d680a9460 0x7f0d680adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.546+0000 7f0d6659c700 1 -- 172.21.15.7:0/3489407212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d680a9460 msgr2=0x7f0d680adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.546+0000 7f0d6659c700 1 --2- 172.21.15.7:0/3489407212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d680a9460 0x7f0d680adc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.546+0000 7f0d6659c700 1 -- 172.21.15.7:0/3489407212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d680ae140 msgr2=0x7f0d68163f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.546+0000 7f0d6659c700 1 --2- 172.21.15.7:0/3489407212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d680ae140 0x7f0d68163f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.546+0000 7f0d6659c700 1 -- 172.21.15.7:0/3489407212 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d680a8980 con 0x7f0d54004870 2024-09-13T18:54:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.546+0000 7f0d66d9d700 1 --2- 172.21.15.7:0/3489407212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d680ae140 0x7f0d68163f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.546+0000 7f0d6659c700 1 --2- 172.21.15.7:0/3489407212 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0d54004870 0x7f0d681597a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0d58011900 tx=0x7f0d58011c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.546+0000 7f0d5effd700 1 -- 172.21.15.7:0/3489407212 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d58004440 con 0x7f0d54004870 2024-09-13T18:54:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.546+0000 7f0d5effd700 1 -- 172.21.15.7:0/3489407212 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d580045a0 con 0x7f0d54004870 2024-09-13T18:54:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.546+0000 7f0d6759e700 1 -- 172.21.15.7:0/3489407212 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d680a8c60 con 0x7f0d54004870 2024-09-13T18:54:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.546+0000 7f0d5effd700 1 -- 172.21.15.7:0/3489407212 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d580038c0 con 0x7f0d54004870 2024-09-13T18:54:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.546+0000 7f0d6759e700 1 -- 172.21.15.7:0/3489407212 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d680a98e0 con 0x7f0d54004870 2024-09-13T18:54:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.546+0000 7f0d6759e700 1 -- 172.21.15.7:0/3489407212 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d68164d40 con 0x7f0d54004870 2024-09-13T18:54:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.550+0000 7f0d5effd700 1 -- 172.21.15.7:0/3489407212 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f0d58003a20 con 0x7f0d54004870 2024-09-13T18:54:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.550+0000 7f0d5effd700 1 --2- 172.21.15.7:0/3489407212 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0d48066730 0x7f0d48068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.550+0000 7f0d5effd700 1 -- 172.21.15.7:0/3489407212 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0d580843d0 con 0x7f0d54004870 2024-09-13T18:54:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.550+0000 7f0d5ffff700 1 --2- 172.21.15.7:0/3489407212 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0d48066730 0x7f0d48068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.550+0000 7f0d5ffff700 1 --2- 172.21.15.7:0/3489407212 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0d48066730 0x7f0d48068bf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0d68163d10 tx=0x7f0d50007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.558+0000 7f0d5effd700 1 -- 172.21.15.7:0/3489407212 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0d58051770 con 0x7f0d54004870 2024-09-13T18:54:48.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.770+0000 7f0d6759e700 1 -- 172.21.15.7:0/3489407212 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d680b4010 con 0x7f0d54004870 2024-09-13T18:54:48.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.770+0000 7f0d5effd700 1 -- 172.21.15.7:0/3489407212 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f0d58054d90 con 0x7f0d54004870 2024-09-13T18:54:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:48.776 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1829974483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1829974483},{"type":"v1","addr":"172.21.15.171:6835","nonce":1829974483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1948505526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1948505526},{"type":"v1","addr":"172.21.15.171:6837","nonce":1948505526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3617964311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3617964311},{"type":"v1","addr":"172.21.15.7:6833","nonce":3617964311}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:21.419627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/3768750832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3768750832},{"type":"v1","addr":"172.21.15.7:6835","nonce":3768750832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.774+0000 7f0d6759e700 1 -- 172.21.15.7:0/3489407212 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0d48066730 msgr2=0x7f0d48068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.774+0000 7f0d6759e700 1 --2- 172.21.15.7:0/3489407212 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0d48066730 0x7f0d48068bf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0d68163d10 tx=0x7f0d50007000 comp rx=0 tx=0).stop 2024-09-13T18:54:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.774+0000 7f0d6759e700 1 -- 172.21.15.7:0/3489407212 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0d54004870 msgr2=0x7f0d681597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.774+0000 7f0d6759e700 1 --2- 172.21.15.7:0/3489407212 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0d54004870 0x7f0d681597a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0d58011900 tx=0x7f0d58011c10 comp rx=0 tx=0).stop 2024-09-13T18:54:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.774+0000 7f0d6759e700 1 -- 172.21.15.7:0/3489407212 shutdown_connections 2024-09-13T18:54:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.774+0000 7f0d6759e700 1 --2- 172.21.15.7:0/3489407212 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0d48066730 0x7f0d48068bf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.774+0000 7f0d6759e700 1 --2- 172.21.15.7:0/3489407212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d680ae140 0x7f0d68163f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.774+0000 7f0d6759e700 1 --2- 172.21.15.7:0/3489407212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0d680a9460 0x7f0d680adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.774+0000 7f0d6759e700 1 --2- 172.21.15.7:0/3489407212 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0d54004870 0x7f0d681597a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.774+0000 7f0d6759e700 1 -- 172.21.15.7:0/3489407212 >> 172.21.15.7:0/3489407212 conn(0x7f0d680cf300 msgr2=0x7f0d680b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:48.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.774+0000 7f0d6759e700 1 -- 172.21.15.7:0/3489407212 shutdown_connections 2024-09-13T18:54:48.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.774+0000 7f0d6759e700 1 -- 172.21.15.7:0/3489407212 wait complete. 2024-09-13T18:54:48.780 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-09-13T18:54:48.792 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:54:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.898+0000 7effb21b3700 1 Processor -- start 2024-09-13T18:54:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.898+0000 7effb21b3700 1 -- start start 2024-09-13T18:54:48.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.898+0000 7effb21b3700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7effac1599b0 0x7effac0ada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.898+0000 7effb21b3700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effac0cabf0 0x7effac070740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.898+0000 7effb21b3700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effac0b1aa0 con 0x7effac070c80 2024-09-13T18:54:48.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.898+0000 7effb21b3700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effac06c3c0 con 0x7effac1599b0 2024-09-13T18:54:48.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.898+0000 7effb21b3700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effac071130 con 0x7effac0cabf0 2024-09-13T18:54:48.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.898+0000 7effb19b2700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7effac070c80 0x7effac15d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51522/0 (socket says 172.21.15.7:51522) 2024-09-13T18:54:48.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.898+0000 7effb19b2700 1 -- 172.21.15.7:0/1300597195 learned_addr learned my addr 172.21.15.7:0/1300597195 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:48.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.898+0000 7effb09b0700 1 --2- 172.21.15.7:0/1300597195 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effac0cabf0 0x7effac070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.898+0000 7effb11b1700 1 --2- 172.21.15.7:0/1300597195 >> v2:172.21.15.171:3300/0 conn(0x7effac1599b0 0x7effac0ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.898+0000 7effa3fff700 1 -- 172.21.15.7:0/1300597195 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265649642 0 0) 0x7effac0b1aa0 con 0x7effac070c80 2024-09-13T18:54:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.898+0000 7effa3fff700 1 -- 172.21.15.7:0/1300597195 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff980035d0 con 0x7effac070c80 2024-09-13T18:54:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.898+0000 7effb09b0700 1 -- 172.21.15.7:0/1300597195 >> v2:172.21.15.171:3300/0 conn(0x7effac1599b0 msgr2=0x7effac0ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.898+0000 7effb09b0700 1 --2- 172.21.15.7:0/1300597195 >> v2:172.21.15.171:3300/0 conn(0x7effac1599b0 0x7effac0ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.898+0000 7effb09b0700 1 -- 172.21.15.7:0/1300597195 >> v1:172.21.15.7:6789/0 conn(0x7effac070c80 legacy=0x7effac15d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.898+0000 7effb09b0700 1 -- 172.21.15.7:0/1300597195 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effac06be40 con 0x7effac0cabf0 2024-09-13T18:54:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.902+0000 7effb09b0700 1 --2- 172.21.15.7:0/1300597195 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effac0cabf0 0x7effac070740 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7effa8009850 tx=0x7effa8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7f9b58296139cd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.902+0000 7effa3fff700 1 -- 172.21.15.7:0/1300597195 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8011830 con 0x7effac0cabf0 2024-09-13T18:54:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.902+0000 7effa3fff700 1 -- 172.21.15.7:0/1300597195 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa8011990 con 0x7effac0cabf0 2024-09-13T18:54:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.902+0000 7effa3fff700 1 -- 172.21.15.7:0/1300597195 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8011d40 con 0x7effac0cabf0 2024-09-13T18:54:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.902+0000 7effb21b3700 1 -- 172.21.15.7:0/1300597195 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effac0cabf0 msgr2=0x7effac070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.902+0000 7effb21b3700 1 --2- 172.21.15.7:0/1300597195 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effac0cabf0 0x7effac070740 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7effa8009850 tx=0x7effa8009b60 comp rx=0 tx=0).stop 2024-09-13T18:54:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.902+0000 7effb21b3700 1 -- 172.21.15.7:0/1300597195 shutdown_connections 2024-09-13T18:54:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.902+0000 7effb21b3700 1 --2- 172.21.15.7:0/1300597195 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effac0cabf0 0x7effac070740 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.902+0000 7effb21b3700 1 --2- 172.21.15.7:0/1300597195 >> v2:172.21.15.171:3300/0 conn(0x7effac1599b0 0x7effac0ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.902+0000 7effb21b3700 1 -- 172.21.15.7:0/1300597195 >> 172.21.15.7:0/1300597195 conn(0x7effac0b0570 msgr2=0x7effac0b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.902+0000 7effb21b3700 1 -- 172.21.15.7:0/1300597195 shutdown_connections 2024-09-13T18:54:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.902+0000 7effb21b3700 1 -- 172.21.15.7:0/1300597195 wait complete. 2024-09-13T18:54:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.902+0000 7effb21b3700 1 Processor -- start 2024-09-13T18:54:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.902+0000 7effb21b3700 1 -- start start 2024-09-13T18:54:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.902+0000 7effb21b3700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effac070c80 0x7effac14d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.902+0000 7effb21b3700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effac0cabf0 0x7effac1353c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.902+0000 7effb21b3700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effac1599b0 0x7effac146360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.902+0000 7effb21b3700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effac149350 con 0x7effac0cabf0 2024-09-13T18:54:48.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effb21b3700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effac1468a0 con 0x7effac070c80 2024-09-13T18:54:48.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effb21b3700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effac15cc30 con 0x7effac1599b0 2024-09-13T18:54:48.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effb09b0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effac0cabf0 0x7effac1353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effb09b0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effac0cabf0 0x7effac1353c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40408/0 (socket says 172.21.15.7:40408) 2024-09-13T18:54:48.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effb09b0700 1 -- 172.21.15.7:0/134691923 learned_addr learned my addr 172.21.15.7:0/134691923 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:48.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effb11b1700 1 --2- 172.21.15.7:0/134691923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effac070c80 0x7effac14d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effb19b2700 1 --2- 172.21.15.7:0/134691923 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effac1599b0 0x7effac146360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effb09b0700 1 -- 172.21.15.7:0/134691923 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effac1599b0 msgr2=0x7effac146360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effb09b0700 1 --2- 172.21.15.7:0/134691923 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effac1599b0 0x7effac146360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effb09b0700 1 -- 172.21.15.7:0/134691923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effac070c80 msgr2=0x7effac14d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:48.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effb09b0700 1 --2- 172.21.15.7:0/134691923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effac070c80 0x7effac14d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:48.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effb09b0700 1 -- 172.21.15.7:0/134691923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa80094b0 con 0x7effac0cabf0 2024-09-13T18:54:48.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effb09b0700 1 --2- 172.21.15.7:0/134691923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effac0cabf0 0x7effac1353c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7effac15d470 tx=0x7effa8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:48.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effa1ffb700 1 -- 172.21.15.7:0/134691923 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8004300 con 0x7effac0cabf0 2024-09-13T18:54:48.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effa1ffb700 1 -- 172.21.15.7:0/134691923 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa801f070 con 0x7effac0cabf0 2024-09-13T18:54:48.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effb21b3700 1 -- 172.21.15.7:0/134691923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effac15cec0 con 0x7effac0cabf0 2024-09-13T18:54:48.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effa1ffb700 1 -- 172.21.15.7:0/134691923 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa80036c0 con 0x7effac0cabf0 2024-09-13T18:54:48.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effb21b3700 1 -- 172.21.15.7:0/134691923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effac14ef90 con 0x7effac0cabf0 2024-09-13T18:54:48.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effb21b3700 1 -- 172.21.15.7:0/134691923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effac14eb10 con 0x7effac0cabf0 2024-09-13T18:54:48.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.906+0000 7effa1ffb700 1 -- 172.21.15.7:0/134691923 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7effa8021030 con 0x7effac0cabf0 2024-09-13T18:54:48.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.910+0000 7effa1ffb700 1 --2- 172.21.15.7:0/134691923 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7eff90066730 0x7eff90068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:48.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.910+0000 7effa1ffb700 1 -- 172.21.15.7:0/134691923 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7effa80853c0 con 0x7effac0cabf0 2024-09-13T18:54:48.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.910+0000 7effb11b1700 1 --2- 172.21.15.7:0/134691923 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7eff90066730 0x7eff90068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:48.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.910+0000 7effb11b1700 1 --2- 172.21.15.7:0/134691923 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7eff90066730 0x7eff90068bf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009b50 tx=0x7eff9c009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:48.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:48.914+0000 7effa1ffb700 1 -- 172.21.15.7:0/134691923 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7effac14eb10 con 0x7effac0cabf0 2024-09-13T18:54:49.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.126+0000 7effb21b3700 1 -- 172.21.15.7:0/134691923 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effac14eb10 con 0x7effac0cabf0 2024-09-13T18:54:49.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.126+0000 7effa1ffb700 1 -- 172.21.15.7:0/134691923 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4735 (secure 0 0 0) 0x7effac14eb10 con 0x7effac0cabf0 2024-09-13T18:54:49.132 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:49.132 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1829974483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1829974483},{"type":"v1","addr":"172.21.15.171:6835","nonce":1829974483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1948505526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1948505526},{"type":"v1","addr":"172.21.15.171:6837","nonce":1948505526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3617964311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3617964311},{"type":"v1","addr":"172.21.15.7:6833","nonce":3617964311}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T18:54:09.004150+0000","modified":"2024-09-13T18:54:21.419627+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5910},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5910":{"gid":5910,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":5,"addr":"172.21.15.7:6835/3768750832","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":3768750832},{"type":"v1","addr":"172.21.15.7:6835","nonce":3768750832}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T18:54:49.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.126+0000 7effb21b3700 1 -- 172.21.15.7:0/134691923 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7eff90066730 msgr2=0x7eff90068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:49.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.126+0000 7effb21b3700 1 --2- 172.21.15.7:0/134691923 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7eff90066730 0x7eff90068bf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009b50 tx=0x7eff9c009240 comp rx=0 tx=0).stop 2024-09-13T18:54:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.126+0000 7effb21b3700 1 -- 172.21.15.7:0/134691923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effac0cabf0 msgr2=0x7effac1353c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.126+0000 7effb21b3700 1 --2- 172.21.15.7:0/134691923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effac0cabf0 0x7effac1353c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7effac15d470 tx=0x7effa8004770 comp rx=0 tx=0).stop 2024-09-13T18:54:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.126+0000 7effb21b3700 1 -- 172.21.15.7:0/134691923 shutdown_connections 2024-09-13T18:54:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.126+0000 7effb21b3700 1 --2- 172.21.15.7:0/134691923 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7eff90066730 0x7eff90068bf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.126+0000 7effb21b3700 1 --2- 172.21.15.7:0/134691923 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7effac1599b0 0x7effac146360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.126+0000 7effb21b3700 1 --2- 172.21.15.7:0/134691923 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effac0cabf0 0x7effac1353c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.126+0000 7effb21b3700 1 --2- 172.21.15.7:0/134691923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effac070c80 0x7effac14d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.126+0000 7effb21b3700 1 -- 172.21.15.7:0/134691923 >> 172.21.15.7:0/134691923 conn(0x7effac0b0570 msgr2=0x7effac0b2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.130+0000 7effb21b3700 1 -- 172.21.15.7:0/134691923 shutdown_connections 2024-09-13T18:54:49.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.130+0000 7effb21b3700 1 -- 172.21.15.7:0/134691923 wait complete. 2024-09-13T18:54:49.136 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-09-13T18:54:49.149 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:54:49.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9d21c6700 1 Processor -- start 2024-09-13T18:54:49.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9d21c6700 1 -- start start 2024-09-13T18:54:49.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9d21c6700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd9cc159a70 0x7fd9cc0ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9d21c6700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd9cc0ca860 0x7fd9cc0713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9d21c6700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9cc0718e0 con 0x7fd9cc070f60 2024-09-13T18:54:49.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9d21c6700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9cc0cb0f0 con 0x7fd9cc159a70 2024-09-13T18:54:49.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9d21c6700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc0cae10 con 0x7fd9cc0ca860 2024-09-13T18:54:49.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9caffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd9cc070f60 0x7fd9cc0ca150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51526/0 (socket says 172.21.15.7:51526) 2024-09-13T18:54:49.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9caffd700 1 -- 172.21.15.7:0/2927425950 learned_addr learned my addr 172.21.15.7:0/2927425950 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:49.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9cb7fe700 1 --2- 172.21.15.7:0/2927425950 >> v2:172.21.15.171:3300/0 conn(0x7fd9cc159a70 0x7fd9cc0ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9cbfff700 1 --2- 172.21.15.7:0/2927425950 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd9cc0ca860 0x7fd9cc0713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9ca7fc700 1 -- 172.21.15.7:0/2927425950 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4257779183 0 0) 0x7fd9cc0718e0 con 0x7fd9cc070f60 2024-09-13T18:54:49.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9ca7fc700 1 -- 172.21.15.7:0/2927425950 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9b80035d0 con 0x7fd9cc070f60 2024-09-13T18:54:49.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9cb7fe700 1 -- 172.21.15.7:0/2927425950 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd9cc0ca860 msgr2=0x7fd9cc0713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:49.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9cb7fe700 1 --2- 172.21.15.7:0/2927425950 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd9cc0ca860 0x7fd9cc0713a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9cb7fe700 1 -- 172.21.15.7:0/2927425950 >> v1:172.21.15.7:6789/0 conn(0x7fd9cc070f60 legacy=0x7fd9cc0ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:49.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9cb7fe700 1 -- 172.21.15.7:0/2927425950 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc0cd800 con 0x7fd9cc159a70 2024-09-13T18:54:49.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9cb7fe700 1 --2- 172.21.15.7:0/2927425950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9cc159a70 0x7fd9cc0ce2e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009850 tx=0x7fd9b4009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15600661d10655e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:49.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9ca7fc700 1 -- 172.21.15.7:0/2927425950 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4011830 con 0x7fd9cc159a70 2024-09-13T18:54:49.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9ca7fc700 1 -- 172.21.15.7:0/2927425950 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b4011990 con 0x7fd9cc159a70 2024-09-13T18:54:49.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.258+0000 7fd9ca7fc700 1 -- 172.21.15.7:0/2927425950 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4011d40 con 0x7fd9cc159a70 2024-09-13T18:54:49.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.262+0000 7fd9d21c6700 1 -- 172.21.15.7:0/2927425950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9cc159a70 msgr2=0x7fd9cc0ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:49.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.262+0000 7fd9d21c6700 1 --2- 172.21.15.7:0/2927425950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9cc159a70 0x7fd9cc0ce2e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009850 tx=0x7fd9b4009b60 comp rx=0 tx=0).stop 2024-09-13T18:54:49.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.262+0000 7fd9d21c6700 1 -- 172.21.15.7:0/2927425950 shutdown_connections 2024-09-13T18:54:49.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.262+0000 7fd9d21c6700 1 --2- 172.21.15.7:0/2927425950 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd9cc0ca860 0x7fd9cc0713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.262+0000 7fd9d21c6700 1 --2- 172.21.15.7:0/2927425950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9cc159a70 0x7fd9cc0ce2e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.262+0000 7fd9d21c6700 1 -- 172.21.15.7:0/2927425950 >> 172.21.15.7:0/2927425950 conn(0x7fd9cc0cfb00 msgr2=0x7fd9cc0cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:49.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.262+0000 7fd9d21c6700 1 -- 172.21.15.7:0/2927425950 shutdown_connections 2024-09-13T18:54:49.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.262+0000 7fd9d21c6700 1 -- 172.21.15.7:0/2927425950 wait complete. 2024-09-13T18:54:49.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.262+0000 7fd9d21c6700 1 Processor -- start 2024-09-13T18:54:49.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.262+0000 7fd9d21c6700 1 -- start start 2024-09-13T18:54:49.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.262+0000 7fd9d21c6700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd9cc070f60 0x7fd9cc14a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.262+0000 7fd9d21c6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9cc0ca860 0x7fd9cc14b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.262+0000 7fd9d21c6700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9cc159a70 0x7fd9cc14bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9d21c6700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc0b3730 con 0x7fd9cc0ca860 2024-09-13T18:54:49.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9d21c6700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc14ed10 con 0x7fd9cc159a70 2024-09-13T18:54:49.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9d21c6700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc14eff0 con 0x7fd9cc070f60 2024-09-13T18:54:49.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9caffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9cc0ca860 0x7fd9cc14b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9caffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9cc0ca860 0x7fd9cc14b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40412/0 (socket says 172.21.15.7:40412) 2024-09-13T18:54:49.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9caffd700 1 -- 172.21.15.7:0/277948689 learned_addr learned my addr 172.21.15.7:0/277948689 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:49.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9cbfff700 1 --2- 172.21.15.7:0/277948689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9cc159a70 0x7fd9cc14bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9cb7fe700 1 --2- 172.21.15.7:0/277948689 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd9cc070f60 0x7fd9cc14a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9caffd700 1 -- 172.21.15.7:0/277948689 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd9cc070f60 msgr2=0x7fd9cc14a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:49.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9caffd700 1 --2- 172.21.15.7:0/277948689 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd9cc070f60 0x7fd9cc14a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9caffd700 1 -- 172.21.15.7:0/277948689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9cc159a70 msgr2=0x7fd9cc14bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:49.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9caffd700 1 --2- 172.21.15.7:0/277948689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9cc159a70 0x7fd9cc14bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9caffd700 1 -- 172.21.15.7:0/277948689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b40094b0 con 0x7fd9cc0ca860 2024-09-13T18:54:49.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9caffd700 1 --2- 172.21.15.7:0/277948689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9cc0ca860 0x7fd9cc14b790 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400cbc0 tx=0x7fd9c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:49.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9c3fff700 1 -- 172.21.15.7:0/277948689 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c400a770 con 0x7fd9cc0ca860 2024-09-13T18:54:49.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9c3fff700 1 -- 172.21.15.7:0/277948689 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c400a8d0 con 0x7fd9cc0ca860 2024-09-13T18:54:49.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9d21c6700 1 -- 172.21.15.7:0/277948689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9cc14f2d0 con 0x7fd9cc0ca860 2024-09-13T18:54:49.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9c3fff700 1 -- 172.21.15.7:0/277948689 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c400ac60 con 0x7fd9cc0ca860 2024-09-13T18:54:49.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9d21c6700 1 -- 172.21.15.7:0/277948689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9cc14f890 con 0x7fd9cc0ca860 2024-09-13T18:54:49.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9d21c6700 1 -- 172.21.15.7:0/277948689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9cc14fa80 con 0x7fd9cc0ca860 2024-09-13T18:54:49.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.266+0000 7fd9c3fff700 1 -- 172.21.15.7:0/277948689 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd9c401b3b0 con 0x7fd9cc0ca860 2024-09-13T18:54:49.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.270+0000 7fd9c3fff700 1 --2- 172.21.15.7:0/277948689 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd9ac066730 0x7fd9ac068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.270+0000 7fd9c3fff700 1 -- 172.21.15.7:0/277948689 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd9c4083c40 con 0x7fd9cc0ca860 2024-09-13T18:54:49.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.270+0000 7fd9cb7fe700 1 --2- 172.21.15.7:0/277948689 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd9ac066730 0x7fd9ac068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.270+0000 7fd9cb7fe700 1 --2- 172.21.15.7:0/277948689 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd9ac066730 0x7fd9ac068bf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40090b0 tx=0x7fd9b401c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:49.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.278+0000 7fd9c3fff700 1 -- 172.21.15.7:0/277948689 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd9c4050f50 con 0x7fd9cc0ca860 2024-09-13T18:54:49.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.454+0000 7fd9d21c6700 1 -- 172.21.15.7:0/277948689 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9cc0ce2e0 con 0x7fd9cc0ca860 2024-09-13T18:54:49.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.454+0000 7fd9c3fff700 1 -- 172.21.15.7:0/277948689 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16592 (secure 0 0 0) 0x7fd9c4017090 con 0x7fd9cc0ca860 2024-09-13T18:54:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:49.462 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":56,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:54:17.402890+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T18:54:07.330160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T18:54:07.981407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/3168373873":"2024-09-14T18:54:17.388088+0000","172.21.15.7:6834/3168373873":"2024-09-14T18:54:17.388088+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:54:49.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.458+0000 7fd9d21c6700 1 -- 172.21.15.7:0/277948689 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd9ac066730 msgr2=0x7fd9ac068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:49.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.458+0000 7fd9d21c6700 1 --2- 172.21.15.7:0/277948689 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd9ac066730 0x7fd9ac068bf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40090b0 tx=0x7fd9b401c040 comp rx=0 tx=0).stop 2024-09-13T18:54:49.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.458+0000 7fd9d21c6700 1 -- 172.21.15.7:0/277948689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9cc0ca860 msgr2=0x7fd9cc14b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:49.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.458+0000 7fd9d21c6700 1 --2- 172.21.15.7:0/277948689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9cc0ca860 0x7fd9cc14b790 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400cbc0 tx=0x7fd9c400cbf0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.458+0000 7fd9d21c6700 1 -- 172.21.15.7:0/277948689 shutdown_connections 2024-09-13T18:54:49.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.458+0000 7fd9d21c6700 1 --2- 172.21.15.7:0/277948689 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd9ac066730 0x7fd9ac068bf0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.458+0000 7fd9d21c6700 1 --2- 172.21.15.7:0/277948689 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd9cc159a70 0x7fd9cc14bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.458+0000 7fd9d21c6700 1 --2- 172.21.15.7:0/277948689 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9cc0ca860 0x7fd9cc14b790 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.458+0000 7fd9d21c6700 1 --2- 172.21.15.7:0/277948689 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd9cc070f60 0x7fd9cc14a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.458+0000 7fd9d21c6700 1 -- 172.21.15.7:0/277948689 >> 172.21.15.7:0/277948689 conn(0x7fd9cc0cfb00 msgr2=0x7fd9cc14aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:49.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.458+0000 7fd9d21c6700 1 -- 172.21.15.7:0/277948689 shutdown_connections 2024-09-13T18:54:49.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.458+0000 7fd9d21c6700 1 -- 172.21.15.7:0/277948689 wait complete. 2024-09-13T18:54:49.477 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T18:54:49.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.586+0000 7ff6ee7c9700 1 Processor -- start 2024-09-13T18:54:49.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.586+0000 7ff6ee7c9700 1 -- start start 2024-09-13T18:54:49.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ee7c9700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff6e81606a0 0x7ff6e80d3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ee7c9700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff6e80d17a0 0x7ff6e80ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ee7c9700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7ff6e8160b10 0x7ff6e8160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ee7c9700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e8161500 con 0x7ff6e80d17a0 2024-09-13T18:54:49.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ee7c9700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e80ced20 con 0x7ff6e8160b10 2024-09-13T18:54:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ee7c9700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80d1ca0 con 0x7ff6e81606a0 2024-09-13T18:54:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ecfc6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff6e80d17a0 0x7ff6e80ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ecfc6700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff6e80d17a0 0x7ff6e80ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40418/0 (socket says 172.21.15.7:40418) 2024-09-13T18:54:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ecfc6700 1 -- 172.21.15.7:0/1420448342 learned_addr learned my addr 172.21.15.7:0/1420448342 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ed7c7700 1 --2- 172.21.15.7:0/1420448342 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff6e81606a0 0x7ff6e80d3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6edfc8700 1 --2- 172.21.15.7:0/1420448342 >> v2:172.21.15.171:3300/0 conn(0x7ff6e8160b10 0x7ff6e8160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ecfc6700 1 -- 172.21.15.7:0/1420448342 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff6e81606a0 msgr2=0x7ff6e80d3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ecfc6700 1 --2- 172.21.15.7:0/1420448342 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff6e81606a0 0x7ff6e80d3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ecfc6700 1 -- 172.21.15.7:0/1420448342 >> v2:172.21.15.171:3300/0 conn(0x7ff6e8160b10 msgr2=0x7ff6e8160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ecfc6700 1 --2- 172.21.15.7:0/1420448342 >> v2:172.21.15.171:3300/0 conn(0x7ff6e8160b10 0x7ff6e8160fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ecfc6700 1 -- 172.21.15.7:0/1420448342 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e80cdd00 con 0x7ff6e80d17a0 2024-09-13T18:54:49.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6edfc8700 1 --2- 172.21.15.7:0/1420448342 >> v2:172.21.15.171:3300/0 conn(0x7ff6e8160b10 0x7ff6e8160fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:49.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ed7c7700 1 --2- 172.21.15.7:0/1420448342 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff6e81606a0 0x7ff6e80d3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:49.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ecfc6700 1 --2- 172.21.15.7:0/1420448342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e80d17a0 0x7ff6e80ce7e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff6e808b990 tx=0x7ff6e4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbcde4f9e5d4e69 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:49.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6dbfff700 1 -- 172.21.15.7:0/1420448342 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400b8a0 con 0x7ff6e80d17a0 2024-09-13T18:54:49.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6dbfff700 1 -- 172.21.15.7:0/1420448342 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e400ba00 con 0x7ff6e80d17a0 2024-09-13T18:54:49.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6dbfff700 1 -- 172.21.15.7:0/1420448342 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400bd90 con 0x7ff6e80d17a0 2024-09-13T18:54:49.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ee7c9700 1 -- 172.21.15.7:0/1420448342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e80d17a0 msgr2=0x7ff6e80ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:49.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ee7c9700 1 --2- 172.21.15.7:0/1420448342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e80d17a0 0x7ff6e80ce7e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff6e808b990 tx=0x7ff6e4009bf0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ee7c9700 1 -- 172.21.15.7:0/1420448342 shutdown_connections 2024-09-13T18:54:49.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ee7c9700 1 --2- 172.21.15.7:0/1420448342 >> v2:172.21.15.171:3300/0 conn(0x7ff6e8160b10 0x7ff6e8160fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ee7c9700 1 --2- 172.21.15.7:0/1420448342 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e80d17a0 0x7ff6e80ce7e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ee7c9700 1 --2- 172.21.15.7:0/1420448342 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff6e81606a0 0x7ff6e80d3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ee7c9700 1 -- 172.21.15.7:0/1420448342 >> 172.21.15.7:0/1420448342 conn(0x7ff6e80d2d10 msgr2=0x7ff6e80d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:49.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ee7c9700 1 -- 172.21.15.7:0/1420448342 shutdown_connections 2024-09-13T18:54:49.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.590+0000 7ff6ee7c9700 1 -- 172.21.15.7:0/1420448342 wait complete. 2024-09-13T18:54:49.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ee7c9700 1 Processor -- start 2024-09-13T18:54:49.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ee7c9700 1 -- start start 2024-09-13T18:54:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ee7c9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e80d17a0 0x7ff6e8156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ee7c9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e81606a0 0x7ff6e80c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ee7c9700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff6e8160b10 0x7ff6e81f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ee7c9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e81566b0 con 0x7ff6e80d17a0 2024-09-13T18:54:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ee7c9700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80c97a0 con 0x7ff6e81606a0 2024-09-13T18:54:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ee7c9700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80b7740 con 0x7ff6e8160b10 2024-09-13T18:54:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ed7c7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e80d17a0 0x7ff6e8156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ed7c7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e80d17a0 0x7ff6e8156170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40426/0 (socket says 172.21.15.7:40426) 2024-09-13T18:54:49.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ed7c7700 1 -- 172.21.15.7:0/3503505700 learned_addr learned my addr 172.21.15.7:0/3503505700 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6edfc8700 1 --2- 172.21.15.7:0/3503505700 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff6e8160b10 0x7ff6e81f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ecfc6700 1 --2- 172.21.15.7:0/3503505700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e81606a0 0x7ff6e80c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ed7c7700 1 -- 172.21.15.7:0/3503505700 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff6e8160b10 msgr2=0x7ff6e81f6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ed7c7700 1 --2- 172.21.15.7:0/3503505700 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff6e8160b10 0x7ff6e81f6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ed7c7700 1 -- 172.21.15.7:0/3503505700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e81606a0 msgr2=0x7ff6e80c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ed7c7700 1 --2- 172.21.15.7:0/3503505700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e81606a0 0x7ff6e80c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ed7c7700 1 -- 172.21.15.7:0/3503505700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e4009570 con 0x7ff6e80d17a0 2024-09-13T18:54:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6edfc8700 1 --2- 172.21.15.7:0/3503505700 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff6e8160b10 0x7ff6e81f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ecfc6700 1 --2- 172.21.15.7:0/3503505700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e81606a0 0x7ff6e80c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ed7c7700 1 --2- 172.21.15.7:0/3503505700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e80d17a0 0x7ff6e8156170 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00d830 tx=0x7ff6dc00dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6d9ffb700 1 -- 172.21.15.7:0/3503505700 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc004030 con 0x7ff6e80d17a0 2024-09-13T18:54:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6d9ffb700 1 -- 172.21.15.7:0/3503505700 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6dc004190 con 0x7ff6e80d17a0 2024-09-13T18:54:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6ee7c9700 1 -- 172.21.15.7:0/3503505700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e80ca320 con 0x7ff6e80d17a0 2024-09-13T18:54:49.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.594+0000 7ff6d9ffb700 1 -- 172.21.15.7:0/3503505700 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc004500 con 0x7ff6e80d17a0 2024-09-13T18:54:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.598+0000 7ff6ee7c9700 1 -- 172.21.15.7:0/3503505700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e80b79e0 con 0x7ff6e80d17a0 2024-09-13T18:54:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.598+0000 7ff6ee7c9700 1 -- 172.21.15.7:0/3503505700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e80b7570 con 0x7ff6e80d17a0 2024-09-13T18:54:49.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.598+0000 7ff6d9ffb700 1 -- 172.21.15.7:0/3503505700 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff6dc0036c0 con 0x7ff6e80d17a0 2024-09-13T18:54:49.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.598+0000 7ff6d9ffb700 1 --2- 172.21.15.7:0/3503505700 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff6cc066730 0x7ff6cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.598+0000 7ff6d9ffb700 1 -- 172.21.15.7:0/3503505700 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff6dc0846f0 con 0x7ff6e80d17a0 2024-09-13T18:54:49.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.598+0000 7ff6ecfc6700 1 --2- 172.21.15.7:0/3503505700 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff6cc066730 0x7ff6cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.602+0000 7ff6ecfc6700 1 --2- 172.21.15.7:0/3503505700 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff6cc066730 0x7ff6cc068bf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff6e808b990 tx=0x7ff6e4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:49.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.606+0000 7ff6d9ffb700 1 -- 172.21.15.7:0/3503505700 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff6dc051a00 con 0x7ff6e80d17a0 2024-09-13T18:54:49.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.818+0000 7ff6ee7c9700 1 -- 172.21.15.7:0/3503505700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff6e80ce7e0 con 0x7ff6e80d17a0 2024-09-13T18:54:49.824 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-13T18:54:49.818+0000 7f28f0ff8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T18:54:49.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.830+0000 7ff6d9ffb700 1 -- 172.21.15.7:0/3503505700 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff6dc00b440 con 0x7ff6e80d17a0 2024-09-13T18:54:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.834+0000 7ff6ee7c9700 1 -- 172.21.15.7:0/3503505700 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff6cc066730 msgr2=0x7ff6cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.834+0000 7ff6ee7c9700 1 --2- 172.21.15.7:0/3503505700 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff6cc066730 0x7ff6cc068bf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff6e808b990 tx=0x7ff6e4008030 comp rx=0 tx=0).stop 2024-09-13T18:54:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.834+0000 7ff6ee7c9700 1 -- 172.21.15.7:0/3503505700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e80d17a0 msgr2=0x7ff6e8156170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.834+0000 7ff6ee7c9700 1 --2- 172.21.15.7:0/3503505700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e80d17a0 0x7ff6e8156170 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00d830 tx=0x7ff6dc00dbf0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.834+0000 7ff6ee7c9700 1 -- 172.21.15.7:0/3503505700 shutdown_connections 2024-09-13T18:54:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.834+0000 7ff6ee7c9700 1 --2- 172.21.15.7:0/3503505700 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff6cc066730 0x7ff6cc068bf0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.834+0000 7ff6ee7c9700 1 --2- 172.21.15.7:0/3503505700 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff6e8160b10 0x7ff6e81f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.834+0000 7ff6ee7c9700 1 --2- 172.21.15.7:0/3503505700 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6e81606a0 0x7ff6e80c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.834+0000 7ff6ee7c9700 1 --2- 172.21.15.7:0/3503505700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e80d17a0 0x7ff6e8156170 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.834+0000 7ff6ee7c9700 1 -- 172.21.15.7:0/3503505700 >> 172.21.15.7:0/3503505700 conn(0x7ff6e80d2d10 msgr2=0x7ff6e806ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:49.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.834+0000 7ff6ee7c9700 1 -- 172.21.15.7:0/3503505700 shutdown_connections 2024-09-13T18:54:49.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.834+0000 7ff6ee7c9700 1 -- 172.21.15.7:0/3503505700 wait complete. 2024-09-13T18:54:49.841 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T18:54:49.841 INFO:tasks.ceph.mds.c.smithi007.stderr: -1607> 2024-09-13T18:54:22.403+0000 7fc20ae24700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T18:54:49.852 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T18:54:49.869 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-13T18:54:49.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.958+0000 7f068eb2e700 1 Processor -- start 2024-09-13T18:54:49.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.958+0000 7f068eb2e700 1 -- start start 2024-09-13T18:54:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068eb2e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0688157530 0x7f068815d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068eb2e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f068815e3b0 con 0x7f068806df10 2024-09-13T18:54:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068eb2e700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f068815ded0 con 0x7f068815dab0 2024-09-13T18:54:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068eb2e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06880b2b00 con 0x7f0688157530 2024-09-13T18:54:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068e32d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f068806df10 0x7f06880b2f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51540/0 (socket says 172.21.15.7:51540) 2024-09-13T18:54:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068e32d700 1 -- 172.21.15.7:0/1473241983 learned_addr learned my addr 172.21.15.7:0/1473241983 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068db2c700 1 --2- 172.21.15.7:0/1473241983 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0688157530 0x7f068815d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 -- 172.21.15.7:0/1473241983 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3319779071 0 0) 0x7f068815e3b0 con 0x7f068806df10 2024-09-13T18:54:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 -- 172.21.15.7:0/1473241983 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06740035d0 con 0x7f068806df10 2024-09-13T18:54:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 -- 172.21.15.7:0/1473241983 <== mon.1 v1:172.21.15.171:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2883712219 0 0) 0x7f068815ded0 con 0x7f068815dab0 2024-09-13T18:54:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 -- 172.21.15.7:0/1473241983 --> v1:172.21.15.171:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f068815e3b0 con 0x7f068815dab0 2024-09-13T18:54:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 -- 172.21.15.7:0/1473241983 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3788588666 0 0) 0x7f06740035d0 con 0x7f068806df10 2024-09-13T18:54:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 -- 172.21.15.7:0/1473241983 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0688157530 msgr2=0x7f068815d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 --2- 172.21.15.7:0/1473241983 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0688157530 0x7f068815d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 -- 172.21.15.7:0/1473241983 >> v1:172.21.15.171:6789/0 conn(0x7f068815dab0 legacy=0x7f068806d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 -- 172.21.15.7:0/1473241983 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068806ec80 con 0x7f068806df10 2024-09-13T18:54:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 -- 172.21.15.7:0/1473241983 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7f06800019f0 con 0x7f068806df10 2024-09-13T18:54:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068db2c700 1 --2- 172.21.15.7:0/1473241983 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0688157530 0x7f068815d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:49.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 -- 172.21.15.7:0/1473241983 >> v1:172.21.15.7:6789/0 conn(0x7f068806df10 legacy=0x7f06880b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:49.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 --2- 172.21.15.7:0/1473241983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0674003be0 0x7f0674003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 --2- 172.21.15.7:0/1473241983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0674004870 0x7f0674004cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 --2- 172.21.15.7:0/1473241983 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0688157530 0x7f0674005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 -- 172.21.15.7:0/1473241983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06880b2b00 con 0x7f0674004870 2024-09-13T18:54:49.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 -- 172.21.15.7:0/1473241983 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067400ba60 con 0x7f0674003be0 2024-09-13T18:54:49.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 -- 172.21.15.7:0/1473241983 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067400bc20 con 0x7f0688157530 2024-09-13T18:54:49.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068d32b700 1 --2- 172.21.15.7:0/1473241983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0674004870 0x7f0674004cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068db2c700 1 --2- 172.21.15.7:0/1473241983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0674003be0 0x7f0674003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068e32d700 1 --2- 172.21.15.7:0/1473241983 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0688157530 0x7f0674005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068d32b700 1 -- 172.21.15.7:0/1473241983 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0688157530 msgr2=0x7f0674005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:49.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068d32b700 1 --2- 172.21.15.7:0/1473241983 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f0688157530 0x7f0674005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068d32b700 1 -- 172.21.15.7:0/1473241983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0674003be0 msgr2=0x7f0674003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:49.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068d32b700 1 --2- 172.21.15.7:0/1473241983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0674003be0 0x7f0674003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068d32b700 1 -- 172.21.15.7:0/1473241983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06840094b0 con 0x7f0674004870 2024-09-13T18:54:49.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068d32b700 1 --2- 172.21.15.7:0/1473241983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0674004870 0x7f0674004cf0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0678007fd0 tx=0x7f06740058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91b1255fc38ada2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:49.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 -- 172.21.15.7:0/1473241983 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0678001450 con 0x7f0674004870 2024-09-13T18:54:49.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 -- 172.21.15.7:0/1473241983 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06780052f0 con 0x7f0674004870 2024-09-13T18:54:49.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.962+0000 7f068cb2a700 1 -- 172.21.15.7:0/1473241983 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0678001d60 con 0x7f0674004870 2024-09-13T18:54:49.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.966+0000 7f068eb2e700 1 -- 172.21.15.7:0/1473241983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0674004870 msgr2=0x7f0674004cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:49.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.966+0000 7f068eb2e700 1 --2- 172.21.15.7:0/1473241983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0674004870 0x7f0674004cf0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0678007fd0 tx=0x7f06740058a0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.966+0000 7f068e32d700 1 -- 172.21.15.7:0/1473241983 reap_dead start 2024-09-13T18:54:49.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.966+0000 7f068eb2e700 1 -- 172.21.15.7:0/1473241983 shutdown_connections 2024-09-13T18:54:49.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.966+0000 7f068eb2e700 1 -- 172.21.15.7:0/1473241983 >> 172.21.15.7:0/1473241983 conn(0x7f06880cf9b0 msgr2=0x7f06880cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:49.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.966+0000 7f068eb2e700 1 -- 172.21.15.7:0/1473241983 shutdown_connections 2024-09-13T18:54:49.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.966+0000 7f068eb2e700 1 -- 172.21.15.7:0/1473241983 wait complete. 2024-09-13T18:54:49.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.966+0000 7f068eb2e700 1 Processor -- start 2024-09-13T18:54:49.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.966+0000 7f068eb2e700 1 -- start start 2024-09-13T18:54:49.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068eb2e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0674004870 0x7f068815a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068eb2e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f068815a0d0 0x7f068814f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068eb2e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06880cdce0 0x7f06880ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068eb2e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06880ce6a0 con 0x7f06880cdce0 2024-09-13T18:54:49.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068eb2e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068814fa60 con 0x7f0674004870 2024-09-13T18:54:49.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068eb2e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068815b000 con 0x7f068815a0d0 2024-09-13T18:54:49.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068e32d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06880cdce0 0x7f06880ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068e32d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06880cdce0 0x7f06880ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40454/0 (socket says 172.21.15.7:40454) 2024-09-13T18:54:49.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068e32d700 1 -- 172.21.15.7:0/2499268777 learned_addr learned my addr 172.21.15.7:0/2499268777 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:49.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068d32b700 1 --2- 172.21.15.7:0/2499268777 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f068815a0d0 0x7f068814f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068db2c700 1 --2- 172.21.15.7:0/2499268777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0674004870 0x7f068815a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068d32b700 1 -- 172.21.15.7:0/2499268777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0674004870 msgr2=0x7f068815a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:49.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068d32b700 1 --2- 172.21.15.7:0/2499268777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0674004870 0x7f068815a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068d32b700 1 -- 172.21.15.7:0/2499268777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06880cdce0 msgr2=0x7f06880ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:49.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068d32b700 1 --2- 172.21.15.7:0/2499268777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06880cdce0 0x7f06880ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:49.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068d32b700 1 -- 172.21.15.7:0/2499268777 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06881f2b20 con 0x7f068815a0d0 2024-09-13T18:54:49.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068e32d700 1 --2- 172.21.15.7:0/2499268777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06880cdce0 0x7f06880ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:49.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068d32b700 1 --2- 172.21.15.7:0/2499268777 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f068815a0d0 0x7f068814f520 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0678014000 tx=0x7f06780012d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:49.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f067e7fc700 1 -- 172.21.15.7:0/2499268777 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0678006c90 con 0x7f068815a0d0 2024-09-13T18:54:49.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f067e7fc700 1 -- 172.21.15.7:0/2499268777 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0678006e10 con 0x7f068815a0d0 2024-09-13T18:54:49.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f067e7fc700 1 -- 172.21.15.7:0/2499268777 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0678007280 con 0x7f068815a0d0 2024-09-13T18:54:49.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068eb2e700 1 -- 172.21.15.7:0/2499268777 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06881f2e00 con 0x7f068815a0d0 2024-09-13T18:54:49.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068eb2e700 1 -- 172.21.15.7:0/2499268777 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06881f33c0 con 0x7f068815a0d0 2024-09-13T18:54:49.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.970+0000 7f068eb2e700 1 -- 172.21.15.7:0/2499268777 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0688067830 con 0x7f068815a0d0 2024-09-13T18:54:49.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.974+0000 7f067e7fc700 1 -- 172.21.15.7:0/2499268777 <== mon.2 v2:172.21.15.171:3301/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f06780073e0 con 0x7f068815a0d0 2024-09-13T18:54:49.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.974+0000 7f067e7fc700 1 --2- 172.21.15.7:0/2499268777 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f066c066760 0x7f066c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:49.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.974+0000 7f067e7fc700 1 -- 172.21.15.7:0/2499268777 <== mon.2 v2:172.21.15.171:3301/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f0678085460 con 0x7f068815a0d0 2024-09-13T18:54:49.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.974+0000 7f068db2c700 1 --2- 172.21.15.7:0/2499268777 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f066c066760 0x7f066c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:49.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.974+0000 7f068db2c700 1 --2- 172.21.15.7:0/2499268777 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f066c066760 0x7f066c068c20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0684009d10 tx=0x7f0684009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:49.982+0000 7f067e7fc700 1 -- 172.21.15.7:0/2499268777 <== mon.2 v2:172.21.15.171:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0678052810 con 0x7f068815a0d0 2024-09-13T18:54:50.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.190+0000 7f068eb2e700 1 -- 172.21.15.7:0/2499268777 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f068815d570 con 0x7f068815a0d0 2024-09-13T18:54:50.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.842+0000 7f067e7fc700 1 -- 172.21.15.7:0/2499268777 <== mon.2 v2:172.21.15.171:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0678055e30 con 0x7f068815a0d0 2024-09-13T18:54:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.846+0000 7f068eb2e700 1 -- 172.21.15.7:0/2499268777 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f066c066760 msgr2=0x7f066c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.846+0000 7f068eb2e700 1 --2- 172.21.15.7:0/2499268777 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f066c066760 0x7f066c068c20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0684009d10 tx=0x7f0684009240 comp rx=0 tx=0).stop 2024-09-13T18:54:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.846+0000 7f068eb2e700 1 -- 172.21.15.7:0/2499268777 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f068815a0d0 msgr2=0x7f068814f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.846+0000 7f068eb2e700 1 --2- 172.21.15.7:0/2499268777 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f068815a0d0 0x7f068814f520 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0678014000 tx=0x7f06780012d0 comp rx=0 tx=0).stop 2024-09-13T18:54:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.846+0000 7f068eb2e700 1 -- 172.21.15.7:0/2499268777 shutdown_connections 2024-09-13T18:54:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.846+0000 7f068eb2e700 1 --2- 172.21.15.7:0/2499268777 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f066c066760 0x7f066c068c20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:50.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.846+0000 7f068eb2e700 1 --2- 172.21.15.7:0/2499268777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06880cdce0 0x7f06880ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.846+0000 7f068eb2e700 1 --2- 172.21.15.7:0/2499268777 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f068815a0d0 0x7f068814f520 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.846+0000 7f068eb2e700 1 --2- 172.21.15.7:0/2499268777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0674004870 0x7f068815a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.846+0000 7f068eb2e700 1 -- 172.21.15.7:0/2499268777 >> 172.21.15.7:0/2499268777 conn(0x7f06880cf9b0 msgr2=0x7f0688070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.846+0000 7f068eb2e700 1 -- 172.21.15.7:0/2499268777 shutdown_connections 2024-09-13T18:54:50.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.846+0000 7f068eb2e700 1 -- 172.21.15.7:0/2499268777 wait complete. 2024-09-13T18:54:50.870 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T18:54:51.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.994+0000 7f2446fdf700 1 Processor -- start 2024-09-13T18:54:51.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.994+0000 7f2446fdf700 1 -- start start 2024-09-13T18:54:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f2446fdf700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f244015cdf0 0x7f24400d3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f2446fdf700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f24400cfce0 0x7f24400b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f2446fdf700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24400d1cf0 con 0x7f24400d11c0 2024-09-13T18:54:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f2446fdf700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244015d1d0 con 0x7f24400cfce0 2024-09-13T18:54:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f2446fdf700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24400d1730 con 0x7f244015cdf0 2024-09-13T18:54:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f24467de700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f24400cfce0 0x7f24400b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f24467de700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f24400cfce0 0x7f24400b63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40458/0 (socket says 172.21.15.7:40458) 2024-09-13T18:54:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f24467de700 1 -- 172.21.15.7:0/1963783216 learned_addr learned my addr 172.21.15.7:0/1963783216 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f2445fdd700 1 --2- 172.21.15.7:0/1963783216 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f244015cdf0 0x7f24400d3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f24467de700 1 -- 172.21.15.7:0/1963783216 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f244015cdf0 msgr2=0x7f24400d3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:51.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f24467de700 1 --2- 172.21.15.7:0/1963783216 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f244015cdf0 0x7f24400d3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:51.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f24467de700 1 -- 172.21.15.7:0/1963783216 >> v1:172.21.15.171:6789/0 conn(0x7f24400d11c0 legacy=0x7f24400cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:51.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f24467de700 1 -- 172.21.15.7:0/1963783216 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24400d1a10 con 0x7f24400cfce0 2024-09-13T18:54:51.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f24467de700 1 --2- 172.21.15.7:0/1963783216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24400cfce0 0x7f24400b63d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2438009e50 tx=0x7f243800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc38345fab0f1202 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:51.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f2444fdb700 1 -- 172.21.15.7:0/1963783216 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800e070 con 0x7f24400cfce0 2024-09-13T18:54:51.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f2444fdb700 1 -- 172.21.15.7:0/1963783216 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243800b910 con 0x7f24400cfce0 2024-09-13T18:54:51.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f2444fdb700 1 -- 172.21.15.7:0/1963783216 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800bcc0 con 0x7f24400cfce0 2024-09-13T18:54:51.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f2446fdf700 1 -- 172.21.15.7:0/1963783216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24400cfce0 msgr2=0x7f24400b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:51.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f2446fdf700 1 --2- 172.21.15.7:0/1963783216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24400cfce0 0x7f24400b63d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2438009e50 tx=0x7f243800b330 comp rx=0 tx=0).stop 2024-09-13T18:54:51.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f2446fdf700 1 -- 172.21.15.7:0/1963783216 shutdown_connections 2024-09-13T18:54:51.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f2446fdf700 1 --2- 172.21.15.7:0/1963783216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24400cfce0 0x7f24400b63d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:51.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f2446fdf700 1 --2- 172.21.15.7:0/1963783216 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f244015cdf0 0x7f24400d3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:51.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f2446fdf700 1 -- 172.21.15.7:0/1963783216 >> 172.21.15.7:0/1963783216 conn(0x7f24400d2be0 msgr2=0x7f24400d2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:51.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f2446fdf700 1 -- 172.21.15.7:0/1963783216 shutdown_connections 2024-09-13T18:54:51.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:50.998+0000 7f2446fdf700 1 -- 172.21.15.7:0/1963783216 wait complete. 2024-09-13T18:54:51.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f2446fdf700 1 Processor -- start 2024-09-13T18:54:51.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f2446fdf700 1 -- start start 2024-09-13T18:54:51.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f2446fdf700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24400cfce0 0x7f24400c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:51.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f2446fdf700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24400d11c0 0x7f24400c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:51.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f2446fdf700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f244015cdf0 0x7f244015d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:51.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f2446fdf700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24400c8230 con 0x7f244015cdf0 2024-09-13T18:54:51.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f2446fdf700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24400c6f50 con 0x7f24400cfce0 2024-09-13T18:54:51.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f2446fdf700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24400c8f10 con 0x7f24400d11c0 2024-09-13T18:54:51.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f24467de700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f244015cdf0 0x7f244015d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:51.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f24467de700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f244015cdf0 0x7f244015d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40462/0 (socket says 172.21.15.7:40462) 2024-09-13T18:54:51.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f24467de700 1 -- 172.21.15.7:0/291344572 learned_addr learned my addr 172.21.15.7:0/291344572 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:51.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f2445fdd700 1 --2- 172.21.15.7:0/291344572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24400cfce0 0x7f24400c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:51.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f24457dc700 1 --2- 172.21.15.7:0/291344572 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24400d11c0 0x7f24400c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f2445fdd700 1 -- 172.21.15.7:0/291344572 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24400d11c0 msgr2=0x7f24400c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f2445fdd700 1 --2- 172.21.15.7:0/291344572 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24400d11c0 0x7f24400c7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f2445fdd700 1 -- 172.21.15.7:0/291344572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f244015cdf0 msgr2=0x7f244015d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f2445fdd700 1 --2- 172.21.15.7:0/291344572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f244015cdf0 0x7f244015d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f2445fdd700 1 -- 172.21.15.7:0/291344572 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2438009b30 con 0x7f24400cfce0 2024-09-13T18:54:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f24467de700 1 --2- 172.21.15.7:0/291344572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f244015cdf0 0x7f244015d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.002+0000 7f2445fdd700 1 --2- 172.21.15.7:0/291344572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24400cfce0 0x7f24400c88a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f243400e8a0 tx=0x7f243400ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.006+0000 7f243e7fc700 1 -- 172.21.15.7:0/291344572 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434013680 con 0x7f24400cfce0 2024-09-13T18:54:51.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.006+0000 7f2446fdf700 1 -- 172.21.15.7:0/291344572 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24400c7210 con 0x7f24400cfce0 2024-09-13T18:54:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.006+0000 7f243e7fc700 1 -- 172.21.15.7:0/291344572 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24340137e0 con 0x7f24400cfce0 2024-09-13T18:54:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.006+0000 7f243e7fc700 1 -- 172.21.15.7:0/291344572 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434013b70 con 0x7f24400cfce0 2024-09-13T18:54:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.006+0000 7f2446fdf700 1 -- 172.21.15.7:0/291344572 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24400c77d0 con 0x7f24400cfce0 2024-09-13T18:54:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.006+0000 7f2446fdf700 1 -- 172.21.15.7:0/291344572 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24400b63d0 con 0x7f24400cfce0 2024-09-13T18:54:51.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.006+0000 7f243e7fc700 1 -- 172.21.15.7:0/291344572 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2434013cd0 con 0x7f24400cfce0 2024-09-13T18:54:51.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.010+0000 7f243e7fc700 1 --2- 172.21.15.7:0/291344572 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2424066730 0x7f2424068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:51.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.010+0000 7f243e7fc700 1 -- 172.21.15.7:0/291344572 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f2434083f30 con 0x7f24400cfce0 2024-09-13T18:54:51.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.010+0000 7f24457dc700 1 --2- 172.21.15.7:0/291344572 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2424066730 0x7f2424068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:51.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.010+0000 7f24457dc700 1 --2- 172.21.15.7:0/291344572 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2424066730 0x7f2424068bf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2440157060 tx=0x7f242c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:51.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.018+0000 7f243e7fc700 1 -- 172.21.15.7:0/291344572 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2434051200 con 0x7f24400cfce0 2024-09-13T18:54:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.190+0000 7f2446fdf700 1 -- 172.21.15.7:0/291344572 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f24400b7c90 con 0x7f24400cfce0 2024-09-13T18:54:51.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.854+0000 7f243e7fc700 1 -- 172.21.15.7:0/291344572 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2434054820 con 0x7f24400cfce0 2024-09-13T18:54:51.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.858+0000 7f2446fdf700 1 -- 172.21.15.7:0/291344572 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2424066730 msgr2=0x7f2424068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:51.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.858+0000 7f2446fdf700 1 --2- 172.21.15.7:0/291344572 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2424066730 0x7f2424068bf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2440157060 tx=0x7f242c009040 comp rx=0 tx=0).stop 2024-09-13T18:54:51.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.858+0000 7f2446fdf700 1 -- 172.21.15.7:0/291344572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24400cfce0 msgr2=0x7f24400c88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:51.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.858+0000 7f2446fdf700 1 --2- 172.21.15.7:0/291344572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24400cfce0 0x7f24400c88a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f243400e8a0 tx=0x7f243400ebb0 comp rx=0 tx=0).stop 2024-09-13T18:54:51.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.858+0000 7f2446fdf700 1 -- 172.21.15.7:0/291344572 shutdown_connections 2024-09-13T18:54:51.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.858+0000 7f2446fdf700 1 --2- 172.21.15.7:0/291344572 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2424066730 0x7f2424068bf0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:51.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.858+0000 7f2446fdf700 1 --2- 172.21.15.7:0/291344572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f244015cdf0 0x7f244015d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:51.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.858+0000 7f2446fdf700 1 --2- 172.21.15.7:0/291344572 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f24400d11c0 0x7f24400c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:51.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.858+0000 7f2446fdf700 1 --2- 172.21.15.7:0/291344572 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f24400cfce0 0x7f24400c88a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:51.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.858+0000 7f2446fdf700 1 -- 172.21.15.7:0/291344572 >> 172.21.15.7:0/291344572 conn(0x7f24400d2be0 msgr2=0x7f24400b8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:51.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.858+0000 7f2446fdf700 1 -- 172.21.15.7:0/291344572 shutdown_connections 2024-09-13T18:54:51.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.858+0000 7f2446fdf700 1 -- 172.21.15.7:0/291344572 wait complete. 2024-09-13T18:54:51.866 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-09-13T18:54:51.878 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T18:54:52.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.994+0000 7f545d076700 1 Processor -- start 2024-09-13T18:54:52.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.994+0000 7f545d076700 1 -- start start 2024-09-13T18:54:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.998+0000 7f545d076700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f545806aa70 0x7f545815af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.998+0000 7f545d076700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545815bc00 con 0x7f54580aec70 2024-09-13T18:54:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.998+0000 7f545d076700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545815e6a0 con 0x7f545815f180 2024-09-13T18:54:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.998+0000 7f545d076700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545815bee0 con 0x7f545806aa70 2024-09-13T18:54:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.998+0000 7f5457fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f54580aec70 0x7f545806a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57968/0 (socket says 172.21.15.7:57968) 2024-09-13T18:54:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.998+0000 7f5457fff700 1 -- 172.21.15.7:0/2440208064 learned_addr learned my addr 172.21.15.7:0/2440208064 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.998+0000 7f544f7fe700 1 --2- 172.21.15.7:0/2440208064 >> v2:172.21.15.7:3300/0 conn(0x7f545806aa70 0x7f545815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.998+0000 7f54577fe700 1 -- 172.21.15.7:0/2440208064 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504034911 0 0) 0x7f545815bc00 con 0x7f54580aec70 2024-09-13T18:54:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.998+0000 7f54577fe700 1 -- 172.21.15.7:0/2440208064 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54440035d0 con 0x7f54580aec70 2024-09-13T18:54:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.998+0000 7f544f7fe700 1 -- 172.21.15.7:0/2440208064 >> v1:172.21.15.171:6789/0 conn(0x7f545815f180 legacy=0x7f545815b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.998+0000 7f544f7fe700 1 -- 172.21.15.7:0/2440208064 >> v1:172.21.15.7:6789/0 conn(0x7f54580aec70 legacy=0x7f545806a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:52.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.998+0000 7f544f7fe700 1 -- 172.21.15.7:0/2440208064 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545815e980 con 0x7f545806aa70 2024-09-13T18:54:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.998+0000 7f544f7fe700 1 --2- 172.21.15.7:0/2440208064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545806aa70 0x7f545815af10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5440009880 tx=0x7f5440009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98df11105638c89f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.998+0000 7f54577fe700 1 -- 172.21.15.7:0/2440208064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440011830 con 0x7f545806aa70 2024-09-13T18:54:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.998+0000 7f54577fe700 1 -- 172.21.15.7:0/2440208064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5440011990 con 0x7f545806aa70 2024-09-13T18:54:52.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:51.998+0000 7f54577fe700 1 -- 172.21.15.7:0/2440208064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440011d20 con 0x7f545806aa70 2024-09-13T18:54:52.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.002+0000 7f545d076700 1 -- 172.21.15.7:0/2440208064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545806aa70 msgr2=0x7f545815af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:52.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.002+0000 7f545d076700 1 --2- 172.21.15.7:0/2440208064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545806aa70 0x7f545815af10 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5440009880 tx=0x7f5440009b90 comp rx=0 tx=0).stop 2024-09-13T18:54:52.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.002+0000 7f545d076700 1 -- 172.21.15.7:0/2440208064 shutdown_connections 2024-09-13T18:54:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.002+0000 7f545d076700 1 --2- 172.21.15.7:0/2440208064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545806aa70 0x7f545815af10 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.002+0000 7f545d076700 1 -- 172.21.15.7:0/2440208064 >> 172.21.15.7:0/2440208064 conn(0x7f545806baa0 msgr2=0x7f54580b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.002+0000 7f545d076700 1 -- 172.21.15.7:0/2440208064 shutdown_connections 2024-09-13T18:54:52.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.002+0000 7f545d076700 1 -- 172.21.15.7:0/2440208064 wait complete. 2024-09-13T18:54:52.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f545d076700 1 Processor -- start 2024-09-13T18:54:52.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f545d076700 1 -- start start 2024-09-13T18:54:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f545d076700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f545806aa70 0x7f54581464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f545d076700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54580aec70 0x7f54580b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f545d076700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545815f180 0x7f5458158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f545d076700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54580b1220 con 0x7f545815f180 2024-09-13T18:54:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f545d076700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54580b0c60 con 0x7f54580aec70 2024-09-13T18:54:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f545d076700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54580b0f40 con 0x7f545806aa70 2024-09-13T18:54:52.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f545c875700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545815f180 0x7f5458158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f545c875700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545815f180 0x7f5458158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40742/0 (socket says 172.21.15.7:40742) 2024-09-13T18:54:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f545c875700 1 -- 172.21.15.7:0/332478356 learned_addr learned my addr 172.21.15.7:0/332478356 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f5457fff700 1 --2- 172.21.15.7:0/332478356 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f545806aa70 0x7f54581464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:52.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f544f7fe700 1 --2- 172.21.15.7:0/332478356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54580aec70 0x7f54580b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f545c875700 1 -- 172.21.15.7:0/332478356 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f545806aa70 msgr2=0x7f54581464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:52.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f545c875700 1 --2- 172.21.15.7:0/332478356 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f545806aa70 0x7f54581464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f545c875700 1 -- 172.21.15.7:0/332478356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54580aec70 msgr2=0x7f54580b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f545c875700 1 --2- 172.21.15.7:0/332478356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54580aec70 0x7f54580b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f545c875700 1 -- 172.21.15.7:0/332478356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54400094b0 con 0x7f545815f180 2024-09-13T18:54:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.006+0000 7f545c875700 1 --2- 172.21.15.7:0/332478356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545815f180 0x7f5458158880 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f545000cbc0 tx=0x7f545000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.010+0000 7f54557fa700 1 -- 172.21.15.7:0/332478356 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450003c20 con 0x7f545815f180 2024-09-13T18:54:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.010+0000 7f54557fa700 1 -- 172.21.15.7:0/332478356 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5450003d80 con 0x7f545815f180 2024-09-13T18:54:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.010+0000 7f54557fa700 1 -- 172.21.15.7:0/332478356 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545000aa20 con 0x7f545815f180 2024-09-13T18:54:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.010+0000 7f545d076700 1 -- 172.21.15.7:0/332478356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5458158080 con 0x7f545815f180 2024-09-13T18:54:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.010+0000 7f545d076700 1 -- 172.21.15.7:0/332478356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54580b0950 con 0x7f545815f180 2024-09-13T18:54:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.010+0000 7f545d076700 1 -- 172.21.15.7:0/332478356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5458157ee0 con 0x7f545815f180 2024-09-13T18:54:52.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.010+0000 7f54557fa700 1 -- 172.21.15.7:0/332478356 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f545000ab80 con 0x7f545815f180 2024-09-13T18:54:52.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.010+0000 7f54557fa700 1 --2- 172.21.15.7:0/332478356 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5438066730 0x7f5438068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:52.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.010+0000 7f54557fa700 1 -- 172.21.15.7:0/332478356 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5443+0+0 (secure 0 0 0) 0x7f5450083ff0 con 0x7f545815f180 2024-09-13T18:54:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.010+0000 7f5457fff700 1 --2- 172.21.15.7:0/332478356 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5438066730 0x7f5438068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.014+0000 7f5457fff700 1 --2- 172.21.15.7:0/332478356 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5438066730 0x7f5438068bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5448007fd0 tx=0x7f5448007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:52.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.014+0000 7f54557fa700 1 -- 172.21.15.7:0/332478356 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5458157ee0 con 0x7f545815f180 2024-09-13T18:54:52.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.182+0000 7f545d076700 1 -- 172.21.15.7:0/332478356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f54581585a0 con 0x7f545815f180 2024-09-13T18:54:52.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.862+0000 7f54557fa700 1 -- 172.21.15.7:0/332478356 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) v1 ==== 169+0+0 (secure 0 0 0) 0x7f54581585a0 con 0x7f545815f180 2024-09-13T18:54:52.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.866+0000 7f545d076700 1 -- 172.21.15.7:0/332478356 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5438066730 msgr2=0x7f5438068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:52.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.866+0000 7f545d076700 1 --2- 172.21.15.7:0/332478356 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5438066730 0x7f5438068bf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5448007fd0 tx=0x7f5448007de0 comp rx=0 tx=0).stop 2024-09-13T18:54:52.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.866+0000 7f545d076700 1 -- 172.21.15.7:0/332478356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545815f180 msgr2=0x7f5458158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:52.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.866+0000 7f545d076700 1 --2- 172.21.15.7:0/332478356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545815f180 0x7f5458158880 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f545000cbc0 tx=0x7f545000ced0 comp rx=0 tx=0).stop 2024-09-13T18:54:52.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.866+0000 7f545d076700 1 -- 172.21.15.7:0/332478356 shutdown_connections 2024-09-13T18:54:52.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.866+0000 7f545d076700 1 --2- 172.21.15.7:0/332478356 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f5438066730 0x7f5438068bf0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:52.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.866+0000 7f545d076700 1 --2- 172.21.15.7:0/332478356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f545815f180 0x7f5458158880 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:52.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.866+0000 7f545d076700 1 --2- 172.21.15.7:0/332478356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f54580aec70 0x7f54580b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:52.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.866+0000 7f545d076700 1 --2- 172.21.15.7:0/332478356 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f545806aa70 0x7f54581464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:52.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.866+0000 7f545d076700 1 -- 172.21.15.7:0/332478356 >> 172.21.15.7:0/332478356 conn(0x7f545806baa0 msgr2=0x7f5458157230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:52.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.866+0000 7f545d076700 1 -- 172.21.15.7:0/332478356 shutdown_connections 2024-09-13T18:54:52.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:52.866+0000 7f545d076700 1 -- 172.21.15.7:0/332478356 wait complete. 2024-09-13T18:54:52.875 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-09-13T18:54:52.888 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T18:54:52.890 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T18:54:52.913 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-09-13T18:54:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-09-13T18:54:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T18:54:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-09-13T18:54:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.913 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-09-13T18:54:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-09-13T18:54:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-09-13T18:54:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-09-13T18:54:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-09-13T18:54:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-09-13T18:54:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T18:54:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-09-13T18:54:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.914 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T18:54:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T18:54:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-09-13T18:54:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-09-13T18:54:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-09-13T18:54:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-09-13T18:54:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-09-13T18:54:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-09-13T18:54:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-09-13T18:54:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-09-13T18:54:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T18:54:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T18:54:52.915 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-09-13T18:54:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T18:54:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-09-13T18:54:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-09-13T18:54:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-09-13T18:54:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-09-13T18:54:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-09-13T18:54:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.171,[v2:172.21.15.171:3301,v1:172.21.15.171:6790]" 2024-09-13T18:54:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 33406d92-f401-438a-876e-a56de3962d8e 2024-09-13T18:54:52.916 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.917 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-09-13T18:54:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-09-13T18:54:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-09-13T18:54:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-09-13T18:54:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-09-13T18:54:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-09-13T18:54:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-09-13T18:54:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-09-13T18:54:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-09-13T18:54:52.917 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-09-13T18:54:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-09-13T18:54:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-09-13T18:54:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-09-13T18:54:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-09-13T18:54:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-09-13T18:54:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-09-13T18:54:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-09-13T18:54:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.918 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-09-13T18:54:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-09-13T18:54:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: bdev async discard = True 2024-09-13T18:54:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: bdev enable discard = True 2024-09-13T18:54:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore allocator = bitmap 2024-09-13T18:54:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-09-13T18:54:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-09-13T18:54:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 1/20 2024-09-13T18:54:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 1/20 2024-09-13T18:54:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:54:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-09-13T18:54:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 4/10 2024-09-13T18:54:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T18:54:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-09-13T18:54:52.919 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-09-13T18:54:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2024-09-13T18:54:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2024-09-13T18:54:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-09-13T18:54:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.920 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-09-13T18:54:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:54:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-09-13T18:54:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-13T18:54:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-13T18:54:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:54:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:54:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-09-13T18:54:52.920 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-09-13T18:54:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.921 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-09-13T18:54:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:54:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-13T18:54:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-09-13T18:54:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-13T18:54:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-09-13T18:54:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-09-13T18:54:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:54:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-09-13T18:54:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-09-13T18:54:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:54:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.921 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T18:54:52.922 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T18:54:52.922 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T18:54:52.922 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.922 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T18:54:52.922 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T18:54:52.922 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T18:54:52.922 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.922 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-09-13T18:54:52.922 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-09-13T18:54:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-09-13T18:54:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: mon op complaint time = 120 2024-09-13T18:54:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on pool no app = False 2024-09-13T18:54:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:52.923 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-09-13T18:54:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-09-13T18:54:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-09-13T18:54:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-09-13T18:54:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T18:54:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T18:54:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 600 2024-09-13T18:54:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-09-13T18:54:52.923 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:54:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = "" 2024-09-13T18:54:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-09-13T18:54:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-09-13T18:54:52.924 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-09-13T18:54:52.924 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-09-13T18:54:52.924 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-09-13T18:54:52.924 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-09-13T18:54:52.924 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-09-13T18:54:52.924 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-09-13T18:54:52.924 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-09-13T18:54:52.924 INFO:teuthology.orchestra.run.smithi007.stdout:[mds] 2024-09-13T18:54:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds = 20 2024-09-13T18:54:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds balancer = 20 2024-09-13T18:54:52.924 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-13T18:54:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug frag = True 2024-09-13T18:54:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug scatterstat = True 2024-09-13T18:54:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: mds op complaint time = 180 2024-09-13T18:54:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: mds verify scatter = True 2024-09-13T18:54:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2024-09-13T18:54:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2024-09-13T18:54:52.925 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2024-09-13T18:54:52.926 INFO:teuthology.orchestra.run.smithi171.stdout:[global] 2024-09-13T18:54:52.926 INFO:teuthology.orchestra.run.smithi171.stdout: chdir = "" 2024-09-13T18:54:52.926 INFO:teuthology.orchestra.run.smithi171.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T18:54:52.926 INFO:teuthology.orchestra.run.smithi171.stdout: auth supported = cephx 2024-09-13T18:54:52.926 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.926 INFO:teuthology.orchestra.run.smithi171.stdout: filestore xattr use omap = true 2024-09-13T18:54:52.926 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.926 INFO:teuthology.orchestra.run.smithi171.stdout: mon clock drift allowed = 1.000 2024-09-13T18:54:52.926 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.926 INFO:teuthology.orchestra.run.smithi171.stdout: osd crush chooseleaf type = 0 2024-09-13T18:54:52.927 INFO:teuthology.orchestra.run.smithi171.stdout: auth debug = true 2024-09-13T18:54:52.927 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.927 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on old message = true 2024-09-13T18:54:52.927 INFO:teuthology.orchestra.run.smithi171.stdout: ms die on bug = true 2024-09-13T18:54:52.927 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.927 INFO:teuthology.orchestra.run.smithi171.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T18:54:52.927 INFO:teuthology.orchestra.run.smithi171.stdout: mon pg warn max object skew = 0 2024-09-13T18:54:52.927 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.927 INFO:teuthology.orchestra.run.smithi171.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T18:54:52.927 INFO:teuthology.orchestra.run.smithi171.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T18:54:52.927 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.927 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default size = 2 2024-09-13T18:54:52.927 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow primary affinity = true 2024-09-13T18:54:52.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd allow pg remap = true 2024-09-13T18:54:52.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on legacy crush tunables = false 2024-09-13T18:54:52.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on crush straw calc version zero = false 2024-09-13T18:54:52.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on no sortbitwise = false 2024-09-13T18:54:52.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on osd down out interval zero = false 2024-09-13T18:54:52.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on too few osds = false 2024-09-13T18:54:52.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T18:54:52.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T18:54:52.928 INFO:teuthology.orchestra.run.smithi171.stdout: mon_allow_pool_size_one = true 2024-09-13T18:54:52.928 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.928 INFO:teuthology.orchestra.run.smithi171.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T18:54:52.928 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.929 INFO:teuthology.orchestra.run.smithi171.stdout: osd default data pool replay window = 5 2024-09-13T18:54:52.929 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.929 INFO:teuthology.orchestra.run.smithi171.stdout: mon allow pool delete = true 2024-09-13T18:54:52.929 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.929 INFO:teuthology.orchestra.run.smithi171.stdout: mon cluster log file level = debug 2024-09-13T18:54:52.929 INFO:teuthology.orchestra.run.smithi171.stdout: debug asserts on shutdown = true 2024-09-13T18:54:52.929 INFO:teuthology.orchestra.run.smithi171.stdout: mon health detail to clog = false 2024-09-13T18:54:52.929 INFO:teuthology.orchestra.run.smithi171.stdout: mon host = "172.21.15.7,172.21.15.171,[v2:172.21.15.171:3301,v1:172.21.15.171:6790]" 2024-09-13T18:54:52.929 INFO:teuthology.orchestra.run.smithi171.stdout: fsid = 33406d92-f401-438a-876e-a56de3962d8e 2024-09-13T18:54:52.929 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.929 INFO:teuthology.orchestra.run.smithi171.stdout:[osd] 2024-09-13T18:54:52.929 INFO:teuthology.orchestra.run.smithi171.stdout: osd journal size = 100 2024-09-13T18:54:52.929 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.930 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub load threshold = 5.0 2024-09-13T18:54:52.930 INFO:teuthology.orchestra.run.smithi171.stdout: osd scrub max interval = 600 2024-09-13T18:54:52.930 INFO:teuthology.orchestra.run.smithi171.stdout: osd mclock profile = high_recovery_ops 2024-09-13T18:54:52.930 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.930 INFO:teuthology.orchestra.run.smithi171.stdout: osd recover clone overlap = true 2024-09-13T18:54:52.930 INFO:teuthology.orchestra.run.smithi171.stdout: osd recovery max chunk = 1048576 2024-09-13T18:54:52.930 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.930 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug shutdown = true 2024-09-13T18:54:52.930 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug op order = true 2024-09-13T18:54:52.930 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug verify stray on activate = true 2024-09-13T18:54:52.930 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.930 INFO:teuthology.orchestra.run.smithi171.stdout: osd open classes on start = true 2024-09-13T18:54:52.930 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug pg log writeout = true 2024-09-13T18:54:52.931 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.931 INFO:teuthology.orchestra.run.smithi171.stdout: osd deep scrub update digest min age = 30 2024-09-13T18:54:52.931 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.931 INFO:teuthology.orchestra.run.smithi171.stdout: osd map max advance = 10 2024-09-13T18:54:52.931 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.931 INFO:teuthology.orchestra.run.smithi171.stdout: journal zero on create = true 2024-09-13T18:54:52.931 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.931 INFO:teuthology.orchestra.run.smithi171.stdout: filestore ondisk finisher threads = 3 2024-09-13T18:54:52.931 INFO:teuthology.orchestra.run.smithi171.stdout: filestore apply finisher threads = 3 2024-09-13T18:54:52.931 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.931 INFO:teuthology.orchestra.run.smithi171.stdout: bdev debug aio = true 2024-09-13T18:54:52.931 INFO:teuthology.orchestra.run.smithi171.stdout: osd debug misdirected ops = true 2024-09-13T18:54:52.931 INFO:teuthology.orchestra.run.smithi171.stdout: bdev async discard = True 2024-09-13T18:54:52.932 INFO:teuthology.orchestra.run.smithi171.stdout: bdev enable discard = True 2024-09-13T18:54:52.932 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore allocator = bitmap 2024-09-13T18:54:52.932 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore block size = 96636764160 2024-09-13T18:54:52.932 INFO:teuthology.orchestra.run.smithi171.stdout: bluestore fsck on mount = True 2024-09-13T18:54:52.932 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluefs = 1/20 2024-09-13T18:54:52.932 INFO:teuthology.orchestra.run.smithi171.stdout: debug bluestore = 1/20 2024-09-13T18:54:52.932 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:54:52.932 INFO:teuthology.orchestra.run.smithi171.stdout: debug osd = 20 2024-09-13T18:54:52.932 INFO:teuthology.orchestra.run.smithi171.stdout: debug rocksdb = 4/10 2024-09-13T18:54:52.932 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T18:54:52.932 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd full ratio = 0.9 2024-09-13T18:54:52.932 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd nearfull ratio = 0.8 2024-09-13T18:54:52.932 INFO:teuthology.orchestra.run.smithi171.stdout: osd failsafe full ratio = 0.95 2024-09-13T18:54:52.933 INFO:teuthology.orchestra.run.smithi171.stdout: osd objectstore = bluestore 2024-09-13T18:54:52.933 INFO:teuthology.orchestra.run.smithi171.stdout: osd op complaint time = 180 2024-09-13T18:54:52.933 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.933 INFO:teuthology.orchestra.run.smithi171.stdout:[mgr] 2024-09-13T18:54:52.933 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:54:52.933 INFO:teuthology.orchestra.run.smithi171.stdout: debug mgr = 20 2024-09-13T18:54:52.933 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-09-13T18:54:52.933 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-09-13T18:54:52.933 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:54:52.933 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:54:52.933 INFO:teuthology.orchestra.run.smithi171.stdout: mgr/telemetry/nag = false 2024-09-13T18:54:52.933 INFO:teuthology.orchestra.run.smithi171.stdout: debug client = 20 2024-09-13T18:54:52.933 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.933 INFO:teuthology.orchestra.run.smithi171.stdout:[mon] 2024-09-13T18:54:52.934 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:54:52.934 INFO:teuthology.orchestra.run.smithi171.stdout: debug mon = 20 2024-09-13T18:54:52.934 INFO:teuthology.orchestra.run.smithi171.stdout: debug paxos = 20 2024-09-13T18:54:52.934 INFO:teuthology.orchestra.run.smithi171.stdout: debug auth = 20 2024-09-13T18:54:52.934 INFO:teuthology.orchestra.run.smithi171.stdout: mon data avail warn = 5 2024-09-13T18:54:52.934 INFO:teuthology.orchestra.run.smithi171.stdout: mon mgr mkfs grace = 240 2024-09-13T18:54:52.934 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min pgs per osd = 4 2024-09-13T18:54:52.934 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd reporter subtree level = osd 2024-09-13T18:54:52.934 INFO:teuthology.orchestra.run.smithi171.stdout: mon osd prime pg temp = true 2024-09-13T18:54:52.934 INFO:teuthology.orchestra.run.smithi171.stdout: mon reweight min bytes per osd = 10 2024-09-13T18:54:52.934 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.934 INFO:teuthology.orchestra.run.smithi171.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T18:54:52.934 INFO:teuthology.orchestra.run.smithi171.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T18:54:52.934 INFO:teuthology.orchestra.run.smithi171.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T18:54:52.935 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.935 INFO:teuthology.orchestra.run.smithi171.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T18:54:52.935 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T18:54:52.935 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T18:54:52.935 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.935 INFO:teuthology.orchestra.run.smithi171.stdout: # 1m isn't quite enough 2024-09-13T18:54:52.935 INFO:teuthology.orchestra.run.smithi171.stdout: mon_down_mkfs_grace = 2m 2024-09-13T18:54:52.935 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.935 INFO:teuthology.orchestra.run.smithi171.stdout: mon_warn_on_filestore_osds = false 2024-09-13T18:54:52.935 INFO:teuthology.orchestra.run.smithi171.stdout: mon op complaint time = 120 2024-09-13T18:54:52.935 INFO:teuthology.orchestra.run.smithi171.stdout: mon warn on pool no app = False 2024-09-13T18:54:52.935 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-13T18:54:52.935 INFO:teuthology.orchestra.run.smithi171.stdout:[client] 2024-09-13T18:54:52.936 INFO:teuthology.orchestra.run.smithi171.stdout: rgw cache enabled = true 2024-09-13T18:54:52.936 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable ops log = true 2024-09-13T18:54:52.936 INFO:teuthology.orchestra.run.smithi171.stdout: rgw enable usage log = true 2024-09-13T18:54:52.936 INFO:teuthology.orchestra.run.smithi171.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T18:54:52.936 INFO:teuthology.orchestra.run.smithi171.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T18:54:52.936 INFO:teuthology.orchestra.run.smithi171.stdout: client mount timeout = 600 2024-09-13T18:54:52.936 INFO:teuthology.orchestra.run.smithi171.stdout: debug client = 20 2024-09-13T18:54:52.936 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:54:52.936 INFO:teuthology.orchestra.run.smithi171.stdout: pid file = "" 2024-09-13T18:54:52.936 INFO:teuthology.orchestra.run.smithi171.stdout: rados mon op timeout = 900 2024-09-13T18:54:52.936 INFO:teuthology.orchestra.run.smithi171.stdout: rados osd op timeout = 900 2024-09-13T18:54:52.936 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.a] 2024-09-13T18:54:52.936 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.b] 2024-09-13T18:54:52.936 INFO:teuthology.orchestra.run.smithi171.stdout:[mon.c] 2024-09-13T18:54:52.937 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.a] 2024-09-13T18:54:52.937 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.c] 2024-09-13T18:54:52.937 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.b] 2024-09-13T18:54:52.937 INFO:teuthology.orchestra.run.smithi171.stdout:[mds.d] 2024-09-13T18:54:52.937 INFO:teuthology.orchestra.run.smithi171.stdout:[mds] 2024-09-13T18:54:52.937 INFO:teuthology.orchestra.run.smithi171.stdout: debug mds = 20 2024-09-13T18:54:52.937 INFO:teuthology.orchestra.run.smithi171.stdout: debug mds balancer = 20 2024-09-13T18:54:52.937 INFO:teuthology.orchestra.run.smithi171.stdout: debug ms = 1 2024-09-13T18:54:52.937 INFO:teuthology.orchestra.run.smithi171.stdout: mds debug frag = True 2024-09-13T18:54:52.937 INFO:teuthology.orchestra.run.smithi171.stdout: mds debug scatterstat = True 2024-09-13T18:54:52.937 INFO:teuthology.orchestra.run.smithi171.stdout: mds op complaint time = 180 2024-09-13T18:54:52.937 INFO:teuthology.orchestra.run.smithi171.stdout: mds verify scatter = True 2024-09-13T18:54:52.937 INFO:teuthology.orchestra.run.smithi171.stdout: osd op complaint time = 180 2024-09-13T18:54:52.937 INFO:teuthology.orchestra.run.smithi171.stdout: rados mon op timeout = 900 2024-09-13T18:54:52.938 INFO:teuthology.orchestra.run.smithi171.stdout: rados osd op timeout = 900 2024-09-13T18:54:52.939 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-13T18:54:53.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.062+0000 7fa0d1a93700 1 Processor -- start 2024-09-13T18:54:53.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.062+0000 7fa0d1a93700 1 -- start start 2024-09-13T18:54:53.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.062+0000 7fa0d1a93700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa0cc06e390 0x7fa0cc0b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:53.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.062+0000 7fa0d1a93700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0cc06e770 con 0x7fa0cc0b38f0 2024-09-13T18:54:53.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.062+0000 7fa0d1a93700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0cc0b3d90 con 0x7fa0cc165760 2024-09-13T18:54:53.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.062+0000 7fa0d1a93700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0cc15b9f0 con 0x7fa0cc06e390 2024-09-13T18:54:53.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.062+0000 7fa0d0a91700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa0cc06e390 0x7fa0cc0b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:53.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.062+0000 7fa0d1292700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa0cc0b38f0 0x7fa0cc0ae510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57982/0 (socket says 172.21.15.7:57982) 2024-09-13T18:54:53.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.062+0000 7fa0d1292700 1 -- 172.21.15.7:0/2856002733 learned_addr learned my addr 172.21.15.7:0/2856002733 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:53.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.062+0000 7fa0d0a91700 1 -- 172.21.15.7:0/2856002733 >> v1:172.21.15.171:6789/0 conn(0x7fa0cc165760 legacy=0x7fa0cc164b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:53.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.062+0000 7fa0d0a91700 1 -- 172.21.15.7:0/2856002733 >> v1:172.21.15.7:6789/0 conn(0x7fa0cc0b38f0 legacy=0x7fa0cc0ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:53.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.062+0000 7fa0cb7fe700 1 -- 172.21.15.7:0/2856002733 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 475393173 0 0) 0x7fa0cc06e770 con 0x7fa0cc0b38f0 2024-09-13T18:54:53.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.062+0000 7fa0d0a91700 1 -- 172.21.15.7:0/2856002733 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc0b4070 con 0x7fa0cc06e390 2024-09-13T18:54:53.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.062+0000 7fa0d0a91700 1 --2- 172.21.15.7:0/2856002733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0cc06e390 0x7fa0cc0b33b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009d90 tx=0x7fa0c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56140b43a3902ae7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:53.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.062+0000 7fa0cb7fe700 1 -- 172.21.15.7:0/2856002733 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c400e070 con 0x7fa0cc06e390 2024-09-13T18:54:53.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.062+0000 7fa0cb7fe700 1 -- 172.21.15.7:0/2856002733 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c400bbd0 con 0x7fa0cc06e390 2024-09-13T18:54:53.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.062+0000 7fa0cb7fe700 1 -- 172.21.15.7:0/2856002733 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4012430 con 0x7fa0cc06e390 2024-09-13T18:54:53.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.066+0000 7fa0d1a93700 1 -- 172.21.15.7:0/2856002733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0cc06e390 msgr2=0x7fa0cc0b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:53.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.066+0000 7fa0d1a93700 1 --2- 172.21.15.7:0/2856002733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0cc06e390 0x7fa0cc0b33b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009d90 tx=0x7fa0c400b330 comp rx=0 tx=0).stop 2024-09-13T18:54:53.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.066+0000 7fa0d1a93700 1 -- 172.21.15.7:0/2856002733 shutdown_connections 2024-09-13T18:54:53.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.066+0000 7fa0d1a93700 1 --2- 172.21.15.7:0/2856002733 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0cc06e390 0x7fa0cc0b33b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:53.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.066+0000 7fa0d1a93700 1 -- 172.21.15.7:0/2856002733 >> 172.21.15.7:0/2856002733 conn(0x7fa0cc0b1cd0 msgr2=0x7fa0cc0cf910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:53.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.066+0000 7fa0d1a93700 1 -- 172.21.15.7:0/2856002733 shutdown_connections 2024-09-13T18:54:53.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.066+0000 7fa0d1a93700 1 -- 172.21.15.7:0/2856002733 wait complete. 2024-09-13T18:54:53.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.066+0000 7fa0d1a93700 1 Processor -- start 2024-09-13T18:54:53.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.066+0000 7fa0d1a93700 1 -- start start 2024-09-13T18:54:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.066+0000 7fa0d1a93700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0cc06e390 0x7fa0cc15ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.066+0000 7fa0d1a93700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0cc0b38f0 0x7fa0cc15f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.066+0000 7fa0d1a93700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa0cc165760 0x7fa0cc154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:53.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.066+0000 7fa0d1a93700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc06ce00 con 0x7fa0cc0b38f0 2024-09-13T18:54:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.066+0000 7fa0d1a93700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc06c1f0 con 0x7fa0cc06e390 2024-09-13T18:54:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.066+0000 7fa0d1a93700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc06c4d0 con 0x7fa0cc165760 2024-09-13T18:54:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.066+0000 7fa0cbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0cc0b38f0 0x7fa0cc15f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.066+0000 7fa0cbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0cc0b38f0 0x7fa0cc15f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40762/0 (socket says 172.21.15.7:40762) 2024-09-13T18:54:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.070+0000 7fa0cbfff700 1 -- 172.21.15.7:0/410376814 learned_addr learned my addr 172.21.15.7:0/410376814 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.070+0000 7fa0d0a91700 1 --2- 172.21.15.7:0/410376814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0cc06e390 0x7fa0cc15ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.070+0000 7fa0d1292700 1 --2- 172.21.15.7:0/410376814 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa0cc165760 0x7fa0cc154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.070+0000 7fa0cbfff700 1 -- 172.21.15.7:0/410376814 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa0cc165760 msgr2=0x7fa0cc154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.070+0000 7fa0cbfff700 1 --2- 172.21.15.7:0/410376814 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa0cc165760 0x7fa0cc154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.070+0000 7fa0cbfff700 1 -- 172.21.15.7:0/410376814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0cc06e390 msgr2=0x7fa0cc15ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.070+0000 7fa0cbfff700 1 --2- 172.21.15.7:0/410376814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0cc06e390 0x7fa0cc15ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.070+0000 7fa0cbfff700 1 -- 172.21.15.7:0/410376814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c4009a70 con 0x7fa0cc0b38f0 2024-09-13T18:54:53.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.070+0000 7fa0d1292700 1 --2- 172.21.15.7:0/410376814 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa0cc165760 0x7fa0cc154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.070+0000 7fa0cbfff700 1 --2- 172.21.15.7:0/410376814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0cc0b38f0 0x7fa0cc15f340 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400c400 tx=0x7fa0b400c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.070+0000 7fa0c97fa700 1 -- 172.21.15.7:0/410376814 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40052d0 con 0x7fa0cc0b38f0 2024-09-13T18:54:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.070+0000 7fa0c97fa700 1 -- 172.21.15.7:0/410376814 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b4005450 con 0x7fa0cc0b38f0 2024-09-13T18:54:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.070+0000 7fa0d1a93700 1 -- 172.21.15.7:0/410376814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0cc06c7b0 con 0x7fa0cc0b38f0 2024-09-13T18:54:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.070+0000 7fa0c97fa700 1 -- 172.21.15.7:0/410376814 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40038e0 con 0x7fa0cc0b38f0 2024-09-13T18:54:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.070+0000 7fa0d1a93700 1 -- 172.21.15.7:0/410376814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0cc06d290 con 0x7fa0cc0b38f0 2024-09-13T18:54:53.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.070+0000 7fa0d1a93700 1 -- 172.21.15.7:0/410376814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0cc06cfb0 con 0x7fa0cc0b38f0 2024-09-13T18:54:53.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.070+0000 7fa0c97fa700 1 -- 172.21.15.7:0/410376814 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fa0b4003a40 con 0x7fa0cc0b38f0 2024-09-13T18:54:53.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.074+0000 7fa0c97fa700 1 --2- 172.21.15.7:0/410376814 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa0b0066730 0x7fa0b0068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:53.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.074+0000 7fa0c97fa700 1 -- 172.21.15.7:0/410376814 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fa0b40831d0 con 0x7fa0cc0b38f0 2024-09-13T18:54:53.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.074+0000 7fa0d0a91700 1 --2- 172.21.15.7:0/410376814 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa0b0066730 0x7fa0b0068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:53.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.074+0000 7fa0d0a91700 1 --2- 172.21.15.7:0/410376814 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa0b0066730 0x7fa0b0068bf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4014040 tx=0x7fa0c401e040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:53.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.078+0000 7fa0c97fa700 1 -- 172.21.15.7:0/410376814 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa0cc06cfb0 con 0x7fa0cc0b38f0 2024-09-13T18:54:53.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.250+0000 7fa0d1a93700 1 -- 172.21.15.7:0/410376814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa0cc06cfb0 con 0x7fa0cc0b38f0 2024-09-13T18:54:53.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.258+0000 7fa0c97fa700 1 -- 172.21.15.7:0/410376814 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa0cc06cfb0 con 0x7fa0cc0b38f0 2024-09-13T18:54:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.258+0000 7fa0d1a93700 1 -- 172.21.15.7:0/410376814 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa0b0066730 msgr2=0x7fa0b0068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.258+0000 7fa0d1a93700 1 --2- 172.21.15.7:0/410376814 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa0b0066730 0x7fa0b0068bf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4014040 tx=0x7fa0c401e040 comp rx=0 tx=0).stop 2024-09-13T18:54:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.258+0000 7fa0d1a93700 1 -- 172.21.15.7:0/410376814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0cc0b38f0 msgr2=0x7fa0cc15f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.258+0000 7fa0d1a93700 1 --2- 172.21.15.7:0/410376814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0cc0b38f0 0x7fa0cc15f340 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400c400 tx=0x7fa0b400c710 comp rx=0 tx=0).stop 2024-09-13T18:54:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.258+0000 7fa0d1a93700 1 -- 172.21.15.7:0/410376814 shutdown_connections 2024-09-13T18:54:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.258+0000 7fa0d1a93700 1 --2- 172.21.15.7:0/410376814 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fa0b0066730 0x7fa0b0068bf0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:53.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.258+0000 7fa0d1a93700 1 --2- 172.21.15.7:0/410376814 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fa0cc165760 0x7fa0cc154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.258+0000 7fa0d1a93700 1 --2- 172.21.15.7:0/410376814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0cc0b38f0 0x7fa0cc15f340 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.258+0000 7fa0d1a93700 1 --2- 172.21.15.7:0/410376814 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0cc06e390 0x7fa0cc15ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.258+0000 7fa0d1a93700 1 -- 172.21.15.7:0/410376814 >> 172.21.15.7:0/410376814 conn(0x7fa0cc0b1cd0 msgr2=0x7fa0cc0b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.258+0000 7fa0d1a93700 1 -- 172.21.15.7:0/410376814 shutdown_connections 2024-09-13T18:54:53.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.258+0000 7fa0d1a93700 1 -- 172.21.15.7:0/410376814 wait complete. 2024-09-13T18:54:53.280 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters' 2024-09-13T18:54:53.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.394+0000 7fc8290fe700 1 Processor -- start 2024-09-13T18:54:53.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.394+0000 7fc8290fe700 1 -- start start 2024-09-13T18:54:53.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.394+0000 7fc8290fe700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fc8240cb130 0x7fc82415d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:53.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.394+0000 7fc8290fe700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc82415d640 0x7fc824071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:53.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.394+0000 7fc8290fe700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8240cc860 con 0x7fc824071d50 2024-09-13T18:54:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.394+0000 7fc8290fe700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8240713f0 con 0x7fc8240cb130 2024-09-13T18:54:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.394+0000 7fc8290fe700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8240ce0e0 con 0x7fc82415d640 2024-09-13T18:54:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc8288fd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc824071d50 0x7fc8240b33d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57984/0 (socket says 172.21.15.7:57984) 2024-09-13T18:54:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc8288fd700 1 -- 172.21.15.7:0/3489894071 learned_addr learned my addr 172.21.15.7:0/3489894071 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc81bfff700 1 --2- 172.21.15.7:0/3489894071 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc82415d640 0x7fc824071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc823fff700 1 --2- 172.21.15.7:0/3489894071 >> v2:172.21.15.171:3300/0 conn(0x7fc8240cb130 0x7fc82415d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc8237fe700 1 -- 172.21.15.7:0/3489894071 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298196703 0 0) 0x7fc8240cc860 con 0x7fc824071d50 2024-09-13T18:54:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc8237fe700 1 -- 172.21.15.7:0/3489894071 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8100035d0 con 0x7fc824071d50 2024-09-13T18:54:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc81bfff700 1 -- 172.21.15.7:0/3489894071 >> v2:172.21.15.171:3300/0 conn(0x7fc8240cb130 msgr2=0x7fc82415d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:53.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc81bfff700 1 --2- 172.21.15.7:0/3489894071 >> v2:172.21.15.171:3300/0 conn(0x7fc8240cb130 0x7fc82415d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:53.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc81bfff700 1 -- 172.21.15.7:0/3489894071 >> v1:172.21.15.7:6789/0 conn(0x7fc824071d50 legacy=0x7fc8240b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:53.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc81bfff700 1 -- 172.21.15.7:0/3489894071 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8240cb510 con 0x7fc82415d640 2024-09-13T18:54:53.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc81bfff700 1 --2- 172.21.15.7:0/3489894071 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc82415d640 0x7fc824071810 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009850 tx=0x7fc80c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2912f0d090e98d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:53.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc8237fe700 1 -- 172.21.15.7:0/3489894071 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c011830 con 0x7fc82415d640 2024-09-13T18:54:53.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc8237fe700 1 -- 172.21.15.7:0/3489894071 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80c011990 con 0x7fc82415d640 2024-09-13T18:54:53.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc8237fe700 1 -- 172.21.15.7:0/3489894071 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c011d40 con 0x7fc82415d640 2024-09-13T18:54:53.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc8290fe700 1 -- 172.21.15.7:0/3489894071 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc82415d640 msgr2=0x7fc824071810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:53.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc8290fe700 1 --2- 172.21.15.7:0/3489894071 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc82415d640 0x7fc824071810 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009850 tx=0x7fc80c009b60 comp rx=0 tx=0).stop 2024-09-13T18:54:53.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc8290fe700 1 -- 172.21.15.7:0/3489894071 shutdown_connections 2024-09-13T18:54:53.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc8290fe700 1 --2- 172.21.15.7:0/3489894071 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc82415d640 0x7fc824071810 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:53.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc8290fe700 1 --2- 172.21.15.7:0/3489894071 >> v2:172.21.15.171:3300/0 conn(0x7fc8240cb130 0x7fc82415d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:53.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc8290fe700 1 -- 172.21.15.7:0/3489894071 >> 172.21.15.7:0/3489894071 conn(0x7fc8240b0f00 msgr2=0x7fc8240b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:53.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc8290fe700 1 -- 172.21.15.7:0/3489894071 shutdown_connections 2024-09-13T18:54:53.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.398+0000 7fc8290fe700 1 -- 172.21.15.7:0/3489894071 wait complete. 2024-09-13T18:54:53.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc8290fe700 1 Processor -- start 2024-09-13T18:54:53.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc8290fe700 1 -- start start 2024-09-13T18:54:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc8290fe700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc824071d50 0x7fc8240c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc8290fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8240cb130 0x7fc8240c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc8290fe700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc82415d640 0x7fc82415c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc8290fe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8240c3890 con 0x7fc8240cb130 2024-09-13T18:54:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc8290fe700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8241f5b90 con 0x7fc82415d640 2024-09-13T18:54:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc8290fe700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8241f5e70 con 0x7fc824071d50 2024-09-13T18:54:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc81bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8240cb130 0x7fc8240c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:53.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc81bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8240cb130 0x7fc8240c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40768/0 (socket says 172.21.15.7:40768) 2024-09-13T18:54:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc81bfff700 1 -- 172.21.15.7:0/1870834767 learned_addr learned my addr 172.21.15.7:0/1870834767 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc823fff700 1 --2- 172.21.15.7:0/1870834767 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc824071d50 0x7fc8240c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc8288fd700 1 --2- 172.21.15.7:0/1870834767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc82415d640 0x7fc82415c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc81bfff700 1 -- 172.21.15.7:0/1870834767 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc824071d50 msgr2=0x7fc8240c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc81bfff700 1 --2- 172.21.15.7:0/1870834767 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc824071d50 0x7fc8240c52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc81bfff700 1 -- 172.21.15.7:0/1870834767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc82415d640 msgr2=0x7fc82415c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc81bfff700 1 --2- 172.21.15.7:0/1870834767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc82415d640 0x7fc82415c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc81bfff700 1 -- 172.21.15.7:0/1870834767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80c0094b0 con 0x7fc8240cb130 2024-09-13T18:54:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc8288fd700 1 --2- 172.21.15.7:0/1870834767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc82415d640 0x7fc82415c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc81bfff700 1 --2- 172.21.15.7:0/1870834767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8240cb130 0x7fc8240c8090 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc80c014040 tx=0x7fc80c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:53.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc8217fa700 1 -- 172.21.15.7:0/1870834767 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c004590 con 0x7fc8240cb130 2024-09-13T18:54:53.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc8217fa700 1 -- 172.21.15.7:0/1870834767 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80c01e070 con 0x7fc8240cb130 2024-09-13T18:54:53.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc8217fa700 1 -- 172.21.15.7:0/1870834767 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c003900 con 0x7fc8240cb130 2024-09-13T18:54:53.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc8290fe700 1 -- 172.21.15.7:0/1870834767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8241f6150 con 0x7fc8240cb130 2024-09-13T18:54:53.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.402+0000 7fc8290fe700 1 -- 172.21.15.7:0/1870834767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8240c5800 con 0x7fc8240cb130 2024-09-13T18:54:53.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.406+0000 7fc8290fe700 1 -- 172.21.15.7:0/1870834767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc824070780 con 0x7fc8240cb130 2024-09-13T18:54:53.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.406+0000 7fc8217fa700 1 -- 172.21.15.7:0/1870834767 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc80c003a60 con 0x7fc8240cb130 2024-09-13T18:54:53.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.406+0000 7fc8217fa700 1 --2- 172.21.15.7:0/1870834767 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc804066730 0x7fc804068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:53.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.406+0000 7fc8217fa700 1 -- 172.21.15.7:0/1870834767 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fc80c020070 con 0x7fc8240cb130 2024-09-13T18:54:53.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.410+0000 7fc823fff700 1 --2- 172.21.15.7:0/1870834767 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc804066730 0x7fc804068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:53.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.410+0000 7fc823fff700 1 --2- 172.21.15.7:0/1870834767 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc804066730 0x7fc804068bf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc814009bf0 tx=0x7fc814009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:53.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.414+0000 7fc8217fa700 1 -- 172.21.15.7:0/1870834767 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc80c052910 con 0x7fc8240cb130 2024-09-13T18:54:53.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.654+0000 7fc8290fe700 1 -- 172.21.15.7:0/1870834767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters"]} v 0) v1 -- 0x7fc82415d100 con 0x7fc8240cb130 2024-09-13T18:54:53.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.858+0000 7fc8217fa700 1 -- 172.21.15.7:0/1870834767 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_openfiletable.OpenFileTable.test_perf_counters"]}]=0 v152) v1 ==== 143+0+0 (secure 0 0 0) 0x7fc80c006ce0 con 0x7fc8240cb130 2024-09-13T18:54:53.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.862+0000 7fc8290fe700 1 -- 172.21.15.7:0/1870834767 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc804066730 msgr2=0x7fc804068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:53.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.862+0000 7fc8290fe700 1 --2- 172.21.15.7:0/1870834767 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc804066730 0x7fc804068bf0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc814009bf0 tx=0x7fc814009240 comp rx=0 tx=0).stop 2024-09-13T18:54:53.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.862+0000 7fc8290fe700 1 -- 172.21.15.7:0/1870834767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8240cb130 msgr2=0x7fc8240c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:53.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.862+0000 7fc8290fe700 1 --2- 172.21.15.7:0/1870834767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8240cb130 0x7fc8240c8090 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc80c014040 tx=0x7fc80c004770 comp rx=0 tx=0).stop 2024-09-13T18:54:53.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.862+0000 7fc8290fe700 1 -- 172.21.15.7:0/1870834767 shutdown_connections 2024-09-13T18:54:53.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.862+0000 7fc8290fe700 1 --2- 172.21.15.7:0/1870834767 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc804066730 0x7fc804068bf0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:53.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.862+0000 7fc8290fe700 1 --2- 172.21.15.7:0/1870834767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc82415d640 0x7fc82415c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:53.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.862+0000 7fc8290fe700 1 --2- 172.21.15.7:0/1870834767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8240cb130 0x7fc8240c8090 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:53.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.862+0000 7fc8290fe700 1 --2- 172.21.15.7:0/1870834767 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc824071d50 0x7fc8240c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:53.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.862+0000 7fc8290fe700 1 -- 172.21.15.7:0/1870834767 >> 172.21.15.7:0/1870834767 conn(0x7fc8240b0f00 msgr2=0x7fc8240b19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:53.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.862+0000 7fc8290fe700 1 -- 172.21.15.7:0/1870834767 shutdown_connections 2024-09-13T18:54:53.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:53.862+0000 7fc8290fe700 1 -- 172.21.15.7:0/1870834767 wait complete. 2024-09-13T18:54:53.883 INFO:tasks.cephfs_test_runner:Opening a file should increment omap_total_updates by 1. ... ok 2024-09-13T18:54:53.883 INFO:tasks.cephfs_test_runner: 2024-09-13T18:54:53.883 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-13T18:54:53.884 INFO:tasks.cephfs_test_runner:Ran 2 tests in 126.935s 2024-09-13T18:54:53.884 INFO:tasks.cephfs_test_runner: 2024-09-13T18:54:53.884 INFO:tasks.cephfs_test_runner:OK 2024-09-13T18:54:53.884 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-13T18:54:53.893 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-13T18:54:53.944 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-13T18:54:53.944 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:54:53.944 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:54:53.953 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16349432k,nr_inodes=4087358,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276116k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276112k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276116k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T18:54:53.953 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-13T18:54:53.953 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:54:53.960 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16349432k,nr_inodes=4087358,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276116k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17512 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276112k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276116k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-13T18:54:53.960 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:54:53.960 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:54:53.968 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346336k,nr_inodes=4086584,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18533 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T18:54:53.968 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-13T18:54:53.968 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T18:54:53.974 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,noexec,relatime,size=16346336k,nr_inodes=4086584,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 tmpfs /sys/fs/cgroup tmpfs ro,nosuid,nodev,noexec,mode=755 0 0 cgroup2 /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18533 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275492k,mode=700,uid=1000,gid=1267 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275496k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-13T18:54:53.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:53.975 DEBUG:teuthology.orchestra.run.smithi171:> ip netns list 2024-09-13T18:54:53.982 INFO:teuthology.orchestra.run.smithi171.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-13T18:54:53.982 INFO:teuthology.orchestra.run.smithi171.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-13T18:54:53.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:53.982 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-13T18:54:53.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:53.997 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T18:54:54.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:54.009 DEBUG:teuthology.orchestra.run.smithi171:> sudo ip link delete ceph-brx 2024-09-13T18:54:54.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:54.147 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-13T18:54:54.154 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-13T18:54:54.154 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T18:54:54.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:54.155 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-13T18:54:54.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:54.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T18:54:54.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T18:54:54.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-09-13T18:54:54.376 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T18:54:54.386 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T18:54:54.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:54:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.498+0000 7f9b1b59e700 1 Processor -- start 2024-09-13T18:54:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.498+0000 7f9b1b59e700 1 -- start start 2024-09-13T18:54:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.498+0000 7f9b1b59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9b1c0b3ae0 0x7f9b1c06cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.498+0000 7f9b1b59e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f9b1c06aa90 0x7f9b1c0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.498+0000 7f9b1b59e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b1c06af40 con 0x7f9b1c159720 2024-09-13T18:54:54.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.498+0000 7f9b1b59e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1c06d690 con 0x7f9b1c06aa90 2024-09-13T18:54:54.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.498+0000 7f9b1b59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c0cbfe0 con 0x7f9b1c0b3ae0 2024-09-13T18:54:54.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b1a59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9b1c159720 0x7f9b1c0b33d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57986/0 (socket says 172.21.15.7:57986) 2024-09-13T18:54:54.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b1a59c700 1 -- 172.21.15.7:0/856919209 learned_addr learned my addr 172.21.15.7:0/856919209 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:54.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b19d9b700 1 --2- 172.21.15.7:0/856919209 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9b1c0b3ae0 0x7f9b1c06cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:54.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b1ad9d700 1 --2- 172.21.15.7:0/856919209 >> v2:172.21.15.171:3300/0 conn(0x7f9b1c06aa90 0x7f9b1c0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:54.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b1959a700 1 -- 172.21.15.7:0/856919209 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219227295 0 0) 0x7f9b1c06af40 con 0x7f9b1c159720 2024-09-13T18:54:54.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b1959a700 1 -- 172.21.15.7:0/856919209 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b040057e0 con 0x7f9b1c159720 2024-09-13T18:54:54.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b19d9b700 1 -- 172.21.15.7:0/856919209 >> v2:172.21.15.171:3300/0 conn(0x7f9b1c06aa90 msgr2=0x7f9b1c0ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:54.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b19d9b700 1 --2- 172.21.15.7:0/856919209 >> v2:172.21.15.171:3300/0 conn(0x7f9b1c06aa90 0x7f9b1c0ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:54.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b19d9b700 1 -- 172.21.15.7:0/856919209 >> v1:172.21.15.7:6789/0 conn(0x7f9b1c159720 legacy=0x7f9b1c0b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:54.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b19d9b700 1 -- 172.21.15.7:0/856919209 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c0cc2c0 con 0x7f9b1c0b3ae0 2024-09-13T18:54:54.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b19d9b700 1 --2- 172.21.15.7:0/856919209 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9b1c0b3ae0 0x7f9b1c06cf90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009850 tx=0x7f9b0c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32e7a0b53e320c7b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:54.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b1959a700 1 -- 172.21.15.7:0/856919209 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c011830 con 0x7f9b1c0b3ae0 2024-09-13T18:54:54.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b1959a700 1 -- 172.21.15.7:0/856919209 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0c011990 con 0x7f9b1c0b3ae0 2024-09-13T18:54:54.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b1959a700 1 -- 172.21.15.7:0/856919209 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c011d00 con 0x7f9b1c0b3ae0 2024-09-13T18:54:54.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b1b59e700 1 -- 172.21.15.7:0/856919209 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9b1c0b3ae0 msgr2=0x7f9b1c06cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:54.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b1b59e700 1 --2- 172.21.15.7:0/856919209 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9b1c0b3ae0 0x7f9b1c06cf90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009850 tx=0x7f9b0c009b60 comp rx=0 tx=0).stop 2024-09-13T18:54:54.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b1b59e700 1 -- 172.21.15.7:0/856919209 shutdown_connections 2024-09-13T18:54:54.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b1b59e700 1 --2- 172.21.15.7:0/856919209 >> v2:172.21.15.171:3300/0 conn(0x7f9b1c06aa90 0x7f9b1c0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:54.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b1b59e700 1 --2- 172.21.15.7:0/856919209 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9b1c0b3ae0 0x7f9b1c06cf90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:54.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b1b59e700 1 -- 172.21.15.7:0/856919209 >> 172.21.15.7:0/856919209 conn(0x7f9b1c0b0f00 msgr2=0x7f9b1c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:54.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b1b59e700 1 -- 172.21.15.7:0/856919209 shutdown_connections 2024-09-13T18:54:54.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.502+0000 7f9b1b59e700 1 -- 172.21.15.7:0/856919209 wait complete. 2024-09-13T18:54:54.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b1b59e700 1 Processor -- start 2024-09-13T18:54:54.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b1b59e700 1 -- start start 2024-09-13T18:54:54.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b1b59e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b1c06aa90 0x7f9b1c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:54.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b1b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b1c0b3ae0 0x7f9b1c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:54.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b1b59e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9b1c159720 0x7f9b1c152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:54.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b1b59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c1564e0 con 0x7f9b1c0b3ae0 2024-09-13T18:54:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b1b59e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c1544b0 con 0x7f9b1c06aa90 2024-09-13T18:54:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b1b59e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c155e90 con 0x7f9b1c159720 2024-09-13T18:54:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b19d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b1c0b3ae0 0x7f9b1c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b19d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b1c0b3ae0 0x7f9b1c15a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40776/0 (socket says 172.21.15.7:40776) 2024-09-13T18:54:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b19d9b700 1 -- 172.21.15.7:0/2600839186 learned_addr learned my addr 172.21.15.7:0/2600839186 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b1a59c700 1 --2- 172.21.15.7:0/2600839186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b1c06aa90 0x7f9b1c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b1ad9d700 1 --2- 172.21.15.7:0/2600839186 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9b1c159720 0x7f9b1c152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b19d9b700 1 -- 172.21.15.7:0/2600839186 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9b1c159720 msgr2=0x7f9b1c152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b19d9b700 1 --2- 172.21.15.7:0/2600839186 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9b1c159720 0x7f9b1c152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b19d9b700 1 -- 172.21.15.7:0/2600839186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b1c06aa90 msgr2=0x7f9b1c153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b19d9b700 1 --2- 172.21.15.7:0/2600839186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b1c06aa90 0x7f9b1c153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b19d9b700 1 -- 172.21.15.7:0/2600839186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c0094b0 con 0x7f9b1c0b3ae0 2024-09-13T18:54:54.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b19d9b700 1 --2- 172.21.15.7:0/2600839186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b1c0b3ae0 0x7f9b1c15a930 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0090b0 tx=0x7f9b0c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:54.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b0affd700 1 -- 172.21.15.7:0/2600839186 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c0044d0 con 0x7f9b1c0b3ae0 2024-09-13T18:54:54.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b0affd700 1 -- 172.21.15.7:0/2600839186 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0c01f070 con 0x7f9b1c0b3ae0 2024-09-13T18:54:54.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b1b59e700 1 -- 172.21.15.7:0/2600839186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1c156170 con 0x7f9b1c0b3ae0 2024-09-13T18:54:54.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b1b59e700 1 -- 172.21.15.7:0/2600839186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1c155ab0 con 0x7f9b1c0b3ae0 2024-09-13T18:54:54.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.506+0000 7f9b0affd700 1 -- 172.21.15.7:0/2600839186 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c0038c0 con 0x7f9b1c0b3ae0 2024-09-13T18:54:54.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.510+0000 7f9b1b59e700 1 -- 172.21.15.7:0/2600839186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b1c0ccac0 con 0x7f9b1c0b3ae0 2024-09-13T18:54:54.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.510+0000 7f9b0affd700 1 -- 172.21.15.7:0/2600839186 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f9b0c003a20 con 0x7f9b1c0b3ae0 2024-09-13T18:54:54.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.510+0000 7f9b0affd700 1 --2- 172.21.15.7:0/2600839186 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9afc066730 0x7f9afc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:54.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.510+0000 7f9b0affd700 1 -- 172.21.15.7:0/2600839186 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f9b0c021070 con 0x7f9b1c0b3ae0 2024-09-13T18:54:54.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.514+0000 7f9b1a59c700 1 --2- 172.21.15.7:0/2600839186 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9afc066730 0x7f9afc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:54.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.514+0000 7f9b1a59c700 1 --2- 172.21.15.7:0/2600839186 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9afc066730 0x7f9afc068bf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9b10007fd0 tx=0x7f9b10009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:54.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.518+0000 7f9b0affd700 1 -- 172.21.15.7:0/2600839186 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9b0c053150 con 0x7f9b1c0b3ae0 2024-09-13T18:54:54.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.686+0000 7f9b1b59e700 1 -- 172.21.15.7:0/2600839186 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9b1c159b00 con 0x7f9afc066730 2024-09-13T18:54:54.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.690+0000 7f9b0affd700 1 -- 172.21.15.7:0/2600839186 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49300 (secure 0 0 0) 0x7f9b10001d70 con 0x7f9afc066730 2024-09-13T18:54:54.697 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:54.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.694+0000 7f9b1b59e700 1 -- 172.21.15.7:0/2600839186 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9afc066730 msgr2=0x7f9afc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:54.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.694+0000 7f9b1b59e700 1 --2- 172.21.15.7:0/2600839186 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9afc066730 0x7f9afc068bf0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9b10007fd0 tx=0x7f9b10009040 comp rx=0 tx=0).stop 2024-09-13T18:54:54.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.694+0000 7f9b1b59e700 1 -- 172.21.15.7:0/2600839186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b1c0b3ae0 msgr2=0x7f9b1c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:54.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.694+0000 7f9b1b59e700 1 --2- 172.21.15.7:0/2600839186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b1c0b3ae0 0x7f9b1c15a930 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0090b0 tx=0x7f9b0c004770 comp rx=0 tx=0).stop 2024-09-13T18:54:54.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.694+0000 7f9b1b59e700 1 -- 172.21.15.7:0/2600839186 shutdown_connections 2024-09-13T18:54:54.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.694+0000 7f9b1b59e700 1 --2- 172.21.15.7:0/2600839186 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f9afc066730 0x7f9afc068bf0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:54.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.694+0000 7f9b1b59e700 1 --2- 172.21.15.7:0/2600839186 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f9b1c159720 0x7f9b1c152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:54.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.694+0000 7f9b1b59e700 1 --2- 172.21.15.7:0/2600839186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b1c0b3ae0 0x7f9b1c15a930 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:54.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.694+0000 7f9b1b59e700 1 --2- 172.21.15.7:0/2600839186 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b1c06aa90 0x7f9b1c153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:54.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.694+0000 7f9b1b59e700 1 -- 172.21.15.7:0/2600839186 >> 172.21.15.7:0/2600839186 conn(0x7f9b1c0b0f00 msgr2=0x7f9b1c0afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:54.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.694+0000 7f9b1b59e700 1 -- 172.21.15.7:0/2600839186 shutdown_connections 2024-09-13T18:54:54.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.694+0000 7f9b1b59e700 1 -- 172.21.15.7:0/2600839186 wait complete. 2024-09-13T18:54:54.700 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-13T18:54:54.712 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2024-09-13T18:54:54.264149+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247224,"kb_used_data":6488,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754727496,"statfs":{"total":773094113280,"available":772840955904,"internally_reserved":0,"allocated":6643712,"data_stored":3461972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":246402617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8998,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-79,"num_read_kb":-83,"num_write":-71,"num_write_kb":-32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002122"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":188,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T18:54:17.415803+0000","last_change":"2024-09-13T18:52:15.080080+0000","last_active":"2024-09-13T18:54:17.415803+0000","last_peered":"2024-09-13T18:54:17.415803+0000","last_clean":"2024-09-13T18:54:17.415803+0000","last_became_active":"2024-09-13T18:52:15.078100+0000","last_became_peered":"2024-09-13T18:52:15.078100+0000","last_unstale":"2024-09-13T18:54:17.415803+0000","last_undegraded":"2024-09-13T18:54:17.415803+0000","last_fullsized":"2024-09-13T18:54:17.415803+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_clean_scrub_stamp":"2024-09-13T18:52:13.196473+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:29:59.452757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542179,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1269760,"data_stored":873363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.85999999999999999,"15min":0.85499999999999998},"min":{"1min":0.58899999999999997,"5min":0.311,"15min":0.311},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.83199999999999996,"15min":0.84099999999999997},"min":{"1min":0.48299999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1,"5min":1.161,"15min":1.161},"last":0.79300000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.83299999999999996,"15min":0.82599999999999996},"min":{"1min":0.53400000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.49099999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.161,"5min":1.2,"15min":1.2},"last":0.873}]},{"osd":2,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.90300000000000002,"15min":0.91500000000000004},"min":{"1min":0.52500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.022,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.90600000000000003,"15min":0.91800000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1439999999999999,"5min":1.232,"15min":1.232},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.90900000000000003,"15min":0.91200000000000003},"min":{"1min":0.65000000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94899999999999995,"15min":0.95699999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2589999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.0509999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.89600000000000002,"15min":0.89700000000000002},"min":{"1min":0.50800000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.998},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0309999999999999,"15min":1.0389999999999999},"min":{"1min":0.65000000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":1.0469999999999999,"15min":1.069},"min":{"1min":0.60499999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2130000000000001,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.998,"5min":1.0269999999999999,"15min":1.032},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3260000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.137}]},{"osd":6,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.095,"15min":1.103},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.379,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.194},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.127,"15min":1.139},"min":{"1min":0.76500000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.75800000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542179,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":684032,"data_stored":283584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.91900000000000004,"15min":0.93100000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.171,"5min":1.706,"15min":1.706},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.90800000000000003,"15min":0.90200000000000002},"min":{"1min":0.56299999999999994,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2869999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.94599999999999995,"15min":0.95799999999999996},"min":{"1min":0.60599999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.153,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.91700000000000004,"15min":0.89900000000000002},"min":{"1min":0.66400000000000003,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.371,"5min":1.643,"15min":1.643},"last":0.81499999999999995}]},{"osd":2,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.84999999999999998,"15min":0.84099999999999997},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.96099999999999997,"15min":0.96299999999999997},"min":{"1min":0.78300000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.196,"5min":1.579,"15min":1.579},"last":0.97099999999999997}]},{"osd":3,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.92400000000000004,"15min":0.91800000000000004},"min":{"1min":0.73199999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.444,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.91700000000000004,"15min":0.90400000000000003},"min":{"1min":0.67700000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.754}]},{"osd":4,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.94999999999999996,"15min":0.95799999999999996},"min":{"1min":0.68200000000000005,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.171,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.871},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.002,"15min":1.004},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4870000000000001,"5min":1.879,"15min":1.879},"last":0.54600000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.753,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.1020000000000001,"15min":1.117},"min":{"1min":0.75700000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.2589999999999999,"5min":2.093,"15min":2.093},"last":0.69399999999999995}]},{"osd":7,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1799999999999999,"15min":1.1879999999999999},"min":{"1min":0.94499999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.5269999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.099,"15min":1.0960000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5840000000000001,"5min":1.843,"15min":1.843},"last":0.89800000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":282975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.91100000000000003,"15min":0.91500000000000004},"min":{"1min":0.58199999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.103,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.94599999999999995,"15min":0.94999999999999996},"min":{"1min":0.66600000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2529999999999999,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.89700000000000002,"15min":0.90500000000000003},"min":{"1min":0.59599999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.3440000000000001,"5min":2.145,"15min":2.145},"last":0.874},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.999,"15min":1.0029999999999999},"min":{"1min":0.70399999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.2330000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.0349999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.85499999999999998,"15min":0.84999999999999998},"min":{"1min":0.69399999999999995,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0760000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.93500000000000005,"15min":0.92700000000000005},"min":{"1min":0.65800000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.208,"5min":1.427,"15min":1.427},"last":0.58099999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.86599999999999999,"15min":0.872},"min":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0720000000000001,"5min":1.704,"15min":1.704},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.92500000000000004,"15min":0.91800000000000004},"min":{"1min":0.69299999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1479999999999999,"5min":1.335,"15min":1.335},"last":0.96799999999999997}]},{"osd":4,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.004,"15min":1.0269999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.198,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.032,"15min":1.044},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.3,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.85099999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0569999999999999,"15min":1.0649999999999999},"min":{"1min":0.69099999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.264,"5min":1.857,"15min":1.857},"last":1.125},{"interface":"front","average":{"1min":1.05,"5min":1.139,"15min":1.1539999999999999},"min":{"1min":0.78000000000000003,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.2749999999999999,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.93999999999999995}]},{"osd":7,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0780000000000001,"15min":1.0920000000000001},"min":{"1min":0.70099999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.2529999999999999,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.002},{"interface":"front","average":{"1min":1.131,"5min":1.236,"15min":1.254},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.325,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":1.0800000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542179,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":688128,"data_stored":291098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.76900000000000002,"15min":0.749},"min":{"1min":0.48999999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.81999999999999995,"15min":0.80900000000000005},"min":{"1min":0.66700000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.90000000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.84299999999999997,"15min":0.83799999999999997},"min":{"1min":0.63,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.82699999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.121,"5min":1.242,"15min":1.242},"last":1.0029999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.52400000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.274,"5min":1.694,"15min":1.694},"last":0.94999999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.91800000000000004,"15min":0.91300000000000003},"min":{"1min":0.76100000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.98099999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.86899999999999999,"15min":0.86199999999999999},"min":{"1min":0.68400000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.006,"5min":0.96599999999999997,"15min":0.95899999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3400000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.876}]},{"osd":6,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.997,"15min":1.0089999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0740000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0209999999999999,"15min":1.0169999999999999},"min":{"1min":0.79300000000000004,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.1830000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.036,"15min":1.0249999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.302,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.0960000000000001,"15min":1.095},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.093}]}]},{"osd":3,"up_from":14,"seq":60129542179,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":684032,"data_stored":283543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.51600000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.434},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93999999999999995,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66500000000000004,"15min":0.65500000000000003},"min":{"1min":0.47699999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.0169999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78000000000000003,"15min":0.78200000000000003},"min":{"1min":0.60699999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.147,"5min":1.181,"15min":1.181},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.82799999999999996,"15min":0.84799999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.98299999999999998,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.624},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.82299999999999995,"15min":0.81899999999999995},"min":{"1min":0.59799999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.167,"5min":1.571,"15min":1.571},"last":0.72299999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":1.0189999999999999,"15min":1.0309999999999999},"min":{"1min":0.78500000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.27,"5min":1.837,"15min":1.837},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.3680000000000001,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.80000000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.0620000000000001,"15min":1.0680000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.3700000000000001,"5min":1.893,"15min":1.893},"last":1.014},{"interface":"front","average":{"1min":1.083,"5min":1.115,"15min":1.1200000000000001},"min":{"1min":0.86199999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.325,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1180000000000001,"15min":1.1160000000000001},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.3959999999999999,"5min":1.421,"15min":1.421},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.04,"5min":1.163,"15min":1.1830000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.2130000000000001,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.98399999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.2529999999999999,"15min":1.278},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.3029999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.177,"5min":1.159,"15min":1.1559999999999999},"min":{"1min":0.89800000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.635,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.90600000000000003}]}]},{"osd":2,"up_from":14,"seq":60129542179,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35888,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335952,"statfs":{"total":96636764160,"available":96600014848,"internally_reserved":0,"allocated":692224,"data_stored":291120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0609999999999999,"5min":1.073,"15min":1.073},"last":0.443},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.83299999999999996,"15min":0.85099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.091,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.79800000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.78800000000000003},"min":{"1min":0.55500000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.98299999999999998,"5min":1.359,"15min":1.359},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.80700000000000005,"15min":0.81799999999999995},"min":{"1min":0.46500000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.042,"5min":1.696,"15min":1.696},"last":1.2629999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.66200000000000003,"15min":0.65200000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.91500000000000004,"15min":0.91700000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0900000000000001,"5min":1.288,"15min":1.288},"last":0.71599999999999997}]},{"osd":4,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0680000000000001,"15min":1.0780000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.1850000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.014,"5min":1.0640000000000001,"15min":1.0720000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.238,"5min":1.879,"15min":1.879},"last":1.022}]},{"osd":5,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0900000000000001,"15min":1.1040000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.387,"5min":1.796,"15min":1.796},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.038,"15min":1.0429999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2729999999999999,"5min":1.516,"15min":1.516},"last":0.91200000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0800000000000001,"15min":1.0920000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.286,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.999},{"interface":"front","average":{"1min":1.119,"5min":1.1719999999999999,"15min":1.181},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.97699999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.171,"15min":1.1910000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.3360000000000001,"5min":1.579,"15min":1.579},"last":1.101},{"interface":"front","average":{"1min":1.123,"5min":1.143,"15min":1.1459999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3580000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.171}]}]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31776,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":675840,"data_stored":282975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.51900000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86899999999999999,"5min":2.786,"15min":2.786},"last":0.625},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.629,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0009999999999999,"5min":1.159,"15min":1.159},"last":0.78800000000000003}]},{"osd":2,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.78100000000000003,"15min":0.79700000000000004},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2030000000000001,"5min":3.113,"15min":3.113},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.232,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.107}]},{"osd":3,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84599999999999997,"15min":0.84799999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.117,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81999999999999995,"15min":0.81899999999999995},"min":{"1min":0.626,"5min":0.502,"15min":0.502},"max":{"1min":1.1619999999999999,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.495}]},{"osd":4,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.99199999999999999,"15min":1.004},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2070000000000001,"5min":2.387,"15min":2.387},"last":1.014},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0589999999999999,"15min":1.0620000000000001},"min":{"1min":0.83799999999999997,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.51,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.272}]},{"osd":5,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0469999999999999,"15min":1.0529999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4139999999999999,"5min":1.419,"15min":1.419},"last":1.175},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0680000000000001,"15min":1.0860000000000001},"min":{"1min":0.74099999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2789999999999999,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":0.85599999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.135,"15min":1.139},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.6020000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.196},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.141,"15min":1.151},"min":{"1min":0.82899999999999996,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.5760000000000001,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.0640000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.2030000000000001,"15min":1.2150000000000001},"min":{"1min":0.89700000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.659,"5min":1.825,"15min":1.825},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.1599999999999999,"15min":1.1679999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.7350000000000001,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.1459999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542180,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1273856,"data_stored":873314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.95099999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68899999999999995,"15min":0.68799999999999994},"min":{"1min":0.438,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.83599999999999997,"15min":0.82399999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.66100000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.75900000000000001,"15min":0.74299999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.5859999999999999,"5min":2.5859999999999999,"15min":2.5859999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.79300000000000004,"15min":0.76900000000000002},"min":{"1min":0.42799999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.82799999999999996}]},{"osd":4,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.018,"15min":1.0169999999999999},"min":{"1min":0.68100000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.018,"15min":1.03},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.3009999999999999,"5min":1.579,"15min":1.579},"last":0.95199999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.97199999999999998,"15min":0.96599999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.073,"15min":1.075},"min":{"1min":0.68600000000000005,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.98899999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.083,"15min":1.093},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3839999999999999,"5min":1.504,"15min":1.504},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.044,"15min":1.0369999999999999},"min":{"1min":0.79800000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.2270000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1759999999999999,"15min":1.1839999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.452,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.022},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0860000000000001,"15min":1.081},"min":{"1min":0.64800000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.113}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:54:54.714 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:54:54.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.826+0000 7f1eef093700 1 Processor -- start 2024-09-13T18:54:54.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.826+0000 7f1eef093700 1 -- start start 2024-09-13T18:54:54.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eef093700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1ee815cae0 0x7f1ee80cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:54.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eef093700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f1ee815cec0 0x7f1ee80d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:54.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eef093700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee80d21e0 con 0x7f1ee806ac00 2024-09-13T18:54:54.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eef093700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee80d1750 con 0x7f1ee815cae0 2024-09-13T18:54:54.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eef093700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee80d1940 con 0x7f1ee815cec0 2024-09-13T18:54:54.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eee091700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1ee815cae0 0x7f1ee80cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:54.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eee892700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1ee806ac00 0x7f1ee80cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57990/0 (socket says 172.21.15.7:57990) 2024-09-13T18:54:54.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eee892700 1 -- 172.21.15.7:0/149084070 learned_addr learned my addr 172.21.15.7:0/149084070 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:54.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eed890700 1 --2- 172.21.15.7:0/149084070 >> v2:172.21.15.171:3300/0 conn(0x7f1ee815cec0 0x7f1ee80d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:54.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eed08f700 1 -- 172.21.15.7:0/149084070 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144925696 0 0) 0x7f1ee80d21e0 con 0x7f1ee806ac00 2024-09-13T18:54:54.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eed08f700 1 -- 172.21.15.7:0/149084070 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ed00035d0 con 0x7f1ee806ac00 2024-09-13T18:54:54.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eee091700 1 -- 172.21.15.7:0/149084070 >> v2:172.21.15.171:3300/0 conn(0x7f1ee815cec0 msgr2=0x7f1ee80d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:54.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eee091700 1 --2- 172.21.15.7:0/149084070 >> v2:172.21.15.171:3300/0 conn(0x7f1ee815cec0 0x7f1ee80d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:54.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eee091700 1 -- 172.21.15.7:0/149084070 >> v1:172.21.15.7:6789/0 conn(0x7f1ee806ac00 legacy=0x7f1ee80cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:54.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eee091700 1 -- 172.21.15.7:0/149084070 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee80d1c20 con 0x7f1ee815cae0 2024-09-13T18:54:54.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eee091700 1 --2- 172.21.15.7:0/149084070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ee815cae0 0x7f1ee80cf030 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009850 tx=0x7f1ed8009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3b0b32ddce1015 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:54.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eed08f700 1 -- 172.21.15.7:0/149084070 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8011830 con 0x7f1ee815cae0 2024-09-13T18:54:54.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eed08f700 1 -- 172.21.15.7:0/149084070 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed8011990 con 0x7f1ee815cae0 2024-09-13T18:54:54.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eed08f700 1 -- 172.21.15.7:0/149084070 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8011d20 con 0x7f1ee815cae0 2024-09-13T18:54:54.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eef093700 1 -- 172.21.15.7:0/149084070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ee815cae0 msgr2=0x7f1ee80cf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:54.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eef093700 1 --2- 172.21.15.7:0/149084070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ee815cae0 0x7f1ee80cf030 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009850 tx=0x7f1ed8009b60 comp rx=0 tx=0).stop 2024-09-13T18:54:54.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eef093700 1 -- 172.21.15.7:0/149084070 shutdown_connections 2024-09-13T18:54:54.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eef093700 1 --2- 172.21.15.7:0/149084070 >> v2:172.21.15.171:3300/0 conn(0x7f1ee815cec0 0x7f1ee80d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:54.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eef093700 1 --2- 172.21.15.7:0/149084070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ee815cae0 0x7f1ee80cf030 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:54.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eef093700 1 -- 172.21.15.7:0/149084070 >> 172.21.15.7:0/149084070 conn(0x7f1ee80d2c80 msgr2=0x7f1ee80d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:54.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eef093700 1 -- 172.21.15.7:0/149084070 shutdown_connections 2024-09-13T18:54:54.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.830+0000 7f1eef093700 1 -- 172.21.15.7:0/149084070 wait complete. 2024-09-13T18:54:54.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eef093700 1 Processor -- start 2024-09-13T18:54:54.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eef093700 1 -- start start 2024-09-13T18:54:54.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eef093700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1ee806ac00 0x7f1ee8159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:54.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eef093700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ee815cae0 0x7f1ee81587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:54.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eef093700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ee815cec0 0x7f1ee81e8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eef093700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee80cbf40 con 0x7f1ee815cae0 2024-09-13T18:54:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eef093700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee8158ce0 con 0x7f1ee815cec0 2024-09-13T18:54:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eef093700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee80cb610 con 0x7f1ee806ac00 2024-09-13T18:54:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eed890700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ee815cae0 0x7f1ee81587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eed890700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ee815cae0 0x7f1ee81587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40798/0 (socket says 172.21.15.7:40798) 2024-09-13T18:54:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eed890700 1 -- 172.21.15.7:0/505273374 learned_addr learned my addr 172.21.15.7:0/505273374 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eee091700 1 --2- 172.21.15.7:0/505273374 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1ee806ac00 0x7f1ee8159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eee892700 1 --2- 172.21.15.7:0/505273374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ee815cec0 0x7f1ee81e8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eed890700 1 -- 172.21.15.7:0/505273374 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1ee806ac00 msgr2=0x7f1ee8159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:54.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eed890700 1 --2- 172.21.15.7:0/505273374 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1ee806ac00 0x7f1ee8159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:54.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eed890700 1 -- 172.21.15.7:0/505273374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ee815cec0 msgr2=0x7f1ee81e8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:54.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eed890700 1 --2- 172.21.15.7:0/505273374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ee815cec0 0x7f1ee81e8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:54.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eed890700 1 -- 172.21.15.7:0/505273374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed80094b0 con 0x7f1ee815cae0 2024-09-13T18:54:54.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eed890700 1 --2- 172.21.15.7:0/505273374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ee815cae0 0x7f1ee81587a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400e9d0 tx=0x7f1ee400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:54.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1edeffd700 1 -- 172.21.15.7:0/505273374 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4009720 con 0x7f1ee815cae0 2024-09-13T18:54:54.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1edeffd700 1 -- 172.21.15.7:0/505273374 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee400ee70 con 0x7f1ee815cae0 2024-09-13T18:54:54.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eef093700 1 -- 172.21.15.7:0/505273374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee80cb8f0 con 0x7f1ee815cae0 2024-09-13T18:54:54.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1edeffd700 1 -- 172.21.15.7:0/505273374 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee400ca50 con 0x7f1ee815cae0 2024-09-13T18:54:54.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.834+0000 7f1eef093700 1 -- 172.21.15.7:0/505273374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee81e86d0 con 0x7f1ee815cae0 2024-09-13T18:54:54.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.838+0000 7f1eef093700 1 -- 172.21.15.7:0/505273374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee80cc0b0 con 0x7f1ee815cae0 2024-09-13T18:54:54.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.838+0000 7f1edeffd700 1 -- 172.21.15.7:0/505273374 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f1ee400cbb0 con 0x7f1ee815cae0 2024-09-13T18:54:54.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.838+0000 7f1edeffd700 1 --2- 172.21.15.7:0/505273374 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1ed4066730 0x7f1ed4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:54.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.838+0000 7f1edeffd700 1 -- 172.21.15.7:0/505273374 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f1ee4083ed0 con 0x7f1ee815cae0 2024-09-13T18:54:54.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.838+0000 7f1eee091700 1 --2- 172.21.15.7:0/505273374 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1ed4066730 0x7f1ed4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:54.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.838+0000 7f1eee091700 1 --2- 172.21.15.7:0/505273374 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1ed4066730 0x7f1ed4068bf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8000c00 tx=0x7f1ed8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:54.846+0000 7f1edeffd700 1 -- 172.21.15.7:0/505273374 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1ee40514d0 con 0x7f1ee815cae0 2024-09-13T18:54:55.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.014+0000 7f1eef093700 1 -- 172.21.15.7:0/505273374 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ee80b5d90 con 0x7f1ed4066730 2024-09-13T18:54:55.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.018+0000 7f1edeffd700 1 -- 172.21.15.7:0/505273374 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49300 (secure 0 0 0) 0x7f1ed8011b00 con 0x7f1ed4066730 2024-09-13T18:54:55.025 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:55.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.022+0000 7f1eef093700 1 -- 172.21.15.7:0/505273374 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1ed4066730 msgr2=0x7f1ed4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:55.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.022+0000 7f1eef093700 1 --2- 172.21.15.7:0/505273374 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1ed4066730 0x7f1ed4068bf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8000c00 tx=0x7f1ed8008030 comp rx=0 tx=0).stop 2024-09-13T18:54:55.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.022+0000 7f1eef093700 1 -- 172.21.15.7:0/505273374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ee815cae0 msgr2=0x7f1ee81587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:55.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.022+0000 7f1eef093700 1 --2- 172.21.15.7:0/505273374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ee815cae0 0x7f1ee81587a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400e9d0 tx=0x7f1ee400ece0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.022+0000 7f1eef093700 1 -- 172.21.15.7:0/505273374 shutdown_connections 2024-09-13T18:54:55.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.022+0000 7f1eef093700 1 --2- 172.21.15.7:0/505273374 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f1ed4066730 0x7f1ed4068bf0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.022+0000 7f1eef093700 1 --2- 172.21.15.7:0/505273374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1ee815cec0 0x7f1ee81e8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.022+0000 7f1eef093700 1 --2- 172.21.15.7:0/505273374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ee815cae0 0x7f1ee81587a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.022+0000 7f1eef093700 1 --2- 172.21.15.7:0/505273374 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f1ee806ac00 0x7f1ee8159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.022+0000 7f1eef093700 1 -- 172.21.15.7:0/505273374 >> 172.21.15.7:0/505273374 conn(0x7f1ee80d2c80 msgr2=0x7f1ee80b7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:55.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.022+0000 7f1eef093700 1 -- 172.21.15.7:0/505273374 shutdown_connections 2024-09-13T18:54:55.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.022+0000 7f1eef093700 1 -- 172.21.15.7:0/505273374 wait complete. 2024-09-13T18:54:55.030 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-13T18:54:55.045 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2024-09-13T18:54:54.264149+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247224,"kb_used_data":6488,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754727496,"statfs":{"total":773094113280,"available":772840955904,"internally_reserved":0,"allocated":6643712,"data_stored":3461972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":246402617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8998,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-79,"num_read_kb":-83,"num_write":-71,"num_write_kb":-32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002122"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":188,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T18:54:17.415803+0000","last_change":"2024-09-13T18:52:15.080080+0000","last_active":"2024-09-13T18:54:17.415803+0000","last_peered":"2024-09-13T18:54:17.415803+0000","last_clean":"2024-09-13T18:54:17.415803+0000","last_became_active":"2024-09-13T18:52:15.078100+0000","last_became_peered":"2024-09-13T18:52:15.078100+0000","last_unstale":"2024-09-13T18:54:17.415803+0000","last_undegraded":"2024-09-13T18:54:17.415803+0000","last_fullsized":"2024-09-13T18:54:17.415803+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_clean_scrub_stamp":"2024-09-13T18:52:13.196473+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:29:59.452757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542179,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1269760,"data_stored":873363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.85999999999999999,"15min":0.85499999999999998},"min":{"1min":0.58899999999999997,"5min":0.311,"15min":0.311},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.83199999999999996,"15min":0.84099999999999997},"min":{"1min":0.48299999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1,"5min":1.161,"15min":1.161},"last":0.79300000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.83299999999999996,"15min":0.82599999999999996},"min":{"1min":0.53400000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.49099999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.161,"5min":1.2,"15min":1.2},"last":0.873}]},{"osd":2,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.90300000000000002,"15min":0.91500000000000004},"min":{"1min":0.52500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.022,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.90600000000000003,"15min":0.91800000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1439999999999999,"5min":1.232,"15min":1.232},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.90900000000000003,"15min":0.91200000000000003},"min":{"1min":0.65000000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94899999999999995,"15min":0.95699999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2589999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.0509999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.89600000000000002,"15min":0.89700000000000002},"min":{"1min":0.50800000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.998},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0309999999999999,"15min":1.0389999999999999},"min":{"1min":0.65000000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":1.0469999999999999,"15min":1.069},"min":{"1min":0.60499999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2130000000000001,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.998,"5min":1.0269999999999999,"15min":1.032},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3260000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.137}]},{"osd":6,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.095,"15min":1.103},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.379,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.194},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.127,"15min":1.139},"min":{"1min":0.76500000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.75800000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542179,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":684032,"data_stored":283584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.91900000000000004,"15min":0.93100000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.171,"5min":1.706,"15min":1.706},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.90800000000000003,"15min":0.90200000000000002},"min":{"1min":0.56299999999999994,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2869999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.94599999999999995,"15min":0.95799999999999996},"min":{"1min":0.60599999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.153,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.91700000000000004,"15min":0.89900000000000002},"min":{"1min":0.66400000000000003,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.371,"5min":1.643,"15min":1.643},"last":0.81499999999999995}]},{"osd":2,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.84999999999999998,"15min":0.84099999999999997},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.96099999999999997,"15min":0.96299999999999997},"min":{"1min":0.78300000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.196,"5min":1.579,"15min":1.579},"last":0.97099999999999997}]},{"osd":3,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.92400000000000004,"15min":0.91800000000000004},"min":{"1min":0.73199999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.444,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.91700000000000004,"15min":0.90400000000000003},"min":{"1min":0.67700000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.754}]},{"osd":4,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.94999999999999996,"15min":0.95799999999999996},"min":{"1min":0.68200000000000005,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.171,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.871},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.002,"15min":1.004},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4870000000000001,"5min":1.879,"15min":1.879},"last":0.54600000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.753,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.1020000000000001,"15min":1.117},"min":{"1min":0.75700000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.2589999999999999,"5min":2.093,"15min":2.093},"last":0.69399999999999995}]},{"osd":7,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1799999999999999,"15min":1.1879999999999999},"min":{"1min":0.94499999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.5269999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.099,"15min":1.0960000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5840000000000001,"5min":1.843,"15min":1.843},"last":0.89800000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":282975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.91100000000000003,"15min":0.91500000000000004},"min":{"1min":0.58199999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.103,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.94599999999999995,"15min":0.94999999999999996},"min":{"1min":0.66600000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2529999999999999,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.89700000000000002,"15min":0.90500000000000003},"min":{"1min":0.59599999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.3440000000000001,"5min":2.145,"15min":2.145},"last":0.874},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.999,"15min":1.0029999999999999},"min":{"1min":0.70399999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.2330000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.0349999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.85499999999999998,"15min":0.84999999999999998},"min":{"1min":0.69399999999999995,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0760000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.93500000000000005,"15min":0.92700000000000005},"min":{"1min":0.65800000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.208,"5min":1.427,"15min":1.427},"last":0.58099999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.86599999999999999,"15min":0.872},"min":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0720000000000001,"5min":1.704,"15min":1.704},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.92500000000000004,"15min":0.91800000000000004},"min":{"1min":0.69299999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1479999999999999,"5min":1.335,"15min":1.335},"last":0.96799999999999997}]},{"osd":4,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.004,"15min":1.0269999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.198,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.032,"15min":1.044},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.3,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.85099999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0569999999999999,"15min":1.0649999999999999},"min":{"1min":0.69099999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.264,"5min":1.857,"15min":1.857},"last":1.125},{"interface":"front","average":{"1min":1.05,"5min":1.139,"15min":1.1539999999999999},"min":{"1min":0.78000000000000003,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.2749999999999999,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.93999999999999995}]},{"osd":7,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0780000000000001,"15min":1.0920000000000001},"min":{"1min":0.70099999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.2529999999999999,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.002},{"interface":"front","average":{"1min":1.131,"5min":1.236,"15min":1.254},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.325,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":1.0800000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542179,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":688128,"data_stored":291098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.76900000000000002,"15min":0.749},"min":{"1min":0.48999999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.81999999999999995,"15min":0.80900000000000005},"min":{"1min":0.66700000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.90000000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.84299999999999997,"15min":0.83799999999999997},"min":{"1min":0.63,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.82699999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.121,"5min":1.242,"15min":1.242},"last":1.0029999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.52400000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.274,"5min":1.694,"15min":1.694},"last":0.94999999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.91800000000000004,"15min":0.91300000000000003},"min":{"1min":0.76100000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.98099999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.86899999999999999,"15min":0.86199999999999999},"min":{"1min":0.68400000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.006,"5min":0.96599999999999997,"15min":0.95899999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3400000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.876}]},{"osd":6,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.997,"15min":1.0089999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0740000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0209999999999999,"15min":1.0169999999999999},"min":{"1min":0.79300000000000004,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.1830000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.036,"15min":1.0249999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.302,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.0960000000000001,"15min":1.095},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.093}]}]},{"osd":3,"up_from":14,"seq":60129542179,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":684032,"data_stored":283543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.51600000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.434},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93999999999999995,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66500000000000004,"15min":0.65500000000000003},"min":{"1min":0.47699999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.0169999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78000000000000003,"15min":0.78200000000000003},"min":{"1min":0.60699999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.147,"5min":1.181,"15min":1.181},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.82799999999999996,"15min":0.84799999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.98299999999999998,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.624},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.82299999999999995,"15min":0.81899999999999995},"min":{"1min":0.59799999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.167,"5min":1.571,"15min":1.571},"last":0.72299999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":1.0189999999999999,"15min":1.0309999999999999},"min":{"1min":0.78500000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.27,"5min":1.837,"15min":1.837},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.3680000000000001,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.80000000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.0620000000000001,"15min":1.0680000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.3700000000000001,"5min":1.893,"15min":1.893},"last":1.014},{"interface":"front","average":{"1min":1.083,"5min":1.115,"15min":1.1200000000000001},"min":{"1min":0.86199999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.325,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1180000000000001,"15min":1.1160000000000001},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.3959999999999999,"5min":1.421,"15min":1.421},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.04,"5min":1.163,"15min":1.1830000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.2130000000000001,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.98399999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.2529999999999999,"15min":1.278},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.3029999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.177,"5min":1.159,"15min":1.1559999999999999},"min":{"1min":0.89800000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.635,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.90600000000000003}]}]},{"osd":2,"up_from":14,"seq":60129542179,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35888,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335952,"statfs":{"total":96636764160,"available":96600014848,"internally_reserved":0,"allocated":692224,"data_stored":291120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0609999999999999,"5min":1.073,"15min":1.073},"last":0.443},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.83299999999999996,"15min":0.85099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.091,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.79800000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.78800000000000003},"min":{"1min":0.55500000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.98299999999999998,"5min":1.359,"15min":1.359},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.80700000000000005,"15min":0.81799999999999995},"min":{"1min":0.46500000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.042,"5min":1.696,"15min":1.696},"last":1.2629999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.66200000000000003,"15min":0.65200000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.91500000000000004,"15min":0.91700000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0900000000000001,"5min":1.288,"15min":1.288},"last":0.71599999999999997}]},{"osd":4,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0680000000000001,"15min":1.0780000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.1850000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.014,"5min":1.0640000000000001,"15min":1.0720000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.238,"5min":1.879,"15min":1.879},"last":1.022}]},{"osd":5,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0900000000000001,"15min":1.1040000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.387,"5min":1.796,"15min":1.796},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.038,"15min":1.0429999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2729999999999999,"5min":1.516,"15min":1.516},"last":0.91200000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0800000000000001,"15min":1.0920000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.286,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.999},{"interface":"front","average":{"1min":1.119,"5min":1.1719999999999999,"15min":1.181},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.97699999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.171,"15min":1.1910000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.3360000000000001,"5min":1.579,"15min":1.579},"last":1.101},{"interface":"front","average":{"1min":1.123,"5min":1.143,"15min":1.1459999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3580000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.171}]}]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31776,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":675840,"data_stored":282975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.51900000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86899999999999999,"5min":2.786,"15min":2.786},"last":0.625},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.629,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0009999999999999,"5min":1.159,"15min":1.159},"last":0.78800000000000003}]},{"osd":2,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.78100000000000003,"15min":0.79700000000000004},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2030000000000001,"5min":3.113,"15min":3.113},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.232,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.107}]},{"osd":3,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84599999999999997,"15min":0.84799999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.117,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81999999999999995,"15min":0.81899999999999995},"min":{"1min":0.626,"5min":0.502,"15min":0.502},"max":{"1min":1.1619999999999999,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.495}]},{"osd":4,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.99199999999999999,"15min":1.004},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2070000000000001,"5min":2.387,"15min":2.387},"last":1.014},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0589999999999999,"15min":1.0620000000000001},"min":{"1min":0.83799999999999997,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.51,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.272}]},{"osd":5,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0469999999999999,"15min":1.0529999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4139999999999999,"5min":1.419,"15min":1.419},"last":1.175},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0680000000000001,"15min":1.0860000000000001},"min":{"1min":0.74099999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2789999999999999,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":0.85599999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.135,"15min":1.139},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.6020000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.196},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.141,"15min":1.151},"min":{"1min":0.82899999999999996,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.5760000000000001,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.0640000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.2030000000000001,"15min":1.2150000000000001},"min":{"1min":0.89700000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.659,"5min":1.825,"15min":1.825},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.1599999999999999,"15min":1.1679999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.7350000000000001,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.1459999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542180,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1273856,"data_stored":873314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.95099999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68899999999999995,"15min":0.68799999999999994},"min":{"1min":0.438,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.83599999999999997,"15min":0.82399999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.66100000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.75900000000000001,"15min":0.74299999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.5859999999999999,"5min":2.5859999999999999,"15min":2.5859999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.79300000000000004,"15min":0.76900000000000002},"min":{"1min":0.42799999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.82799999999999996}]},{"osd":4,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.018,"15min":1.0169999999999999},"min":{"1min":0.68100000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.018,"15min":1.03},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.3009999999999999,"5min":1.579,"15min":1.579},"last":0.95199999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.97199999999999998,"15min":0.96599999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.073,"15min":1.075},"min":{"1min":0.68600000000000005,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.98899999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.083,"15min":1.093},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3839999999999999,"5min":1.504,"15min":1.504},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.044,"15min":1.0369999999999999},"min":{"1min":0.79800000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.2270000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1759999999999999,"15min":1.1839999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.452,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.022},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0860000000000001,"15min":1.081},"min":{"1min":0.64800000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.113}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:54:55.047 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T18:54:55.047 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:54:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.154+0000 7f4b7c88b700 1 Processor -- start 2024-09-13T18:54:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.154+0000 7f4b7c88b700 1 -- start start 2024-09-13T18:54:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.154+0000 7f4b7c88b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b7806baa0 0x7f4b7806ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.154+0000 7f4b7c88b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b780ce240 0x7f4b780ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.154+0000 7f4b7c88b700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b7815d160 con 0x7f4b7815cd80 2024-09-13T18:54:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.154+0000 7f4b7c88b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b780d1570 con 0x7f4b7806baa0 2024-09-13T18:54:55.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.154+0000 7f4b7c88b700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b780d25b0 con 0x7f4b780ce240 2024-09-13T18:54:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.154+0000 7f4b76ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b7806baa0 0x7f4b7806ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.154+0000 7f4b777fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4b7815cd80 0x7f4b7806a640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58004/0 (socket says 172.21.15.7:58004) 2024-09-13T18:54:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.154+0000 7f4b777fe700 1 -- 172.21.15.7:0/3998020961 learned_addr learned my addr 172.21.15.7:0/3998020961 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.154+0000 7f4b77fff700 1 --2- 172.21.15.7:0/3998020961 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b780ce240 0x7f4b780ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.154+0000 7f4b76ffd700 1 -- 172.21.15.7:0/3998020961 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b780ce240 msgr2=0x7f4b780ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.154+0000 7f4b76ffd700 1 --2- 172.21.15.7:0/3998020961 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b780ce240 0x7f4b780ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.154+0000 7f4b76ffd700 1 -- 172.21.15.7:0/3998020961 >> v1:172.21.15.7:6789/0 conn(0x7f4b7815cd80 legacy=0x7f4b7806a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.154+0000 7f4b76ffd700 1 -- 172.21.15.7:0/3998020961 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b780d2120 con 0x7f4b7806baa0 2024-09-13T18:54:55.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.158+0000 7f4b76ffd700 1 --2- 172.21.15.7:0/3998020961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7806baa0 0x7f4b7806ad50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009850 tx=0x7f4b68009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a6d37d9473aadf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:55.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.158+0000 7f4b767fc700 1 -- 172.21.15.7:0/3998020961 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68011830 con 0x7f4b7806baa0 2024-09-13T18:54:55.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.158+0000 7f4b767fc700 1 -- 172.21.15.7:0/3998020961 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b68011990 con 0x7f4b7806baa0 2024-09-13T18:54:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.158+0000 7f4b767fc700 1 -- 172.21.15.7:0/3998020961 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68011d40 con 0x7f4b7806baa0 2024-09-13T18:54:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.158+0000 7f4b7c88b700 1 -- 172.21.15.7:0/3998020961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7806baa0 msgr2=0x7f4b7806ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.158+0000 7f4b7c88b700 1 --2- 172.21.15.7:0/3998020961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7806baa0 0x7f4b7806ad50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009850 tx=0x7f4b68009b60 comp rx=0 tx=0).stop 2024-09-13T18:54:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.158+0000 7f4b7c88b700 1 -- 172.21.15.7:0/3998020961 shutdown_connections 2024-09-13T18:54:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.158+0000 7f4b7c88b700 1 --2- 172.21.15.7:0/3998020961 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b780ce240 0x7f4b780ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.158+0000 7f4b7c88b700 1 --2- 172.21.15.7:0/3998020961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7806baa0 0x7f4b7806ad50 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.158+0000 7f4b7c88b700 1 -- 172.21.15.7:0/3998020961 >> 172.21.15.7:0/3998020961 conn(0x7f4b780d30b0 msgr2=0x7f4b780d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.158+0000 7f4b7c88b700 1 -- 172.21.15.7:0/3998020961 shutdown_connections 2024-09-13T18:54:55.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.158+0000 7f4b7c88b700 1 -- 172.21.15.7:0/3998020961 wait complete. 2024-09-13T18:54:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b7c88b700 1 Processor -- start 2024-09-13T18:54:55.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b7c88b700 1 -- start start 2024-09-13T18:54:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b7c88b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7806baa0 0x7f4b780cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b7c88b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b780ce240 0x7f4b780c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b7c88b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b7815cd80 0x7f4b780c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b7c88b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b780c9bc0 con 0x7f4b7806baa0 2024-09-13T18:54:55.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b7c88b700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b780cc6f0 con 0x7f4b7815cd80 2024-09-13T18:54:55.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b7c88b700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7814bcc0 con 0x7f4b780ce240 2024-09-13T18:54:55.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7806baa0 0x7f4b780cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:55.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7806baa0 0x7f4b780cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40814/0 (socket says 172.21.15.7:40814) 2024-09-13T18:54:55.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b777fe700 1 -- 172.21.15.7:0/2291886532 learned_addr learned my addr 172.21.15.7:0/2291886532 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:55.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b77fff700 1 --2- 172.21.15.7:0/2291886532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b7815cd80 0x7f4b780c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:55.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b76ffd700 1 --2- 172.21.15.7:0/2291886532 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b780ce240 0x7f4b780c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:55.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b777fe700 1 -- 172.21.15.7:0/2291886532 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b780ce240 msgr2=0x7f4b780c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:55.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b777fe700 1 --2- 172.21.15.7:0/2291886532 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b780ce240 0x7f4b780c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b777fe700 1 -- 172.21.15.7:0/2291886532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b7815cd80 msgr2=0x7f4b780c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:55.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b777fe700 1 --2- 172.21.15.7:0/2291886532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b7815cd80 0x7f4b780c9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b777fe700 1 -- 172.21.15.7:0/2291886532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b680094b0 con 0x7f4b7806baa0 2024-09-13T18:54:55.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b777fe700 1 --2- 172.21.15.7:0/2291886532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7806baa0 0x7f4b780cb760 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000c410 tx=0x7f4b6000c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:55.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b6ffff700 1 -- 172.21.15.7:0/2291886532 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b60005d90 con 0x7f4b7806baa0 2024-09-13T18:54:55.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b6ffff700 1 -- 172.21.15.7:0/2291886532 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6000ce20 con 0x7f4b7806baa0 2024-09-13T18:54:55.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b7c88b700 1 -- 172.21.15.7:0/2291886532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b780cbe50 con 0x7f4b7806baa0 2024-09-13T18:54:55.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b6ffff700 1 -- 172.21.15.7:0/2291886532 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b600072f0 con 0x7f4b7806baa0 2024-09-13T18:54:55.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.162+0000 7f4b7c88b700 1 -- 172.21.15.7:0/2291886532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b780cc410 con 0x7f4b7806baa0 2024-09-13T18:54:55.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.166+0000 7f4b7c88b700 1 -- 172.21.15.7:0/2291886532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b780b6e50 con 0x7f4b7806baa0 2024-09-13T18:54:55.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.166+0000 7f4b6ffff700 1 -- 172.21.15.7:0/2291886532 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f4b60007450 con 0x7f4b7806baa0 2024-09-13T18:54:55.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.166+0000 7f4b6ffff700 1 --2- 172.21.15.7:0/2291886532 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4b58066760 0x7f4b58068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:55.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.166+0000 7f4b6ffff700 1 -- 172.21.15.7:0/2291886532 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f4b60083190 con 0x7f4b7806baa0 2024-09-13T18:54:55.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.166+0000 7f4b76ffd700 1 --2- 172.21.15.7:0/2291886532 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4b58066760 0x7f4b58068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:55.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.170+0000 7f4b76ffd700 1 --2- 172.21.15.7:0/2291886532 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4b58066760 0x7f4b58068c20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4b680090b0 tx=0x7f4b68008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:55.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.174+0000 7f4b6ffff700 1 -- 172.21.15.7:0/2291886532 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4b60050790 con 0x7f4b7806baa0 2024-09-13T18:54:55.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.346+0000 7f4b7c88b700 1 -- 172.21.15.7:0/2291886532 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4b780d1db0 con 0x7f4b58066760 2024-09-13T18:54:55.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.350+0000 7f4b6ffff700 1 -- 172.21.15.7:0/2291886532 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49300 (secure 0 0 0) 0x7f4b68011c00 con 0x7f4b58066760 2024-09-13T18:54:55.357 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.354+0000 7f4b7c88b700 1 -- 172.21.15.7:0/2291886532 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4b58066760 msgr2=0x7f4b58068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.354+0000 7f4b7c88b700 1 --2- 172.21.15.7:0/2291886532 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4b58066760 0x7f4b58068c20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4b680090b0 tx=0x7f4b68008030 comp rx=0 tx=0).stop 2024-09-13T18:54:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.354+0000 7f4b7c88b700 1 -- 172.21.15.7:0/2291886532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7806baa0 msgr2=0x7f4b780cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.354+0000 7f4b7c88b700 1 --2- 172.21.15.7:0/2291886532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7806baa0 0x7f4b780cb760 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4b6000c410 tx=0x7f4b6000c720 comp rx=0 tx=0).stop 2024-09-13T18:54:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.354+0000 7f4b7c88b700 1 -- 172.21.15.7:0/2291886532 shutdown_connections 2024-09-13T18:54:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.354+0000 7f4b7c88b700 1 --2- 172.21.15.7:0/2291886532 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f4b58066760 0x7f4b58068c20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.354+0000 7f4b7c88b700 1 --2- 172.21.15.7:0/2291886532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4b7815cd80 0x7f4b780c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.354+0000 7f4b7c88b700 1 --2- 172.21.15.7:0/2291886532 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f4b780ce240 0x7f4b780c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.354+0000 7f4b7c88b700 1 --2- 172.21.15.7:0/2291886532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b7806baa0 0x7f4b780cb760 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.354+0000 7f4b7c88b700 1 -- 172.21.15.7:0/2291886532 >> 172.21.15.7:0/2291886532 conn(0x7f4b780d30b0 msgr2=0x7f4b7806ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.354+0000 7f4b7c88b700 1 -- 172.21.15.7:0/2291886532 shutdown_connections 2024-09-13T18:54:55.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.354+0000 7f4b7c88b700 1 -- 172.21.15.7:0/2291886532 wait complete. 2024-09-13T18:54:55.360 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-13T18:54:55.372 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2024-09-13T18:54:54.264149+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247224,"kb_used_data":6488,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754727496,"statfs":{"total":773094113280,"available":772840955904,"internally_reserved":0,"allocated":6643712,"data_stored":3461972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":246402617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8998,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-79,"num_read_kb":-83,"num_write":-71,"num_write_kb":-32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002122"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":188,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T18:54:17.415803+0000","last_change":"2024-09-13T18:52:15.080080+0000","last_active":"2024-09-13T18:54:17.415803+0000","last_peered":"2024-09-13T18:54:17.415803+0000","last_clean":"2024-09-13T18:54:17.415803+0000","last_became_active":"2024-09-13T18:52:15.078100+0000","last_became_peered":"2024-09-13T18:52:15.078100+0000","last_unstale":"2024-09-13T18:54:17.415803+0000","last_undegraded":"2024-09-13T18:54:17.415803+0000","last_fullsized":"2024-09-13T18:54:17.415803+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_clean_scrub_stamp":"2024-09-13T18:52:13.196473+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:29:59.452757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542179,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28260,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343580,"statfs":{"total":96636764160,"available":96607825920,"internally_reserved":0,"allocated":1269760,"data_stored":873363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.85999999999999999,"15min":0.85499999999999998},"min":{"1min":0.58899999999999997,"5min":0.311,"15min":0.311},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.83199999999999996,"15min":0.84099999999999997},"min":{"1min":0.48299999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1,"5min":1.161,"15min":1.161},"last":0.79300000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.83299999999999996,"15min":0.82599999999999996},"min":{"1min":0.53400000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.49099999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.161,"5min":1.2,"15min":1.2},"last":0.873}]},{"osd":2,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.90300000000000002,"15min":0.91500000000000004},"min":{"1min":0.52500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.022,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.90600000000000003,"15min":0.91800000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1439999999999999,"5min":1.232,"15min":1.232},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.90900000000000003,"15min":0.91200000000000003},"min":{"1min":0.65000000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94899999999999995,"15min":0.95699999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2589999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.0509999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.89600000000000002,"15min":0.89700000000000002},"min":{"1min":0.50800000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.998},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0309999999999999,"15min":1.0389999999999999},"min":{"1min":0.65000000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":1.0469999999999999,"15min":1.069},"min":{"1min":0.60499999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2130000000000001,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.998,"5min":1.0269999999999999,"15min":1.032},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3260000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":1.137}]},{"osd":6,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.095,"15min":1.103},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.379,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.194},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.127,"15min":1.139},"min":{"1min":0.76500000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.75800000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542179,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":684032,"data_stored":283584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.91900000000000004,"15min":0.93100000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.171,"5min":1.706,"15min":1.706},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.90800000000000003,"15min":0.90200000000000002},"min":{"1min":0.56299999999999994,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2869999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.94599999999999995,"15min":0.95799999999999996},"min":{"1min":0.60599999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.153,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.91700000000000004,"15min":0.89900000000000002},"min":{"1min":0.66400000000000003,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.371,"5min":1.643,"15min":1.643},"last":0.81499999999999995}]},{"osd":2,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.84999999999999998,"15min":0.84099999999999997},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.96099999999999997,"15min":0.96299999999999997},"min":{"1min":0.78300000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.196,"5min":1.579,"15min":1.579},"last":0.97099999999999997}]},{"osd":3,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.92400000000000004,"15min":0.91800000000000004},"min":{"1min":0.73199999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.444,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.91700000000000004,"15min":0.90400000000000003},"min":{"1min":0.67700000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.754}]},{"osd":4,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.94999999999999996,"15min":0.95799999999999996},"min":{"1min":0.68200000000000005,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.171,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.871},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.002,"15min":1.004},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4870000000000001,"5min":1.879,"15min":1.879},"last":0.54600000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.753,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.1020000000000001,"15min":1.117},"min":{"1min":0.75700000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.2589999999999999,"5min":2.093,"15min":2.093},"last":0.69399999999999995}]},{"osd":7,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1799999999999999,"15min":1.1879999999999999},"min":{"1min":0.94499999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.5269999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.099,"15min":1.0960000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5840000000000001,"5min":1.843,"15min":1.843},"last":0.89800000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":282975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.91100000000000003,"15min":0.91500000000000004},"min":{"1min":0.58199999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.103,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.94599999999999995,"15min":0.94999999999999996},"min":{"1min":0.66600000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2529999999999999,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.89700000000000002,"15min":0.90500000000000003},"min":{"1min":0.59599999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.3440000000000001,"5min":2.145,"15min":2.145},"last":0.874},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.999,"15min":1.0029999999999999},"min":{"1min":0.70399999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.2330000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.0349999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.85499999999999998,"15min":0.84999999999999998},"min":{"1min":0.69399999999999995,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0760000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.93500000000000005,"15min":0.92700000000000005},"min":{"1min":0.65800000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.208,"5min":1.427,"15min":1.427},"last":0.58099999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.86599999999999999,"15min":0.872},"min":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0720000000000001,"5min":1.704,"15min":1.704},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.92500000000000004,"15min":0.91800000000000004},"min":{"1min":0.69299999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1479999999999999,"5min":1.335,"15min":1.335},"last":0.96799999999999997}]},{"osd":4,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.004,"15min":1.0269999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.198,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.032,"15min":1.044},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.3,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.85099999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0569999999999999,"15min":1.0649999999999999},"min":{"1min":0.69099999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.264,"5min":1.857,"15min":1.857},"last":1.125},{"interface":"front","average":{"1min":1.05,"5min":1.139,"15min":1.1539999999999999},"min":{"1min":0.78000000000000003,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.2749999999999999,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":0.93999999999999995}]},{"osd":7,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0780000000000001,"15min":1.0920000000000001},"min":{"1min":0.70099999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.2529999999999999,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.002},{"interface":"front","average":{"1min":1.131,"5min":1.236,"15min":1.254},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.325,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":1.0800000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542179,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":688128,"data_stored":291098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.76900000000000002,"15min":0.749},"min":{"1min":0.48999999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.81999999999999995,"15min":0.80900000000000005},"min":{"1min":0.66700000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.90000000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.84299999999999997,"15min":0.83799999999999997},"min":{"1min":0.63,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.82699999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.121,"5min":1.242,"15min":1.242},"last":1.0029999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.52400000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.274,"5min":1.694,"15min":1.694},"last":0.94999999999999996}]},{"osd":3,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.91800000000000004,"15min":0.91300000000000003},"min":{"1min":0.76100000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.98099999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.86899999999999999,"15min":0.86199999999999999},"min":{"1min":0.68400000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.006,"5min":0.96599999999999997,"15min":0.95899999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3400000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.876}]},{"osd":6,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.997,"15min":1.0089999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0740000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0209999999999999,"15min":1.0169999999999999},"min":{"1min":0.79300000000000004,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.1830000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.036,"15min":1.0249999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.302,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.0960000000000001,"15min":1.095},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.093}]}]},{"osd":3,"up_from":14,"seq":60129542179,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":684032,"data_stored":283543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.51600000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.434},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93999999999999995,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66500000000000004,"15min":0.65500000000000003},"min":{"1min":0.47699999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.0169999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78000000000000003,"15min":0.78200000000000003},"min":{"1min":0.60699999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.147,"5min":1.181,"15min":1.181},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.82799999999999996,"15min":0.84799999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.98299999999999998,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.624},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.82299999999999995,"15min":0.81899999999999995},"min":{"1min":0.59799999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.167,"5min":1.571,"15min":1.571},"last":0.72299999999999998}]},{"osd":4,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":1.0189999999999999,"15min":1.0309999999999999},"min":{"1min":0.78500000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.27,"5min":1.837,"15min":1.837},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.3680000000000001,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.80000000000000004}]},{"osd":5,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.0620000000000001,"15min":1.0680000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.3700000000000001,"5min":1.893,"15min":1.893},"last":1.014},{"interface":"front","average":{"1min":1.083,"5min":1.115,"15min":1.1200000000000001},"min":{"1min":0.86199999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.325,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.75600000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1180000000000001,"15min":1.1160000000000001},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.3959999999999999,"5min":1.421,"15min":1.421},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.04,"5min":1.163,"15min":1.1830000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.2130000000000001,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.98399999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.2529999999999999,"15min":1.278},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.3029999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.177,"5min":1.159,"15min":1.1559999999999999},"min":{"1min":0.89800000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.635,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.90600000000000003}]}]},{"osd":2,"up_from":14,"seq":60129542179,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35888,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335952,"statfs":{"total":96636764160,"available":96600014848,"internally_reserved":0,"allocated":692224,"data_stored":291120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0609999999999999,"5min":1.073,"15min":1.073},"last":0.443},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.83299999999999996,"15min":0.85099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.091,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.79800000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.78800000000000003},"min":{"1min":0.55500000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.98299999999999998,"5min":1.359,"15min":1.359},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.80700000000000005,"15min":0.81799999999999995},"min":{"1min":0.46500000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.042,"5min":1.696,"15min":1.696},"last":1.2629999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.66200000000000003,"15min":0.65200000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.91500000000000004,"15min":0.91700000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0900000000000001,"5min":1.288,"15min":1.288},"last":0.71599999999999997}]},{"osd":4,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0680000000000001,"15min":1.0780000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.1850000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.014,"5min":1.0640000000000001,"15min":1.0720000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.238,"5min":1.879,"15min":1.879},"last":1.022}]},{"osd":5,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0900000000000001,"15min":1.1040000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.387,"5min":1.796,"15min":1.796},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.038,"15min":1.0429999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2729999999999999,"5min":1.516,"15min":1.516},"last":0.91200000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0800000000000001,"15min":1.0920000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.286,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.999},{"interface":"front","average":{"1min":1.119,"5min":1.1719999999999999,"15min":1.181},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.97699999999999998}]},{"osd":7,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.171,"15min":1.1910000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.3360000000000001,"5min":1.579,"15min":1.579},"last":1.101},{"interface":"front","average":{"1min":1.123,"5min":1.143,"15min":1.1459999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3580000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.171}]}]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31776,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":675840,"data_stored":282975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.51900000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86899999999999999,"5min":2.786,"15min":2.786},"last":0.625},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.629,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0009999999999999,"5min":1.159,"15min":1.159},"last":0.78800000000000003}]},{"osd":2,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.78100000000000003,"15min":0.79700000000000004},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2030000000000001,"5min":3.113,"15min":3.113},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.232,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.107}]},{"osd":3,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84599999999999997,"15min":0.84799999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.117,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81999999999999995,"15min":0.81899999999999995},"min":{"1min":0.626,"5min":0.502,"15min":0.502},"max":{"1min":1.1619999999999999,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.495}]},{"osd":4,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.99199999999999999,"15min":1.004},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2070000000000001,"5min":2.387,"15min":2.387},"last":1.014},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0589999999999999,"15min":1.0620000000000001},"min":{"1min":0.83799999999999997,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.51,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.272}]},{"osd":5,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0469999999999999,"15min":1.0529999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4139999999999999,"5min":1.419,"15min":1.419},"last":1.175},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0680000000000001,"15min":1.0860000000000001},"min":{"1min":0.74099999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2789999999999999,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":0.85599999999999998}]},{"osd":6,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.135,"15min":1.139},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.6020000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.196},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.141,"15min":1.151},"min":{"1min":0.82899999999999996,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.5760000000000001,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.0640000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.2030000000000001,"15min":1.2150000000000001},"min":{"1min":0.89700000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.659,"5min":1.825,"15min":1.825},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.1599999999999999,"15min":1.1679999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.7350000000000001,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":1.1459999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542180,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1273856,"data_stored":873314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.95099999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68899999999999995,"15min":0.68799999999999994},"min":{"1min":0.438,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.83599999999999997,"15min":0.82399999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.66100000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.75900000000000001,"15min":0.74299999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.5859999999999999,"5min":2.5859999999999999,"15min":2.5859999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.79300000000000004,"15min":0.76900000000000002},"min":{"1min":0.42799999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.82799999999999996}]},{"osd":4,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.018,"15min":1.0169999999999999},"min":{"1min":0.68100000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.018,"15min":1.03},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.3009999999999999,"5min":1.579,"15min":1.579},"last":0.95199999999999996}]},{"osd":5,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.97199999999999998,"15min":0.96599999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.073,"15min":1.075},"min":{"1min":0.68600000000000005,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.98899999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.083,"15min":1.093},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3839999999999999,"5min":1.504,"15min":1.504},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.044,"15min":1.0369999999999999},"min":{"1min":0.79800000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.2270000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1759999999999999,"15min":1.1839999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.452,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.022},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0860000000000001,"15min":1.081},"min":{"1min":0.64800000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.113}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:54:55.373 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T18:54:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f037cdc2700 1 Processor -- start 2024-09-13T18:54:55.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f037cdc2700 1 -- start start 2024-09-13T18:54:55.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f037cdc2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f037815b890 0x7f0378162220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:55.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f037cdc2700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0378162760 con 0x7f03780cdcc0 2024-09-13T18:54:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f037cdc2700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f037815bc70 con 0x7f03780b2b00 2024-09-13T18:54:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f037cdc2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03780ccc20 con 0x7f037815b890 2024-09-13T18:54:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f037659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f037815b890 0x7f0378162220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f0375d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f03780cdcc0 0x7f03780b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58014/0 (socket says 172.21.15.7:58014) 2024-09-13T18:54:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f0375d9b700 1 -- 172.21.15.7:0/2919055507 learned_addr learned my addr 172.21.15.7:0/2919055507 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f037559a700 1 -- 172.21.15.7:0/2919055507 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265142516 0 0) 0x7f0378162760 con 0x7f03780cdcc0 2024-09-13T18:54:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f037559a700 1 -- 172.21.15.7:0/2919055507 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03600035d0 con 0x7f03780cdcc0 2024-09-13T18:54:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f037659c700 1 -- 172.21.15.7:0/2919055507 >> v1:172.21.15.171:6789/0 conn(0x7f03780b2b00 legacy=0x7f037806ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f037659c700 1 -- 172.21.15.7:0/2919055507 >> v1:172.21.15.7:6789/0 conn(0x7f03780cdcc0 legacy=0x7f03780b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:55.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f037659c700 1 -- 172.21.15.7:0/2919055507 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03780ccf00 con 0x7f037815b890 2024-09-13T18:54:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f037659c700 1 --2- 172.21.15.7:0/2919055507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037815b890 0x7f0378162220 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f036c009850 tx=0x7f036c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84499ba9e7f10ebb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f037559a700 1 -- 172.21.15.7:0/2919055507 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c011830 con 0x7f037815b890 2024-09-13T18:54:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f037559a700 1 -- 172.21.15.7:0/2919055507 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036c011990 con 0x7f037815b890 2024-09-13T18:54:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f037559a700 1 -- 172.21.15.7:0/2919055507 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c011d40 con 0x7f037815b890 2024-09-13T18:54:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f037cdc2700 1 -- 172.21.15.7:0/2919055507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037815b890 msgr2=0x7f0378162220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.478+0000 7f037cdc2700 1 --2- 172.21.15.7:0/2919055507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037815b890 0x7f0378162220 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f036c009850 tx=0x7f036c009b60 comp rx=0 tx=0).stop 2024-09-13T18:54:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.482+0000 7f037cdc2700 1 -- 172.21.15.7:0/2919055507 shutdown_connections 2024-09-13T18:54:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.482+0000 7f037cdc2700 1 --2- 172.21.15.7:0/2919055507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037815b890 0x7f0378162220 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.482+0000 7f037cdc2700 1 -- 172.21.15.7:0/2919055507 >> 172.21.15.7:0/2919055507 conn(0x7f03780ce980 msgr2=0x7f03780ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.482+0000 7f037cdc2700 1 -- 172.21.15.7:0/2919055507 shutdown_connections 2024-09-13T18:54:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.482+0000 7f037cdc2700 1 -- 172.21.15.7:0/2919055507 wait complete. 2024-09-13T18:54:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.482+0000 7f037cdc2700 1 Processor -- start 2024-09-13T18:54:55.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.482+0000 7f037cdc2700 1 -- start start 2024-09-13T18:54:55.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.482+0000 7f037cdc2700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03780b2b00 0x7f037814c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:55.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.482+0000 7f037cdc2700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f03780cdcc0 0x7f0378151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:55.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.482+0000 7f037cdc2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037815b890 0x7f0378164d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:55.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.482+0000 7f037cdc2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0378150940 con 0x7f037815b890 2024-09-13T18:54:55.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.482+0000 7f037cdc2700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03781526e0 con 0x7f03780b2b00 2024-09-13T18:54:55.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.482+0000 7f037cdc2700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03781f6b80 con 0x7f03780cdcc0 2024-09-13T18:54:55.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.482+0000 7f0376d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037815b890 0x7f0378164d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:55.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.482+0000 7f0376d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037815b890 0x7f0378164d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40824/0 (socket says 172.21.15.7:40824) 2024-09-13T18:54:55.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.482+0000 7f0376d9d700 1 -- 172.21.15.7:0/1203154622 learned_addr learned my addr 172.21.15.7:0/1203154622 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:55.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.482+0000 7f0375d9b700 1 --2- 172.21.15.7:0/1203154622 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f03780cdcc0 0x7f0378151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:55.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.482+0000 7f037659c700 1 --2- 172.21.15.7:0/1203154622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03780b2b00 0x7f037814c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:55.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.486+0000 7f0376d9d700 1 -- 172.21.15.7:0/1203154622 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f03780cdcc0 msgr2=0x7f0378151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:55.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.486+0000 7f0376d9d700 1 --2- 172.21.15.7:0/1203154622 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f03780cdcc0 0x7f0378151d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.486+0000 7f0376d9d700 1 -- 172.21.15.7:0/1203154622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03780b2b00 msgr2=0x7f037814c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:55.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.486+0000 7f0376d9d700 1 --2- 172.21.15.7:0/1203154622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03780b2b00 0x7f037814c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.486+0000 7f0376d9d700 1 -- 172.21.15.7:0/1203154622 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036c0094b0 con 0x7f037815b890 2024-09-13T18:54:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.486+0000 7f0376d9d700 1 --2- 172.21.15.7:0/1203154622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037815b890 0x7f0378164d20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f037000c450 tx=0x7f037000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.486+0000 7f0366ffd700 1 -- 172.21.15.7:0/1203154622 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03700052d0 con 0x7f037815b890 2024-09-13T18:54:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.486+0000 7f0366ffd700 1 -- 172.21.15.7:0/1203154622 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0370005430 con 0x7f037815b890 2024-09-13T18:54:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.486+0000 7f0366ffd700 1 -- 172.21.15.7:0/1203154622 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03700055a0 con 0x7f037815b890 2024-09-13T18:54:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.486+0000 7f037cdc2700 1 -- 172.21.15.7:0/1203154622 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03781f6e60 con 0x7f037815b890 2024-09-13T18:54:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.486+0000 7f037cdc2700 1 -- 172.21.15.7:0/1203154622 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03781f6830 con 0x7f037815b890 2024-09-13T18:54:55.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.486+0000 7f037cdc2700 1 -- 172.21.15.7:0/1203154622 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03780ce120 con 0x7f037815b890 2024-09-13T18:54:55.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.490+0000 7f0366ffd700 1 -- 172.21.15.7:0/1203154622 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f03700038e0 con 0x7f037815b890 2024-09-13T18:54:55.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.490+0000 7f0366ffd700 1 --2- 172.21.15.7:0/1203154622 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0358066730 0x7f0358068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:55.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.490+0000 7f0366ffd700 1 -- 172.21.15.7:0/1203154622 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f0370083540 con 0x7f037815b890 2024-09-13T18:54:55.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.490+0000 7f037659c700 1 --2- 172.21.15.7:0/1203154622 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0358066730 0x7f0358068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:55.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.490+0000 7f037659c700 1 --2- 172.21.15.7:0/1203154622 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0358066730 0x7f0358068bf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f036c0090b0 tx=0x7f036c01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:55.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.498+0000 7f0366ffd700 1 -- 172.21.15.7:0/1203154622 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0370050c30 con 0x7f037815b890 2024-09-13T18:54:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.666+0000 7f037cdc2700 1 -- 172.21.15.7:0/1203154622 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03780af010 con 0x7f037815b890 2024-09-13T18:54:55.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.666+0000 7f0366ffd700 1 -- 172.21.15.7:0/1203154622 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+13064 (secure 0 0 0) 0x7f037000ce60 con 0x7f037815b890 2024-09-13T18:54:55.674 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:54:55.674 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":59,"fsid":"33406d92-f401-438a-876e-a56de3962d8e","created":"2024-09-13T18:52:00.650335+0000","modified":"2024-09-13T18:54:52.843342+0000","last_up_change":"2024-09-13T18:52:10.708691+0000","last_in_change":"2024-09-13T18:52:02.723324+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T18:52:12.272418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9c224cb7-67e0-431d-84ca-c42b02ffa864","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6810","nonce":4238805012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6814","nonce":4238805012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6822","nonce":4238805012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238805012},{"type":"v1","addr":"172.21.15.7:6818","nonce":4238805012}]},"public_addr":"172.21.15.7:6810/4238805012","cluster_addr":"172.21.15.7:6814/4238805012","heartbeat_back_addr":"172.21.15.7:6822/4238805012","heartbeat_front_addr":"172.21.15.7:6818/4238805012","state":["exists","up"]},{"osd":1,"uuid":"d638fbca-bf68-46c2-8b13-1e701b58fdc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6811","nonce":2837570651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6813","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6815","nonce":2837570651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6821","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6823","nonce":2837570651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6817","nonce":2837570651},{"type":"v1","addr":"172.21.15.7:6819","nonce":2837570651}]},"public_addr":"172.21.15.7:6811/2837570651","cluster_addr":"172.21.15.7:6815/2837570651","heartbeat_back_addr":"172.21.15.7:6823/2837570651","heartbeat_front_addr":"172.21.15.7:6819/2837570651","state":["exists","up"]},{"osd":2,"uuid":"36f14e49-0413-40df-9769-f1129a36003e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6825","nonce":303159811}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6827","nonce":303159811}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6831","nonce":303159811}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":303159811},{"type":"v1","addr":"172.21.15.7:6829","nonce":303159811}]},"public_addr":"172.21.15.7:6825/303159811","cluster_addr":"172.21.15.7:6827/303159811","heartbeat_back_addr":"172.21.15.7:6831/303159811","heartbeat_front_addr":"172.21.15.7:6829/303159811","state":["exists","up"]},{"osd":3,"uuid":"f0bcc77b-d80a-48f9-8596-a825c5f35b00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6801","nonce":2775757779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6803","nonce":2775757779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6807","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6809","nonce":2775757779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2775757779},{"type":"v1","addr":"172.21.15.7:6805","nonce":2775757779}]},"public_addr":"172.21.15.7:6801/2775757779","cluster_addr":"172.21.15.7:6803/2775757779","heartbeat_back_addr":"172.21.15.7:6809/2775757779","heartbeat_front_addr":"172.21.15.7:6805/2775757779","state":["exists","up"]},{"osd":4,"uuid":"5cc4ed05-2413-4e36-83eb-119e303fed77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6818","nonce":1731498321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6822","nonce":1731498321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6830","nonce":1731498321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1731498321},{"type":"v1","addr":"172.21.15.171:6826","nonce":1731498321}]},"public_addr":"172.21.15.171:6818/1731498321","cluster_addr":"172.21.15.171:6822/1731498321","heartbeat_back_addr":"172.21.15.171:6830/1731498321","heartbeat_front_addr":"172.21.15.171:6826/1731498321","state":["exists","up"]},{"osd":5,"uuid":"d31123fe-975a-47bd-8b4e-f74f55e62135","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6801","nonce":1011659457}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6803","nonce":1011659457}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6807","nonce":1011659457}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":1011659457},{"type":"v1","addr":"172.21.15.171:6805","nonce":1011659457}]},"public_addr":"172.21.15.171:6801/1011659457","cluster_addr":"172.21.15.171:6803/1011659457","heartbeat_back_addr":"172.21.15.171:6807/1011659457","heartbeat_front_addr":"172.21.15.171:6805/1011659457","state":["exists","up"]},{"osd":6,"uuid":"d7b9748c-4637-4802-a1b8-9d99e2abd698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6809","nonce":2277626206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6811","nonce":2277626206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6815","nonce":2277626206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2277626206},{"type":"v1","addr":"172.21.15.171:6813","nonce":2277626206}]},"public_addr":"172.21.15.171:6809/2277626206","cluster_addr":"172.21.15.171:6811/2277626206","heartbeat_back_addr":"172.21.15.171:6815/2277626206","heartbeat_front_addr":"172.21.15.171:6813/2277626206","state":["exists","up"]},{"osd":7,"uuid":"e3d83d03-0918-404a-a548-b985f8e18f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6817","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6819","nonce":2886154603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6821","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6823","nonce":2886154603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6829","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6831","nonce":2886154603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6825","nonce":2886154603},{"type":"v1","addr":"172.21.15.171:6827","nonce":2886154603}]},"public_addr":"172.21.15.171:6819/2886154603","cluster_addr":"172.21.15.171:6823/2886154603","heartbeat_back_addr":"172.21.15.171:6831/2886154603","heartbeat_front_addr":"172.21.15.171:6827/2886154603","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011607+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.011778+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.315663+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:07.768057+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.255553+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.044670+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.270404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T18:52:08.224237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/3768750832":"2024-09-14T18:54:49.820328+0000","172.21.15.7:6834/3768750832":"2024-09-14T18:54:49.820328+0000","172.21.15.7:6835/3168373873":"2024-09-14T18:54:17.388088+0000","172.21.15.7:6834/3168373873":"2024-09-14T18:54:17.388088+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T18:54:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.670+0000 7f037cdc2700 1 -- 172.21.15.7:0/1203154622 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0358066730 msgr2=0x7f0358068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.670+0000 7f037cdc2700 1 --2- 172.21.15.7:0/1203154622 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0358066730 0x7f0358068bf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f036c0090b0 tx=0x7f036c01c040 comp rx=0 tx=0).stop 2024-09-13T18:54:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.670+0000 7f037cdc2700 1 -- 172.21.15.7:0/1203154622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037815b890 msgr2=0x7f0378164d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.670+0000 7f037cdc2700 1 --2- 172.21.15.7:0/1203154622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037815b890 0x7f0378164d20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f037000c450 tx=0x7f037000c760 comp rx=0 tx=0).stop 2024-09-13T18:54:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.670+0000 7f037cdc2700 1 -- 172.21.15.7:0/1203154622 shutdown_connections 2024-09-13T18:54:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.670+0000 7f037cdc2700 1 --2- 172.21.15.7:0/1203154622 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0358066730 0x7f0358068bf0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.670+0000 7f037cdc2700 1 --2- 172.21.15.7:0/1203154622 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f037815b890 0x7f0378164d20 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.670+0000 7f037cdc2700 1 --2- 172.21.15.7:0/1203154622 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f03780cdcc0 0x7f0378151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.670+0000 7f037cdc2700 1 --2- 172.21.15.7:0/1203154622 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03780b2b00 0x7f037814c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.670+0000 7f037cdc2700 1 -- 172.21.15.7:0/1203154622 >> 172.21.15.7:0/1203154622 conn(0x7f03780ce980 msgr2=0x7f03780ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:55.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.670+0000 7f037cdc2700 1 -- 172.21.15.7:0/1203154622 shutdown_connections 2024-09-13T18:54:55.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:55.670+0000 7f037cdc2700 1 -- 172.21.15.7:0/1203154622 wait complete. 2024-09-13T18:54:56.690 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T18:54:56.690 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:54:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.814+0000 7fdb4a8cc700 1 Processor -- start 2024-09-13T18:54:56.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.814+0000 7fdb4a8cc700 1 -- start start 2024-09-13T18:54:56.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb4a8cc700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fdb4415ce30 0x7fdb440b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:56.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb4a8cc700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdb440d1fa0 0x7fdb440cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:56.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb4a8cc700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4406b2a0 con 0x7fdb440b2230 2024-09-13T18:54:56.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb4a8cc700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb4406b540 con 0x7fdb4415ce30 2024-09-13T18:54:56.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb4a8cc700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4406b820 con 0x7fdb440d1fa0 2024-09-13T18:54:56.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb3b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdb440b2230 0x7fdb440d1890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58022/0 (socket says 172.21.15.7:58022) 2024-09-13T18:54:56.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb3b7fe700 1 -- 172.21.15.7:0/2372837877 learned_addr learned my addr 172.21.15.7:0/2372837877 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:56.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb43fff700 1 --2- 172.21.15.7:0/2372837877 >> v2:172.21.15.171:3300/0 conn(0x7fdb4415ce30 0x7fdb440b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:56.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb48e69700 1 --2- 172.21.15.7:0/2372837877 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdb440d1fa0 0x7fdb440cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:56.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb437fe700 1 -- 172.21.15.7:0/2372837877 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443728169 0 0) 0x7fdb4406b2a0 con 0x7fdb440b2230 2024-09-13T18:54:56.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb437fe700 1 -- 172.21.15.7:0/2372837877 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb300035d0 con 0x7fdb440b2230 2024-09-13T18:54:56.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb48e69700 1 -- 172.21.15.7:0/2372837877 >> v2:172.21.15.171:3300/0 conn(0x7fdb4415ce30 msgr2=0x7fdb440b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:56.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb48e69700 1 --2- 172.21.15.7:0/2372837877 >> v2:172.21.15.171:3300/0 conn(0x7fdb4415ce30 0x7fdb440b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:56.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb48e69700 1 -- 172.21.15.7:0/2372837877 >> v1:172.21.15.7:6789/0 conn(0x7fdb440b2230 legacy=0x7fdb440d1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:56.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb48e69700 1 -- 172.21.15.7:0/2372837877 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4406bb00 con 0x7fdb440d1fa0 2024-09-13T18:54:56.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb48e69700 1 --2- 172.21.15.7:0/2372837877 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdb440d1fa0 0x7fdb440cd820 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009d90 tx=0x7fdb3c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20b919492d724ef7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:56.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb437fe700 1 -- 172.21.15.7:0/2372837877 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c00e070 con 0x7fdb440d1fa0 2024-09-13T18:54:56.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb437fe700 1 -- 172.21.15.7:0/2372837877 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3c00bb20 con 0x7fdb440d1fa0 2024-09-13T18:54:56.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.818+0000 7fdb437fe700 1 -- 172.21.15.7:0/2372837877 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c012430 con 0x7fdb440d1fa0 2024-09-13T18:54:56.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb4a8cc700 1 -- 172.21.15.7:0/2372837877 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdb440d1fa0 msgr2=0x7fdb440cd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:56.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb4a8cc700 1 --2- 172.21.15.7:0/2372837877 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdb440d1fa0 0x7fdb440cd820 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009d90 tx=0x7fdb3c00b330 comp rx=0 tx=0).stop 2024-09-13T18:54:56.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb4a8cc700 1 -- 172.21.15.7:0/2372837877 shutdown_connections 2024-09-13T18:54:56.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb4a8cc700 1 --2- 172.21.15.7:0/2372837877 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdb440d1fa0 0x7fdb440cd820 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:56.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb4a8cc700 1 --2- 172.21.15.7:0/2372837877 >> v2:172.21.15.171:3300/0 conn(0x7fdb4415ce30 0x7fdb440b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:56.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb4a8cc700 1 -- 172.21.15.7:0/2372837877 >> 172.21.15.7:0/2372837877 conn(0x7fdb440b47a0 msgr2=0x7fdb440b4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:56.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb4a8cc700 1 -- 172.21.15.7:0/2372837877 shutdown_connections 2024-09-13T18:54:56.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb4a8cc700 1 -- 172.21.15.7:0/2372837877 wait complete. 2024-09-13T18:54:56.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb4a8cc700 1 Processor -- start 2024-09-13T18:54:56.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb4a8cc700 1 -- start start 2024-09-13T18:54:56.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb4a8cc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb440b2230 0x7fdb440c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:56.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb4a8cc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb440d1fa0 0x7fdb440c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:56.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb4a8cc700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdb4415ce30 0x7fdb441542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:56.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb4a8cc700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440cc040 con 0x7fdb440b2230 2024-09-13T18:54:56.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb4a8cc700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440cc320 con 0x7fdb440d1fa0 2024-09-13T18:54:56.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb4a8cc700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440cc600 con 0x7fdb4415ce30 2024-09-13T18:54:56.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb43fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb440b2230 0x7fdb440c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:56.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb43fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb440b2230 0x7fdb440c7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40836/0 (socket says 172.21.15.7:40836) 2024-09-13T18:54:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb43fff700 1 -- 172.21.15.7:0/548692983 learned_addr learned my addr 172.21.15.7:0/548692983 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb48e69700 1 --2- 172.21.15.7:0/548692983 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdb4415ce30 0x7fdb441542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:56.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.822+0000 7fdb3b7fe700 1 --2- 172.21.15.7:0/548692983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb440d1fa0 0x7fdb440c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.826+0000 7fdb43fff700 1 -- 172.21.15.7:0/548692983 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdb4415ce30 msgr2=0x7fdb441542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.826+0000 7fdb43fff700 1 --2- 172.21.15.7:0/548692983 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fdb4415ce30 0x7fdb441542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.826+0000 7fdb43fff700 1 -- 172.21.15.7:0/548692983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb440d1fa0 msgr2=0x7fdb440c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.826+0000 7fdb43fff700 1 --2- 172.21.15.7:0/548692983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdb440d1fa0 0x7fdb440c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.826+0000 7fdb43fff700 1 -- 172.21.15.7:0/548692983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3c009a70 con 0x7fdb440b2230 2024-09-13T18:54:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.826+0000 7fdb43fff700 1 --2- 172.21.15.7:0/548692983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb440b2230 0x7fdb440c7a80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400e840 tx=0x7fdb3400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.826+0000 7fdb417fa700 1 -- 172.21.15.7:0/548692983 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb340095a0 con 0x7fdb440b2230 2024-09-13T18:54:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.826+0000 7fdb417fa700 1 -- 172.21.15.7:0/548692983 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb34013680 con 0x7fdb440b2230 2024-09-13T18:54:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.826+0000 7fdb4a8cc700 1 -- 172.21.15.7:0/548692983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4415d390 con 0x7fdb440b2230 2024-09-13T18:54:56.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.826+0000 7fdb417fa700 1 -- 172.21.15.7:0/548692983 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb340139f0 con 0x7fdb440b2230 2024-09-13T18:54:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.826+0000 7fdb4a8cc700 1 -- 172.21.15.7:0/548692983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb440c8b20 con 0x7fdb440b2230 2024-09-13T18:54:56.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.826+0000 7fdb4a8cc700 1 -- 172.21.15.7:0/548692983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdb440cc1b0 con 0x7fdb440b2230 2024-09-13T18:54:56.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.830+0000 7fdb417fa700 1 -- 172.21.15.7:0/548692983 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fdb34013b50 con 0x7fdb440b2230 2024-09-13T18:54:56.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.830+0000 7fdb417fa700 1 --2- 172.21.15.7:0/548692983 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fdb24066760 0x7fdb24068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.830+0000 7fdb417fa700 1 -- 172.21.15.7:0/548692983 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fdb34083c00 con 0x7fdb440b2230 2024-09-13T18:54:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.830+0000 7fdb417fa700 1 --2- 172.21.15.7:0/548692983 >> [v2:172.21.15.7:6806/4238805012,v1:172.21.15.7:6810/4238805012] conn(0x7fdb2406c660 0x7fdb2406ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.830+0000 7fdb3b7fe700 1 --2- 172.21.15.7:0/548692983 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fdb24066760 0x7fdb24068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.830+0000 7fdb417fa700 1 -- 172.21.15.7:0/548692983 --> [v2:172.21.15.7:6806/4238805012,v1:172.21.15.7:6810/4238805012] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb2406f130 con 0x7fdb2406c660 2024-09-13T18:54:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.830+0000 7fdb417fa700 1 -- 172.21.15.7:0/548692983 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb340862c0 con 0x7fdb440b2230 2024-09-13T18:54:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.830+0000 7fdb48e69700 1 --2- 172.21.15.7:0/548692983 >> [v2:172.21.15.7:6806/4238805012,v1:172.21.15.7:6810/4238805012] conn(0x7fdb2406c660 0x7fdb2406ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:56.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.830+0000 7fdb3b7fe700 1 --2- 172.21.15.7:0/548692983 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fdb24066760 0x7fdb24068c20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdb440c7fc0 tx=0x7fdb2c007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.830+0000 7fdb48e69700 1 --2- 172.21.15.7:0/548692983 >> [v2:172.21.15.7:6806/4238805012,v1:172.21.15.7:6810/4238805012] conn(0x7fdb2406c660 0x7fdb2406ea80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:56.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.830+0000 7fdb417fa700 1 -- 172.21.15.7:0/548692983 <== osd.0 v2:172.21.15.7:6806/4238805012 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fdb2406f130 con 0x7fdb2406c660 2024-09-13T18:54:56.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.850+0000 7fdb4a8cc700 1 -- 172.21.15.7:0/548692983 --> [v2:172.21.15.7:6806/4238805012,v1:172.21.15.7:6810/4238805012] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdb4415d210 con 0x7fdb2406c660 2024-09-13T18:54:56.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.850+0000 7fdb417fa700 1 -- 172.21.15.7:0/548692983 <== osd.0 v2:172.21.15.7:6806/4238805012 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fdb3c003c00 con 0x7fdb2406c660 2024-09-13T18:54:56.857 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:54:56.857 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:54:56.857 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:54:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.850+0000 7fdb4a8cc700 1 -- 172.21.15.7:0/548692983 >> [v2:172.21.15.7:6806/4238805012,v1:172.21.15.7:6810/4238805012] conn(0x7fdb2406c660 msgr2=0x7fdb2406ea80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.850+0000 7fdb4a8cc700 1 --2- 172.21.15.7:0/548692983 >> [v2:172.21.15.7:6806/4238805012,v1:172.21.15.7:6810/4238805012] conn(0x7fdb2406c660 0x7fdb2406ea80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.850+0000 7fdb4a8cc700 1 -- 172.21.15.7:0/548692983 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fdb24066760 msgr2=0x7fdb24068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.850+0000 7fdb4a8cc700 1 --2- 172.21.15.7:0/548692983 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fdb24066760 0x7fdb24068c20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdb440c7fc0 tx=0x7fdb2c007de0 comp rx=0 tx=0).stop 2024-09-13T18:54:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.850+0000 7fdb4a8cc700 1 -- 172.21.15.7:0/548692983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb440b2230 msgr2=0x7fdb440c7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.850+0000 7fdb4a8cc700 1 --2- 172.21.15.7:0/548692983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb440b2230 0x7fdb440c7a80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400e840 tx=0x7fdb3400eb50 comp rx=0 tx=0).stop 2024-09-13T18:54:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.850+0000 7fdb48e69700 1 -- 172.21.15.7:0/548692983 reap_dead start 2024-09-13T18:54:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.850+0000 7fdb4a8cc700 1 -- 172.21.15.7:0/548692983 shutdown_connections 2024-09-13T18:54:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.850+0000 7fdb4a8cc700 1 -- 172.21.15.7:0/548692983 >> 172.21.15.7:0/548692983 conn(0x7fdb440b47a0 msgr2=0x7fdb440b4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.850+0000 7fdb4a8cc700 1 -- 172.21.15.7:0/548692983 shutdown_connections 2024-09-13T18:54:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.850+0000 7fdb4a8cc700 1 -- 172.21.15.7:0/548692983 wait complete. 2024-09-13T18:54:56.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T18:54:56.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc6759e700 1 Processor -- start 2024-09-13T18:54:56.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc6759e700 1 -- start start 2024-09-13T18:54:56.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc6759e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbc6815cda0 0x7fbc6806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:56.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc6759e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc680ceb70 0x7fbc680b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:56.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc6759e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fbc6806a520 0x7fbc6806a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:56.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc6759e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc6806be40 con 0x7fbc680ceb70 2024-09-13T18:54:56.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc6759e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc6806aeb0 con 0x7fbc6806a520 2024-09-13T18:54:56.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc6759e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc680b8430 con 0x7fbc6815cda0 2024-09-13T18:54:56.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc65d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc680ceb70 0x7fbc680b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:56.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc65d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc680ceb70 0x7fbc680b5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40844/0 (socket says 172.21.15.7:40844) 2024-09-13T18:54:56.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc65d9b700 1 -- 172.21.15.7:0/3783856950 learned_addr learned my addr 172.21.15.7:0/3783856950 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:56.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc6659c700 1 --2- 172.21.15.7:0/3783856950 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbc6815cda0 0x7fbc6806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:56.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc66d9d700 1 --2- 172.21.15.7:0/3783856950 >> v2:172.21.15.171:3300/0 conn(0x7fbc6806a520 0x7fbc6806a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:56.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc65d9b700 1 -- 172.21.15.7:0/3783856950 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbc6815cda0 msgr2=0x7fbc6806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:56.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc65d9b700 1 --2- 172.21.15.7:0/3783856950 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbc6815cda0 0x7fbc6806b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:56.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc65d9b700 1 -- 172.21.15.7:0/3783856950 >> v2:172.21.15.171:3300/0 conn(0x7fbc6806a520 msgr2=0x7fbc6806a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:56.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc65d9b700 1 --2- 172.21.15.7:0/3783856950 >> v2:172.21.15.171:3300/0 conn(0x7fbc6806a520 0x7fbc6806a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:56.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc65d9b700 1 -- 172.21.15.7:0/3783856950 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc680cf350 con 0x7fbc680ceb70 2024-09-13T18:54:56.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc65d9b700 1 --2- 172.21.15.7:0/3783856950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc680ceb70 0x7fbc680b5de0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbc681617d0 tx=0x7fbc58009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2353969d7c3cc990 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:56.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc6559a700 1 -- 172.21.15.7:0/3783856950 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5800b8a0 con 0x7fbc680ceb70 2024-09-13T18:54:56.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc6559a700 1 -- 172.21.15.7:0/3783856950 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5800ba00 con 0x7fbc680ceb70 2024-09-13T18:54:56.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.990+0000 7fbc6559a700 1 -- 172.21.15.7:0/3783856950 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5800bd90 con 0x7fbc680ceb70 2024-09-13T18:54:57.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.994+0000 7fbc6759e700 1 -- 172.21.15.7:0/3783856950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc680ceb70 msgr2=0x7fbc680b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.994+0000 7fbc6759e700 1 --2- 172.21.15.7:0/3783856950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc680ceb70 0x7fbc680b5de0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbc681617d0 tx=0x7fbc58009bf0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.994+0000 7fbc6759e700 1 -- 172.21.15.7:0/3783856950 shutdown_connections 2024-09-13T18:54:57.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.994+0000 7fbc6759e700 1 --2- 172.21.15.7:0/3783856950 >> v2:172.21.15.171:3300/0 conn(0x7fbc6806a520 0x7fbc6806a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.994+0000 7fbc6759e700 1 --2- 172.21.15.7:0/3783856950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc680ceb70 0x7fbc680b5de0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.994+0000 7fbc6759e700 1 --2- 172.21.15.7:0/3783856950 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbc6815cda0 0x7fbc6806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.994+0000 7fbc6759e700 1 -- 172.21.15.7:0/3783856950 >> 172.21.15.7:0/3783856950 conn(0x7fbc680d2cb0 msgr2=0x7fbc680d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:57.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.994+0000 7fbc6759e700 1 -- 172.21.15.7:0/3783856950 shutdown_connections 2024-09-13T18:54:57.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.994+0000 7fbc6759e700 1 -- 172.21.15.7:0/3783856950 wait complete. 2024-09-13T18:54:57.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.994+0000 7fbc6759e700 1 Processor -- start 2024-09-13T18:54:57.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.994+0000 7fbc6759e700 1 -- start start 2024-09-13T18:54:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.994+0000 7fbc6759e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbc6806a520 0x7fbc681f2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.994+0000 7fbc6759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc680ceb70 0x7fbc680cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.994+0000 7fbc6759e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc6815cda0 0x7fbc680b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.994+0000 7fbc6759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6815d610 con 0x7fbc680ceb70 2024-09-13T18:54:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.994+0000 7fbc6759e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc680ccaf0 con 0x7fbc6815cda0 2024-09-13T18:54:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.994+0000 7fbc6759e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc680ccdd0 con 0x7fbc6806a520 2024-09-13T18:54:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.994+0000 7fbc65d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc680ceb70 0x7fbc680cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.998+0000 7fbc65d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc680ceb70 0x7fbc680cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40848/0 (socket says 172.21.15.7:40848) 2024-09-13T18:54:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.998+0000 7fbc65d9b700 1 -- 172.21.15.7:0/1096474870 learned_addr learned my addr 172.21.15.7:0/1096474870 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.998+0000 7fbc66d9d700 1 --2- 172.21.15.7:0/1096474870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc6815cda0 0x7fbc680b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.998+0000 7fbc6659c700 1 --2- 172.21.15.7:0/1096474870 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbc6806a520 0x7fbc681f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.998+0000 7fbc65d9b700 1 -- 172.21.15.7:0/1096474870 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbc6806a520 msgr2=0x7fbc681f2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.998+0000 7fbc65d9b700 1 --2- 172.21.15.7:0/1096474870 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbc6806a520 0x7fbc681f2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.998+0000 7fbc65d9b700 1 -- 172.21.15.7:0/1096474870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc6815cda0 msgr2=0x7fbc680b6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.998+0000 7fbc65d9b700 1 --2- 172.21.15.7:0/1096474870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc6815cda0 0x7fbc680b6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.998+0000 7fbc65d9b700 1 -- 172.21.15.7:0/1096474870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc58009570 con 0x7fbc680ceb70 2024-09-13T18:54:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.998+0000 7fbc65d9b700 1 --2- 172.21.15.7:0/1096474870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc680ceb70 0x7fbc680cc0a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbc58013000 tx=0x7fbc58004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:57.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.998+0000 7fbc5effd700 1 -- 172.21.15.7:0/1096474870 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc580045a0 con 0x7fbc680ceb70 2024-09-13T18:54:57.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.998+0000 7fbc5effd700 1 -- 172.21.15.7:0/1096474870 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5801f070 con 0x7fbc680ceb70 2024-09-13T18:54:57.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.998+0000 7fbc6759e700 1 -- 172.21.15.7:0/1096474870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6815d8d0 con 0x7fbc680ceb70 2024-09-13T18:54:57.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.998+0000 7fbc5effd700 1 -- 172.21.15.7:0/1096474870 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58003900 con 0x7fbc680ceb70 2024-09-13T18:54:57.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.998+0000 7fbc6759e700 1 -- 172.21.15.7:0/1096474870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc680cc5e0 con 0x7fbc680ceb70 2024-09-13T18:54:57.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.998+0000 7fbc6759e700 1 -- 172.21.15.7:0/1096474870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6815dae0 con 0x7fbc680ceb70 2024-09-13T18:54:57.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.998+0000 7fbc5effd700 1 -- 172.21.15.7:0/1096474870 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbc58021030 con 0x7fbc680ceb70 2024-09-13T18:54:57.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:56.998+0000 7fbc5effd700 1 --2- 172.21.15.7:0/1096474870 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbc48066730 0x7fbc48068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.002+0000 7fbc5effd700 1 -- 172.21.15.7:0/1096474870 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fbc58085480 con 0x7fbc680ceb70 2024-09-13T18:54:57.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.002+0000 7fbc6659c700 1 --2- 172.21.15.7:0/1096474870 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbc48066730 0x7fbc48068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.002+0000 7fbc6659c700 1 --2- 172.21.15.7:0/1096474870 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbc48066730 0x7fbc48068bf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbc50006b40 tx=0x7fbc50008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:57.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.006+0000 7fbc5effd700 1 -- 172.21.15.7:0/1096474870 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbc58052aa0 con 0x7fbc680ceb70 2024-09-13T18:54:57.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.174+0000 7fbc6759e700 1 -- 172.21.15.7:0/1096474870 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc6815d490 con 0x7fbc48066730 2024-09-13T18:54:57.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.174+0000 7fbc5effd700 1 -- 172.21.15.7:0/1096474870 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbc50007650 con 0x7fbc48066730 2024-09-13T18:54:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.178+0000 7fbc6759e700 1 -- 172.21.15.7:0/1096474870 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbc48066730 msgr2=0x7fbc48068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.178+0000 7fbc6759e700 1 --2- 172.21.15.7:0/1096474870 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbc48066730 0x7fbc48068bf0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbc50006b40 tx=0x7fbc50008040 comp rx=0 tx=0).stop 2024-09-13T18:54:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.178+0000 7fbc6759e700 1 -- 172.21.15.7:0/1096474870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc680ceb70 msgr2=0x7fbc680cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.178+0000 7fbc6759e700 1 --2- 172.21.15.7:0/1096474870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc680ceb70 0x7fbc680cc0a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbc58013000 tx=0x7fbc58004770 comp rx=0 tx=0).stop 2024-09-13T18:54:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.178+0000 7fbc6759e700 1 -- 172.21.15.7:0/1096474870 shutdown_connections 2024-09-13T18:54:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.178+0000 7fbc6759e700 1 --2- 172.21.15.7:0/1096474870 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbc48066730 0x7fbc48068bf0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.178+0000 7fbc6759e700 1 --2- 172.21.15.7:0/1096474870 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbc6815cda0 0x7fbc680b6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.178+0000 7fbc6759e700 1 --2- 172.21.15.7:0/1096474870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc680ceb70 0x7fbc680cc0a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.178+0000 7fbc6759e700 1 --2- 172.21.15.7:0/1096474870 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbc6806a520 0x7fbc681f2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.178+0000 7fbc6759e700 1 -- 172.21.15.7:0/1096474870 >> 172.21.15.7:0/1096474870 conn(0x7fbc680d2cb0 msgr2=0x7fbc6815b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.178+0000 7fbc6759e700 1 -- 172.21.15.7:0/1096474870 shutdown_connections 2024-09-13T18:54:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.178+0000 7fbc6759e700 1 -- 172.21.15.7:0/1096474870 wait complete. 2024-09-13T18:54:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T18:54:57.197 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T18:54:57.197 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:54:57.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.314+0000 7f845359e700 1 Processor -- start 2024-09-13T18:54:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.314+0000 7f845359e700 1 -- start start 2024-09-13T18:54:57.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f845359e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f845406ccf0 0x7f8454159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f845359e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8454070ba0 0x7f84540720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f845359e700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f845406d0d0 0x7f845415af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f845359e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8454071030 con 0x7f8454070ba0 2024-09-13T18:54:57.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f845359e700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8454071310 con 0x7f845406d0d0 2024-09-13T18:54:57.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f845359e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84540715f0 con 0x7f845406ccf0 2024-09-13T18:54:57.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f844bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8454070ba0 0x7f84540720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f844bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8454070ba0 0x7f84540720d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40850/0 (socket says 172.21.15.7:40850) 2024-09-13T18:54:57.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f844bfff700 1 -- 172.21.15.7:0/1281111172 learned_addr learned my addr 172.21.15.7:0/1281111172 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:57.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f845259c700 1 --2- 172.21.15.7:0/1281111172 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f845406ccf0 0x7f8454159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f8452d9d700 1 --2- 172.21.15.7:0/1281111172 >> v2:172.21.15.171:3300/0 conn(0x7f845406d0d0 0x7f845415af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f8452d9d700 1 -- 172.21.15.7:0/1281111172 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f845406ccf0 msgr2=0x7f8454159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f8452d9d700 1 --2- 172.21.15.7:0/1281111172 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f845406ccf0 0x7f8454159dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f8452d9d700 1 -- 172.21.15.7:0/1281111172 >> v2:172.21.15.7:3300/0 conn(0x7f8454070ba0 msgr2=0x7f84540720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f8452d9d700 1 --2- 172.21.15.7:0/1281111172 >> v2:172.21.15.7:3300/0 conn(0x7f8454070ba0 0x7f84540720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f8452d9d700 1 -- 172.21.15.7:0/1281111172 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84540718d0 con 0x7f845406d0d0 2024-09-13T18:54:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f844bfff700 1 --2- 172.21.15.7:0/1281111172 >> v2:172.21.15.7:3300/0 conn(0x7f8454070ba0 0x7f84540720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f8452d9d700 1 --2- 172.21.15.7:0/1281111172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f845406d0d0 0x7f845415af90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f84540b2ff0 tx=0x7f844c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92823c2e7e7392c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f8451d9b700 1 -- 172.21.15.7:0/1281111172 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c00e070 con 0x7f845406d0d0 2024-09-13T18:54:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f8451d9b700 1 -- 172.21.15.7:0/1281111172 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c00bb20 con 0x7f845406d0d0 2024-09-13T18:54:57.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.318+0000 7f8451d9b700 1 -- 172.21.15.7:0/1281111172 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c012430 con 0x7f845406d0d0 2024-09-13T18:54:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.322+0000 7f845359e700 1 -- 172.21.15.7:0/1281111172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f845406d0d0 msgr2=0x7f845415af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.322+0000 7f845359e700 1 --2- 172.21.15.7:0/1281111172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f845406d0d0 0x7f845415af90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f84540b2ff0 tx=0x7f844c00b330 comp rx=0 tx=0).stop 2024-09-13T18:54:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.322+0000 7f845359e700 1 -- 172.21.15.7:0/1281111172 shutdown_connections 2024-09-13T18:54:57.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.322+0000 7f845359e700 1 --2- 172.21.15.7:0/1281111172 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f845406d0d0 0x7f845415af90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.322+0000 7f845359e700 1 --2- 172.21.15.7:0/1281111172 >> v2:172.21.15.7:3300/0 conn(0x7f8454070ba0 0x7f84540720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.322+0000 7f845359e700 1 --2- 172.21.15.7:0/1281111172 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f845406ccf0 0x7f8454159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.322+0000 7f845359e700 1 -- 172.21.15.7:0/1281111172 >> 172.21.15.7:0/1281111172 conn(0x7f84540cfad0 msgr2=0x7f84540cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:57.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.322+0000 7f845359e700 1 -- 172.21.15.7:0/1281111172 shutdown_connections 2024-09-13T18:54:57.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.322+0000 7f845359e700 1 -- 172.21.15.7:0/1281111172 wait complete. 2024-09-13T18:54:57.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.322+0000 7f845359e700 1 Processor -- start 2024-09-13T18:54:57.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.322+0000 7f845359e700 1 -- start start 2024-09-13T18:54:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f845359e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f845406ccf0 0x7f8454164fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f845359e700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f845406d0d0 0x7f8454153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f845359e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8454163980 0x7f845406b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f845359e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84541656c0 con 0x7f8454163980 2024-09-13T18:54:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f845359e700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84540cedd0 con 0x7f845406ccf0 2024-09-13T18:54:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f845359e700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84540ce010 con 0x7f845406d0d0 2024-09-13T18:54:57.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f8452d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8454163980 0x7f845406b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f8452d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8454163980 0x7f845406b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40854/0 (socket says 172.21.15.7:40854) 2024-09-13T18:54:57.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f8452d9d700 1 -- 172.21.15.7:0/710094264 learned_addr learned my addr 172.21.15.7:0/710094264 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:57.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f845259c700 1 --2- 172.21.15.7:0/710094264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f845406ccf0 0x7f8454164fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f844bfff700 1 --2- 172.21.15.7:0/710094264 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f845406d0d0 0x7f8454153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f8452d9d700 1 -- 172.21.15.7:0/710094264 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f845406d0d0 msgr2=0x7f8454153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f8452d9d700 1 --2- 172.21.15.7:0/710094264 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f845406d0d0 0x7f8454153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f8452d9d700 1 -- 172.21.15.7:0/710094264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f845406ccf0 msgr2=0x7f8454164fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f8452d9d700 1 --2- 172.21.15.7:0/710094264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f845406ccf0 0x7f8454164fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f8452d9d700 1 -- 172.21.15.7:0/710094264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c009570 con 0x7f8454163980 2024-09-13T18:54:57.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f8452d9d700 1 --2- 172.21.15.7:0/710094264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8454163980 0x7f845406b540 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f844c014040 tx=0x7f844c004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:57.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f844affd700 1 -- 172.21.15.7:0/710094264 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c012430 con 0x7f8454163980 2024-09-13T18:54:57.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f844affd700 1 -- 172.21.15.7:0/710094264 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c020070 con 0x7f8454163980 2024-09-13T18:54:57.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f845359e700 1 -- 172.21.15.7:0/710094264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f844c009a70 con 0x7f8454163980 2024-09-13T18:54:57.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f844affd700 1 -- 172.21.15.7:0/710094264 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c004280 con 0x7f8454163980 2024-09-13T18:54:57.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f845359e700 1 -- 172.21.15.7:0/710094264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84540ce5d0 con 0x7f8454163980 2024-09-13T18:54:57.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.326+0000 7f845359e700 1 -- 172.21.15.7:0/710094264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8454157640 con 0x7f8454163980 2024-09-13T18:54:57.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.330+0000 7f844affd700 1 -- 172.21.15.7:0/710094264 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f844c0043e0 con 0x7f8454163980 2024-09-13T18:54:57.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.330+0000 7f844affd700 1 --2- 172.21.15.7:0/710094264 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8434066730 0x7f8434068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.330+0000 7f844affd700 1 -- 172.21.15.7:0/710094264 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f844c022070 con 0x7f8454163980 2024-09-13T18:54:57.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.330+0000 7f845259c700 1 --2- 172.21.15.7:0/710094264 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8434066730 0x7f8434068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.330+0000 7f844affd700 1 --2- 172.21.15.7:0/710094264 >> [v2:172.21.15.7:6808/2837570651,v1:172.21.15.7:6811/2837570651] conn(0x7f843406c630 0x7f843406ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.330+0000 7f844affd700 1 -- 172.21.15.7:0/710094264 --> [v2:172.21.15.7:6808/2837570651,v1:172.21.15.7:6811/2837570651] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f843406f100 con 0x7f843406c630 2024-09-13T18:54:57.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.330+0000 7f844affd700 1 -- 172.21.15.7:0/710094264 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f844c08a030 con 0x7f8454163980 2024-09-13T18:54:57.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.330+0000 7f844bfff700 1 --2- 172.21.15.7:0/710094264 >> [v2:172.21.15.7:6808/2837570651,v1:172.21.15.7:6811/2837570651] conn(0x7f843406c630 0x7f843406ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.330+0000 7f844bfff700 1 --2- 172.21.15.7:0/710094264 >> [v2:172.21.15.7:6808/2837570651,v1:172.21.15.7:6811/2837570651] conn(0x7f843406c630 0x7f843406ea50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:57.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.330+0000 7f845259c700 1 --2- 172.21.15.7:0/710094264 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8434066730 0x7f8434068bf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8444009ad0 tx=0x7f8444009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:57.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.330+0000 7f844affd700 1 -- 172.21.15.7:0/710094264 <== osd.1 v2:172.21.15.7:6808/2837570651 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f843406f100 con 0x7f843406c630 2024-09-13T18:54:57.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.350+0000 7f845359e700 1 -- 172.21.15.7:0/710094264 --> [v2:172.21.15.7:6808/2837570651,v1:172.21.15.7:6811/2837570651] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f84540b39b0 con 0x7f843406c630 2024-09-13T18:54:57.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.350+0000 7f844affd700 1 -- 172.21.15.7:0/710094264 <== osd.1 v2:172.21.15.7:6808/2837570651 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f84540b39b0 con 0x7f843406c630 2024-09-13T18:54:57.356 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:54:57.357 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:54:57.357 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:54:57.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.350+0000 7f845359e700 1 -- 172.21.15.7:0/710094264 >> [v2:172.21.15.7:6808/2837570651,v1:172.21.15.7:6811/2837570651] conn(0x7f843406c630 msgr2=0x7f843406ea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.350+0000 7f845359e700 1 --2- 172.21.15.7:0/710094264 >> [v2:172.21.15.7:6808/2837570651,v1:172.21.15.7:6811/2837570651] conn(0x7f843406c630 0x7f843406ea50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.350+0000 7f845359e700 1 -- 172.21.15.7:0/710094264 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8434066730 msgr2=0x7f8434068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.350+0000 7f845359e700 1 --2- 172.21.15.7:0/710094264 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8434066730 0x7f8434068bf0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8444009ad0 tx=0x7f8444009240 comp rx=0 tx=0).stop 2024-09-13T18:54:57.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.350+0000 7f845359e700 1 -- 172.21.15.7:0/710094264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8454163980 msgr2=0x7f845406b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.350+0000 7f845359e700 1 --2- 172.21.15.7:0/710094264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8454163980 0x7f845406b540 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f844c014040 tx=0x7f844c004af0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.350+0000 7f8452d9d700 1 -- 172.21.15.7:0/710094264 reap_dead start 2024-09-13T18:54:57.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.350+0000 7f845359e700 1 -- 172.21.15.7:0/710094264 shutdown_connections 2024-09-13T18:54:57.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.350+0000 7f845359e700 1 -- 172.21.15.7:0/710094264 >> 172.21.15.7:0/710094264 conn(0x7f84540cfad0 msgr2=0x7f8454156100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:57.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.350+0000 7f845359e700 1 -- 172.21.15.7:0/710094264 shutdown_connections 2024-09-13T18:54:57.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.350+0000 7f845359e700 1 -- 172.21.15.7:0/710094264 wait complete. 2024-09-13T18:54:57.370 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T18:54:57.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.482+0000 7fb0dbfff700 1 Processor -- start 2024-09-13T18:54:57.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.482+0000 7fb0dbfff700 1 -- start start 2024-09-13T18:54:57.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0dbfff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fb0dc06d8c0 0x7fb0dc164a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0dbfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0dc164f90 0x7fb0dc0704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0dbfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0dc162570 con 0x7fb0dc070a00 2024-09-13T18:54:57.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0dbfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0dc162e60 con 0x7fb0dc164f90 2024-09-13T18:54:57.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0dbfff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0dc165420 con 0x7fb0dc06d8c0 2024-09-13T18:54:57.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0da7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0dc164f90 0x7fb0dc0704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0da7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0dc164f90 0x7fb0dc0704c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40858/0 (socket says 172.21.15.7:40858) 2024-09-13T18:54:57.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0da7fc700 1 -- 172.21.15.7:0/969903351 learned_addr learned my addr 172.21.15.7:0/969903351 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:57.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0db7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb0dc070a00 0x7fb0dc161460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58038/0 (socket says 172.21.15.7:58038) 2024-09-13T18:54:57.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0daffd700 1 --2- 172.21.15.7:0/969903351 >> v2:172.21.15.171:3300/0 conn(0x7fb0dc06d8c0 0x7fb0dc164a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0da7fc700 1 -- 172.21.15.7:0/969903351 >> v2:172.21.15.171:3300/0 conn(0x7fb0dc06d8c0 msgr2=0x7fb0dc164a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:57.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0da7fc700 1 --2- 172.21.15.7:0/969903351 >> v2:172.21.15.171:3300/0 conn(0x7fb0dc06d8c0 0x7fb0dc164a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0da7fc700 1 -- 172.21.15.7:0/969903351 >> v1:172.21.15.7:6789/0 conn(0x7fb0dc070a00 legacy=0x7fb0dc161460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0da7fc700 1 -- 172.21.15.7:0/969903351 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc071240 con 0x7fb0dc164f90 2024-09-13T18:54:57.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0da7fc700 1 --2- 172.21.15.7:0/969903351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc164f90 0x7fb0dc0704c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0b3c80 tx=0x7fb0d0009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db82cf55e3ec6283 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:57.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0d9ffb700 1 -- 172.21.15.7:0/969903351 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d000b8a0 con 0x7fb0dc164f90 2024-09-13T18:54:57.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0d9ffb700 1 -- 172.21.15.7:0/969903351 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d000ba00 con 0x7fb0dc164f90 2024-09-13T18:54:57.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0d9ffb700 1 -- 172.21.15.7:0/969903351 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d000bd90 con 0x7fb0dc164f90 2024-09-13T18:54:57.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0dbfff700 1 -- 172.21.15.7:0/969903351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc164f90 msgr2=0x7fb0dc0704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0dbfff700 1 --2- 172.21.15.7:0/969903351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc164f90 0x7fb0dc0704c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0b3c80 tx=0x7fb0d0009bf0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0dbfff700 1 -- 172.21.15.7:0/969903351 shutdown_connections 2024-09-13T18:54:57.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0dbfff700 1 --2- 172.21.15.7:0/969903351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc164f90 0x7fb0dc0704c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0dbfff700 1 --2- 172.21.15.7:0/969903351 >> v2:172.21.15.171:3300/0 conn(0x7fb0dc06d8c0 0x7fb0dc164a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0dbfff700 1 -- 172.21.15.7:0/969903351 >> 172.21.15.7:0/969903351 conn(0x7fb0dc0b0e10 msgr2=0x7fb0dc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:57.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0dbfff700 1 -- 172.21.15.7:0/969903351 shutdown_connections 2024-09-13T18:54:57.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.486+0000 7fb0dbfff700 1 -- 172.21.15.7:0/969903351 wait complete. 2024-09-13T18:54:57.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0dbfff700 1 Processor -- start 2024-09-13T18:54:57.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0dbfff700 1 -- start start 2024-09-13T18:54:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0dbfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb0dc06d8c0 0x7fb0dc15e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0dbfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0dc070a00 0x7fb0dc156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc164f90 0x7fb0dc15ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0dbfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc15e6e0 con 0x7fb0dc164f90 2024-09-13T18:54:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0dbfff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc15cab0 con 0x7fb0dc070a00 2024-09-13T18:54:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0dbfff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc15e120 con 0x7fb0dc06d8c0 2024-09-13T18:54:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0db7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc164f90 0x7fb0dc15ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0db7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc164f90 0x7fb0dc15ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40866/0 (socket says 172.21.15.7:40866) 2024-09-13T18:54:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0db7fe700 1 -- 172.21.15.7:0/3166130025 learned_addr learned my addr 172.21.15.7:0/3166130025 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0da7fc700 1 --2- 172.21.15.7:0/3166130025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0dc070a00 0x7fb0dc156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0daffd700 1 --2- 172.21.15.7:0/3166130025 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb0dc06d8c0 0x7fb0dc15e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0db7fe700 1 -- 172.21.15.7:0/3166130025 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb0dc06d8c0 msgr2=0x7fb0dc15e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0db7fe700 1 --2- 172.21.15.7:0/3166130025 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb0dc06d8c0 0x7fb0dc15e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0db7fe700 1 -- 172.21.15.7:0/3166130025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0dc070a00 msgr2=0x7fb0dc156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0db7fe700 1 --2- 172.21.15.7:0/3166130025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0dc070a00 0x7fb0dc156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0db7fe700 1 -- 172.21.15.7:0/3166130025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d0009570 con 0x7fb0dc164f90 2024-09-13T18:54:57.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.490+0000 7fb0db7fe700 1 --2- 172.21.15.7:0/3166130025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc164f90 0x7fb0dc15ced0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400cbd0 tx=0x7fb0d400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.494+0000 7fb0c3fff700 1 -- 172.21.15.7:0/3166130025 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d400aaa0 con 0x7fb0dc164f90 2024-09-13T18:54:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.494+0000 7fb0c3fff700 1 -- 172.21.15.7:0/3166130025 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d400ac00 con 0x7fb0dc164f90 2024-09-13T18:54:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.494+0000 7fb0dbfff700 1 -- 172.21.15.7:0/3166130025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0dc071010 con 0x7fb0dc164f90 2024-09-13T18:54:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.494+0000 7fb0c3fff700 1 -- 172.21.15.7:0/3166130025 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4007070 con 0x7fb0dc164f90 2024-09-13T18:54:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.494+0000 7fb0dbfff700 1 -- 172.21.15.7:0/3166130025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc15c6d0 con 0x7fb0dc164f90 2024-09-13T18:54:57.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.494+0000 7fb0dbfff700 1 -- 172.21.15.7:0/3166130025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0dc15c530 con 0x7fb0dc164f90 2024-09-13T18:54:57.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.494+0000 7fb0c3fff700 1 -- 172.21.15.7:0/3166130025 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fb0d40071d0 con 0x7fb0dc164f90 2024-09-13T18:54:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.498+0000 7fb0c3fff700 1 --2- 172.21.15.7:0/3166130025 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb0c8066730 0x7fb0c8068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.498+0000 7fb0c3fff700 1 -- 172.21.15.7:0/3166130025 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fb0d4083a40 con 0x7fb0dc164f90 2024-09-13T18:54:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.498+0000 7fb0daffd700 1 --2- 172.21.15.7:0/3166130025 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb0c8066730 0x7fb0c8068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.498+0000 7fb0daffd700 1 --2- 172.21.15.7:0/3166130025 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb0c8066730 0x7fb0c8068bf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009a00 tx=0x7fb0cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:57.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.506+0000 7fb0c3fff700 1 -- 172.21.15.7:0/3166130025 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb0dc15c530 con 0x7fb0dc164f90 2024-09-13T18:54:57.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.686+0000 7fb0dbfff700 1 -- 172.21.15.7:0/3166130025 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0dc162a80 con 0x7fb0c8066730 2024-09-13T18:54:57.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.686+0000 7fb0c3fff700 1 -- 172.21.15.7:0/3166130025 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb0cc011750 con 0x7fb0c8066730 2024-09-13T18:54:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.690+0000 7fb0dbfff700 1 -- 172.21.15.7:0/3166130025 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb0c8066730 msgr2=0x7fb0c8068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.690+0000 7fb0dbfff700 1 --2- 172.21.15.7:0/3166130025 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb0c8066730 0x7fb0c8068bf0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009a00 tx=0x7fb0cc008040 comp rx=0 tx=0).stop 2024-09-13T18:54:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.690+0000 7fb0dbfff700 1 -- 172.21.15.7:0/3166130025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc164f90 msgr2=0x7fb0dc15ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.690+0000 7fb0dbfff700 1 --2- 172.21.15.7:0/3166130025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc164f90 0x7fb0dc15ced0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400cbd0 tx=0x7fb0d400cee0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.690+0000 7fb0dbfff700 1 -- 172.21.15.7:0/3166130025 shutdown_connections 2024-09-13T18:54:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.690+0000 7fb0dbfff700 1 --2- 172.21.15.7:0/3166130025 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fb0c8066730 0x7fb0c8068bf0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.690+0000 7fb0dbfff700 1 --2- 172.21.15.7:0/3166130025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0dc164f90 0x7fb0dc15ced0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.690+0000 7fb0dbfff700 1 --2- 172.21.15.7:0/3166130025 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0dc070a00 0x7fb0dc156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.690+0000 7fb0dbfff700 1 --2- 172.21.15.7:0/3166130025 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fb0dc06d8c0 0x7fb0dc15e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.690+0000 7fb0dbfff700 1 -- 172.21.15.7:0/3166130025 >> 172.21.15.7:0/3166130025 conn(0x7fb0dc0b0e10 msgr2=0x7fb0dc0b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.690+0000 7fb0dbfff700 1 -- 172.21.15.7:0/3166130025 shutdown_connections 2024-09-13T18:54:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.690+0000 7fb0dbfff700 1 -- 172.21.15.7:0/3166130025 wait complete. 2024-09-13T18:54:57.699 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T18:54:57.714 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T18:54:57.714 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:54:57.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.834+0000 7f65a9cf7700 1 Processor -- start 2024-09-13T18:54:57.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.834+0000 7f65a9cf7700 1 -- start start 2024-09-13T18:54:57.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.834+0000 7f65a9cf7700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f65a415ce20 0x7f65a40ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.834+0000 7f65a9cf7700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f65a40ce8f0 0x7f65a40b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.834+0000 7f65a9cf7700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65a40ced30 con 0x7f65a40b6600 2024-09-13T18:54:57.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.834+0000 7f65a9cf7700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a40d1540 con 0x7f65a40ce8f0 2024-09-13T18:54:57.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.834+0000 7f65a9cf7700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a40d1820 con 0x7f65a415ce20 2024-09-13T18:54:57.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.834+0000 7f65a3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f65a40ce8f0 0x7f65a40b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f65a40ce8f0 0x7f65a40b60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40878/0 (socket says 172.21.15.7:40878) 2024-09-13T18:54:57.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a94f6700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f65a40b6600 0x7f65a40b6a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58048/0 (socket says 172.21.15.7:58048) 2024-09-13T18:54:57.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a3fff700 1 -- 172.21.15.7:0/4026421407 learned_addr learned my addr 172.21.15.7:0/4026421407 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:57.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a8cf5700 1 --2- 172.21.15.7:0/4026421407 >> v2:172.21.15.171:3300/0 conn(0x7f65a415ce20 0x7f65a40ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a3fff700 1 -- 172.21.15.7:0/4026421407 >> v2:172.21.15.171:3300/0 conn(0x7f65a415ce20 msgr2=0x7f65a40ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:57.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a3fff700 1 --2- 172.21.15.7:0/4026421407 >> v2:172.21.15.171:3300/0 conn(0x7f65a415ce20 0x7f65a40ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a3fff700 1 -- 172.21.15.7:0/4026421407 >> v1:172.21.15.7:6789/0 conn(0x7f65a40b6600 legacy=0x7f65a40b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a37fe700 1 -- 172.21.15.7:0/4026421407 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619971577 0 0) 0x7f65a40ced30 con 0x7f65a40b6600 2024-09-13T18:54:57.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a3fff700 1 -- 172.21.15.7:0/4026421407 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a40d1b00 con 0x7f65a40ce8f0 2024-09-13T18:54:57.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a3fff700 1 --2- 172.21.15.7:0/4026421407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a40ce8f0 0x7f65a40b60c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f65a40d06c0 tx=0x7f6594009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f95d0a08b470ea server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:57.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a37fe700 1 -- 172.21.15.7:0/4026421407 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659400b8a0 con 0x7f65a40ce8f0 2024-09-13T18:54:57.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a37fe700 1 -- 172.21.15.7:0/4026421407 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659400ba00 con 0x7f65a40ce8f0 2024-09-13T18:54:57.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a37fe700 1 -- 172.21.15.7:0/4026421407 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659400bd90 con 0x7f65a40ce8f0 2024-09-13T18:54:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a9cf7700 1 -- 172.21.15.7:0/4026421407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a40ce8f0 msgr2=0x7f65a40b60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a9cf7700 1 --2- 172.21.15.7:0/4026421407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a40ce8f0 0x7f65a40b60c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f65a40d06c0 tx=0x7f6594009bf0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a9cf7700 1 -- 172.21.15.7:0/4026421407 shutdown_connections 2024-09-13T18:54:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a9cf7700 1 --2- 172.21.15.7:0/4026421407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a40ce8f0 0x7f65a40b60c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a9cf7700 1 --2- 172.21.15.7:0/4026421407 >> v2:172.21.15.171:3300/0 conn(0x7f65a415ce20 0x7f65a40ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a9cf7700 1 -- 172.21.15.7:0/4026421407 >> 172.21.15.7:0/4026421407 conn(0x7f65a40d2c30 msgr2=0x7f65a40d3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a9cf7700 1 -- 172.21.15.7:0/4026421407 shutdown_connections 2024-09-13T18:54:57.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.838+0000 7f65a9cf7700 1 -- 172.21.15.7:0/4026421407 wait complete. 2024-09-13T18:54:57.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.842+0000 7f65a9cf7700 1 Processor -- start 2024-09-13T18:54:57.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.842+0000 7f65a9cf7700 1 -- start start 2024-09-13T18:54:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a9cf7700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65a40b6600 0x7f65a415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a9cf7700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65a40ce8f0 0x7f65a4148e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a9cf7700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a415ce20 0x7f65a41f2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a9cf7700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a415f0e0 con 0x7f65a415ce20 2024-09-13T18:54:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a9cf7700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a415e4d0 con 0x7f65a40ce8f0 2024-09-13T18:54:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a9cf7700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a415e7b0 con 0x7f65a40b6600 2024-09-13T18:54:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a94f6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a415ce20 0x7f65a41f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a94f6700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a415ce20 0x7f65a41f2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40888/0 (socket says 172.21.15.7:40888) 2024-09-13T18:54:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a94f6700 1 -- 172.21.15.7:0/3187663400 learned_addr learned my addr 172.21.15.7:0/3187663400 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a8cf5700 1 --2- 172.21.15.7:0/3187663400 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65a40b6600 0x7f65a415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a3fff700 1 --2- 172.21.15.7:0/3187663400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65a40ce8f0 0x7f65a4148e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a94f6700 1 -- 172.21.15.7:0/3187663400 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65a40b6600 msgr2=0x7f65a415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a94f6700 1 --2- 172.21.15.7:0/3187663400 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65a40b6600 0x7f65a415dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a94f6700 1 -- 172.21.15.7:0/3187663400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65a40ce8f0 msgr2=0x7f65a4148e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a94f6700 1 --2- 172.21.15.7:0/3187663400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65a40ce8f0 0x7f65a4148e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a94f6700 1 -- 172.21.15.7:0/3187663400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6594009570 con 0x7f65a415ce20 2024-09-13T18:54:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a8cf5700 1 --2- 172.21.15.7:0/3187663400 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f65a40b6600 0x7f65a415dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a3fff700 1 --2- 172.21.15.7:0/3187663400 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65a40ce8f0 0x7f65a4148e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:57.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a94f6700 1 --2- 172.21.15.7:0/3187663400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a415ce20 0x7f65a41f2780 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f659c00cca0 tx=0x7f659c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:57.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a17fa700 1 -- 172.21.15.7:0/3187663400 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c00cea0 con 0x7f65a415ce20 2024-09-13T18:54:57.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a9cf7700 1 -- 172.21.15.7:0/3187663400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a415ea90 con 0x7f65a415ce20 2024-09-13T18:54:57.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a17fa700 1 -- 172.21.15.7:0/3187663400 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c00ade0 con 0x7f65a415ce20 2024-09-13T18:54:57.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a9cf7700 1 -- 172.21.15.7:0/3187663400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a415e1e0 con 0x7f65a415ce20 2024-09-13T18:54:57.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a17fa700 1 -- 172.21.15.7:0/3187663400 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c0072d0 con 0x7f65a415ce20 2024-09-13T18:54:57.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a9cf7700 1 -- 172.21.15.7:0/3187663400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65a40d1210 con 0x7f65a415ce20 2024-09-13T18:54:57.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.846+0000 7f65a17fa700 1 -- 172.21.15.7:0/3187663400 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f659c007430 con 0x7f65a415ce20 2024-09-13T18:54:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.850+0000 7f65a17fa700 1 --2- 172.21.15.7:0/3187663400 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6590066760 0x7f6590068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.850+0000 7f65a17fa700 1 -- 172.21.15.7:0/3187663400 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f659c084250 con 0x7f65a415ce20 2024-09-13T18:54:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.850+0000 7f65a17fa700 1 --2- 172.21.15.7:0/3187663400 >> [v2:172.21.15.7:6824/303159811,v1:172.21.15.7:6825/303159811] conn(0x7f659006c660 0x7f659006ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.850+0000 7f65a17fa700 1 -- 172.21.15.7:0/3187663400 --> [v2:172.21.15.7:6824/303159811,v1:172.21.15.7:6825/303159811] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f659006f130 con 0x7f659006c660 2024-09-13T18:54:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.850+0000 7f65a17fa700 1 -- 172.21.15.7:0/3187663400 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f659c0846a0 con 0x7f65a415ce20 2024-09-13T18:54:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.850+0000 7f65a8cf5700 1 --2- 172.21.15.7:0/3187663400 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6590066760 0x7f6590068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.850+0000 7f65a3fff700 1 --2- 172.21.15.7:0/3187663400 >> [v2:172.21.15.7:6824/303159811,v1:172.21.15.7:6825/303159811] conn(0x7f659006c660 0x7f659006ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.850+0000 7f65a3fff700 1 --2- 172.21.15.7:0/3187663400 >> [v2:172.21.15.7:6824/303159811,v1:172.21.15.7:6825/303159811] conn(0x7f659006c660 0x7f659006ea80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.850+0000 7f65a8cf5700 1 --2- 172.21.15.7:0/3187663400 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6590066760 0x7f6590068c20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6598007a70 tx=0x7f6598008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.850+0000 7f65a17fa700 1 -- 172.21.15.7:0/3187663400 <== osd.2 v2:172.21.15.7:6824/303159811 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f659006f130 con 0x7f659006c660 2024-09-13T18:54:57.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.862+0000 7f65a9cf7700 1 -- 172.21.15.7:0/3187663400 --> [v2:172.21.15.7:6824/303159811,v1:172.21.15.7:6825/303159811] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f65a40d20c0 con 0x7f659006c660 2024-09-13T18:54:57.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.862+0000 7f65a17fa700 1 -- 172.21.15.7:0/3187663400 <== osd.2 v2:172.21.15.7:6824/303159811 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f659400bb70 con 0x7f659006c660 2024-09-13T18:54:57.870 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:54:57.870 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:54:57.870 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:54:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.862+0000 7f65a9cf7700 1 -- 172.21.15.7:0/3187663400 >> [v2:172.21.15.7:6824/303159811,v1:172.21.15.7:6825/303159811] conn(0x7f659006c660 msgr2=0x7f659006ea80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.866+0000 7f65a9cf7700 1 --2- 172.21.15.7:0/3187663400 >> [v2:172.21.15.7:6824/303159811,v1:172.21.15.7:6825/303159811] conn(0x7f659006c660 0x7f659006ea80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.866+0000 7f65a9cf7700 1 -- 172.21.15.7:0/3187663400 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6590066760 msgr2=0x7f6590068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.866+0000 7f65a9cf7700 1 --2- 172.21.15.7:0/3187663400 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6590066760 0x7f6590068c20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6598007a70 tx=0x7f6598008040 comp rx=0 tx=0).stop 2024-09-13T18:54:57.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.866+0000 7f65a9cf7700 1 -- 172.21.15.7:0/3187663400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a415ce20 msgr2=0x7f65a41f2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.866+0000 7f65a9cf7700 1 --2- 172.21.15.7:0/3187663400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65a415ce20 0x7f65a41f2780 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f659c00cca0 tx=0x7f659c00a340 comp rx=0 tx=0).stop 2024-09-13T18:54:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.866+0000 7f65a94f6700 1 -- 172.21.15.7:0/3187663400 reap_dead start 2024-09-13T18:54:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.866+0000 7f65a9cf7700 1 -- 172.21.15.7:0/3187663400 shutdown_connections 2024-09-13T18:54:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.866+0000 7f65a9cf7700 1 -- 172.21.15.7:0/3187663400 >> 172.21.15.7:0/3187663400 conn(0x7f65a40d2c30 msgr2=0x7f65a40d3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.866+0000 7f65a9cf7700 1 -- 172.21.15.7:0/3187663400 shutdown_connections 2024-09-13T18:54:57.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.866+0000 7f65a9cf7700 1 -- 172.21.15.7:0/3187663400 wait complete. 2024-09-13T18:54:57.883 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T18:54:57.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.986+0000 7f7c42f0c700 1 Processor -- start 2024-09-13T18:54:57.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.986+0000 7f7c42f0c700 1 -- start start 2024-09-13T18:54:57.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c42f0c700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7c3c15cda0 0x7f7c3c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c42f0c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7c3c0ceb70 0x7f7c3c0b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:57.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c42f0c700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c3c0b8430 con 0x7f7c3c06a520 2024-09-13T18:54:57.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c42f0c700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c3c06be40 con 0x7f7c3c0ceb70 2024-09-13T18:54:57.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c42f0c700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c0cf350 con 0x7f7c3c15cda0 2024-09-13T18:54:57.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c41709700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7c3c0ceb70 0x7f7c3c0b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c41709700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7c3c0ceb70 0x7f7c3c0b5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40902/0 (socket says 172.21.15.7:40902) 2024-09-13T18:54:57.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c41709700 1 -- 172.21.15.7:0/2228685118 learned_addr learned my addr 172.21.15.7:0/2228685118 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:57.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c41f0a700 1 --2- 172.21.15.7:0/2228685118 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7c3c15cda0 0x7f7c3c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:57.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c41709700 1 -- 172.21.15.7:0/2228685118 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7c3c15cda0 msgr2=0x7f7c3c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:57.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c41709700 1 --2- 172.21.15.7:0/2228685118 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7c3c15cda0 0x7f7c3c06b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c41709700 1 -- 172.21.15.7:0/2228685118 >> v1:172.21.15.171:6789/0 conn(0x7f7c3c06a520 legacy=0x7f7c3c0d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c41709700 1 -- 172.21.15.7:0/2228685118 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3c0d0880 con 0x7f7c3c0ceb70 2024-09-13T18:54:57.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c41709700 1 --2- 172.21.15.7:0/2228685118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3c0ceb70 0x7f7c3c0b5de0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009910 tx=0x7f7c30009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b325bd1398ac6716 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c40f08700 1 -- 172.21.15.7:0/2228685118 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3000ba60 con 0x7f7c3c0ceb70 2024-09-13T18:54:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c40f08700 1 -- 172.21.15.7:0/2228685118 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3000bbc0 con 0x7f7c3c0ceb70 2024-09-13T18:54:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c40f08700 1 -- 172.21.15.7:0/2228685118 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30009e60 con 0x7f7c3c0ceb70 2024-09-13T18:54:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c42f0c700 1 -- 172.21.15.7:0/2228685118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3c0ceb70 msgr2=0x7f7c3c0b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c42f0c700 1 --2- 172.21.15.7:0/2228685118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3c0ceb70 0x7f7c3c0b5de0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009910 tx=0x7f7c30009c20 comp rx=0 tx=0).stop 2024-09-13T18:54:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c42f0c700 1 -- 172.21.15.7:0/2228685118 shutdown_connections 2024-09-13T18:54:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c42f0c700 1 --2- 172.21.15.7:0/2228685118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3c0ceb70 0x7f7c3c0b5de0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c42f0c700 1 --2- 172.21.15.7:0/2228685118 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7c3c15cda0 0x7f7c3c06b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c42f0c700 1 -- 172.21.15.7:0/2228685118 >> 172.21.15.7:0/2228685118 conn(0x7f7c3c0d2cb0 msgr2=0x7f7c3c0d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c42f0c700 1 -- 172.21.15.7:0/2228685118 shutdown_connections 2024-09-13T18:54:57.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.990+0000 7f7c42f0c700 1 -- 172.21.15.7:0/2228685118 wait complete. 2024-09-13T18:54:57.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c42f0c700 1 Processor -- start 2024-09-13T18:54:57.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c42f0c700 1 -- start start 2024-09-13T18:54:58.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c42f0c700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7c3c06a520 0x7f7c3c1f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c42f0c700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c3c0ceb70 0x7f7c3c0c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c42f0c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3c15cda0 0x7f7c3c0c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c42f0c700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c0c9220 con 0x7f7c3c15cda0 2024-09-13T18:54:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c42f0c700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c15da00 con 0x7f7c3c0ceb70 2024-09-13T18:54:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c42f0c700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c0c8980 con 0x7f7c3c06a520 2024-09-13T18:54:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c4270b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3c15cda0 0x7f7c3c0c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c4270b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3c15cda0 0x7f7c3c0c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40918/0 (socket says 172.21.15.7:40918) 2024-09-13T18:54:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c4270b700 1 -- 172.21.15.7:0/218211035 learned_addr learned my addr 172.21.15.7:0/218211035 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c41709700 1 --2- 172.21.15.7:0/218211035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c3c0ceb70 0x7f7c3c0c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c41f0a700 1 --2- 172.21.15.7:0/218211035 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7c3c06a520 0x7f7c3c1f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c4270b700 1 -- 172.21.15.7:0/218211035 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7c3c06a520 msgr2=0x7f7c3c1f1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c4270b700 1 --2- 172.21.15.7:0/218211035 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7c3c06a520 0x7f7c3c1f1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c4270b700 1 -- 172.21.15.7:0/218211035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c3c0ceb70 msgr2=0x7f7c3c0c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c4270b700 1 --2- 172.21.15.7:0/218211035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c3c0ceb70 0x7f7c3c0c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c4270b700 1 -- 172.21.15.7:0/218211035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c30009570 con 0x7f7c3c15cda0 2024-09-13T18:54:58.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c41709700 1 --2- 172.21.15.7:0/218211035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c3c0ceb70 0x7f7c3c0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:58.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c4270b700 1 --2- 172.21.15.7:0/218211035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3c15cda0 0x7f7c3c0c8300 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400cc50 tx=0x7f7c3400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:58.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c2e7fc700 1 -- 172.21.15.7:0/218211035 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3400ad80 con 0x7f7c3c15cda0 2024-09-13T18:54:58.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c2e7fc700 1 -- 172.21.15.7:0/218211035 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3400ce50 con 0x7f7c3c15cda0 2024-09-13T18:54:58.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c42f0c700 1 -- 172.21.15.7:0/218211035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3c0c8c60 con 0x7f7c3c15cda0 2024-09-13T18:54:58.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c2e7fc700 1 -- 172.21.15.7:0/218211035 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34003c60 con 0x7f7c3c15cda0 2024-09-13T18:54:58.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.994+0000 7f7c42f0c700 1 -- 172.21.15.7:0/218211035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3c1f2390 con 0x7f7c3c15cda0 2024-09-13T18:54:58.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.998+0000 7f7c42f0c700 1 -- 172.21.15.7:0/218211035 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c3c15d7d0 con 0x7f7c3c15cda0 2024-09-13T18:54:58.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.998+0000 7f7c2e7fc700 1 -- 172.21.15.7:0/218211035 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7c340050d0 con 0x7f7c3c15cda0 2024-09-13T18:54:58.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.998+0000 7f7c2e7fc700 1 --2- 172.21.15.7:0/218211035 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7c20066760 0x7f7c20068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.998+0000 7f7c2e7fc700 1 -- 172.21.15.7:0/218211035 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f7c34083f30 con 0x7f7c3c15cda0 2024-09-13T18:54:58.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.998+0000 7f7c41f0a700 1 --2- 172.21.15.7:0/218211035 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7c20066760 0x7f7c20068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:57.998+0000 7f7c41f0a700 1 --2- 172.21.15.7:0/218211035 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7c20066760 0x7f7c20068c20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800b9d0 tx=0x7f7c38008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:58.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.006+0000 7f7c2e7fc700 1 -- 172.21.15.7:0/218211035 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7c3c15d7d0 con 0x7f7c3c15cda0 2024-09-13T18:54:58.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.178+0000 7f7c42f0c700 1 -- 172.21.15.7:0/218211035 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c3c074770 con 0x7f7c20066760 2024-09-13T18:54:58.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.178+0000 7f7c2e7fc700 1 -- 172.21.15.7:0/218211035 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7c3c074770 con 0x7f7c20066760 2024-09-13T18:54:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.178+0000 7f7c42f0c700 1 -- 172.21.15.7:0/218211035 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7c20066760 msgr2=0x7f7c20068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.178+0000 7f7c42f0c700 1 --2- 172.21.15.7:0/218211035 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7c20066760 0x7f7c20068c20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800b9d0 tx=0x7f7c38008040 comp rx=0 tx=0).stop 2024-09-13T18:54:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.178+0000 7f7c42f0c700 1 -- 172.21.15.7:0/218211035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3c15cda0 msgr2=0x7f7c3c0c8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.178+0000 7f7c42f0c700 1 --2- 172.21.15.7:0/218211035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3c15cda0 0x7f7c3c0c8300 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400cc50 tx=0x7f7c3400a340 comp rx=0 tx=0).stop 2024-09-13T18:54:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.182+0000 7f7c42f0c700 1 -- 172.21.15.7:0/218211035 shutdown_connections 2024-09-13T18:54:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.182+0000 7f7c42f0c700 1 --2- 172.21.15.7:0/218211035 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f7c20066760 0x7f7c20068c20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.182+0000 7f7c42f0c700 1 --2- 172.21.15.7:0/218211035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c3c15cda0 0x7f7c3c0c8300 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.182+0000 7f7c42f0c700 1 --2- 172.21.15.7:0/218211035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7c3c0ceb70 0x7f7c3c0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.182+0000 7f7c42f0c700 1 --2- 172.21.15.7:0/218211035 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7c3c06a520 0x7f7c3c1f1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.182+0000 7f7c42f0c700 1 -- 172.21.15.7:0/218211035 >> 172.21.15.7:0/218211035 conn(0x7f7c3c0d2cb0 msgr2=0x7f7c3c0b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:58.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.182+0000 7f7c42f0c700 1 -- 172.21.15.7:0/218211035 shutdown_connections 2024-09-13T18:54:58.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.182+0000 7f7c42f0c700 1 -- 172.21.15.7:0/218211035 wait complete. 2024-09-13T18:54:58.188 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T18:54:58.199 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T18:54:58.199 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:54:58.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.314+0000 7f393132b700 1 Processor -- start 2024-09-13T18:54:58.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.314+0000 7f393132b700 1 -- start start 2024-09-13T18:54:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f393132b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f392c15ce20 0x7f392c0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f393132b700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f392c06b530 0x7f392c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f393132b700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f392c0ced30 con 0x7f392c0ce8f0 2024-09-13T18:54:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f393132b700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f392c0d1540 con 0x7f392c15ce20 2024-09-13T18:54:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f393132b700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f392c0d1820 con 0x7f392c06b530 2024-09-13T18:54:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f392bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f392c15ce20 0x7f392c0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f392bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f392c15ce20 0x7f392c0ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40922/0 (socket says 172.21.15.7:40922) 2024-09-13T18:54:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f392bfff700 1 -- 172.21.15.7:0/1727995508 learned_addr learned my addr 172.21.15.7:0/1727995508 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:58.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f3930b2a700 1 --2- 172.21.15.7:0/1727995508 >> v2:172.21.15.171:3300/0 conn(0x7f392c06b530 0x7f392c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f392bfff700 1 -- 172.21.15.7:0/1727995508 >> v2:172.21.15.171:3300/0 conn(0x7f392c06b530 msgr2=0x7f392c06b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f392bfff700 1 --2- 172.21.15.7:0/1727995508 >> v2:172.21.15.171:3300/0 conn(0x7f392c06b530 0x7f392c06b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f392bfff700 1 -- 172.21.15.7:0/1727995508 >> v1:172.21.15.171:6789/0 conn(0x7f392c0ce8f0 legacy=0x7f392c0b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f392bfff700 1 -- 172.21.15.7:0/1727995508 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c0d1b00 con 0x7f392c15ce20 2024-09-13T18:54:58.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f392bfff700 1 --2- 172.21.15.7:0/1727995508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c15ce20 0x7f392c0ce3b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f392c0b4770 tx=0x7f3920009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1914486ca0ed11e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:58.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f392affd700 1 -- 172.21.15.7:0/1727995508 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3920011830 con 0x7f392c15ce20 2024-09-13T18:54:58.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f392affd700 1 -- 172.21.15.7:0/1727995508 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3920011990 con 0x7f392c15ce20 2024-09-13T18:54:58.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f392affd700 1 -- 172.21.15.7:0/1727995508 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3920011d20 con 0x7f392c15ce20 2024-09-13T18:54:58.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f393132b700 1 -- 172.21.15.7:0/1727995508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c15ce20 msgr2=0x7f392c0ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f393132b700 1 --2- 172.21.15.7:0/1727995508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c15ce20 0x7f392c0ce3b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f392c0b4770 tx=0x7f3920009bf0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f393132b700 1 -- 172.21.15.7:0/1727995508 shutdown_connections 2024-09-13T18:54:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f393132b700 1 --2- 172.21.15.7:0/1727995508 >> v2:172.21.15.171:3300/0 conn(0x7f392c06b530 0x7f392c06b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f393132b700 1 --2- 172.21.15.7:0/1727995508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c15ce20 0x7f392c0ce3b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f393132b700 1 -- 172.21.15.7:0/1727995508 >> 172.21.15.7:0/1727995508 conn(0x7f392c0d2c30 msgr2=0x7f392c0d3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f393132b700 1 -- 172.21.15.7:0/1727995508 shutdown_connections 2024-09-13T18:54:58.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.318+0000 7f393132b700 1 -- 172.21.15.7:0/1727995508 wait complete. 2024-09-13T18:54:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f393132b700 1 Processor -- start 2024-09-13T18:54:58.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f393132b700 1 -- start start 2024-09-13T18:54:58.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f393132b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c06b530 0x7f392c0c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f393132b700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f392c0ce8f0 0x7f392c1f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f393132b700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f392c15ce20 0x7f392c1f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f393132b700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c0c6290 con 0x7f392c06b530 2024-09-13T18:54:58.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f393132b700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c0c96b0 con 0x7f392c15ce20 2024-09-13T18:54:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f393132b700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392c15c0c0 con 0x7f392c0ce8f0 2024-09-13T18:54:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f392bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c06b530 0x7f392c0c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f392bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c06b530 0x7f392c0c9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40934/0 (socket says 172.21.15.7:40934) 2024-09-13T18:54:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f392bfff700 1 -- 172.21.15.7:0/2137681695 learned_addr learned my addr 172.21.15.7:0/2137681695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f3930b2a700 1 --2- 172.21.15.7:0/2137681695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f392c15ce20 0x7f392c1f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f392b7fe700 1 --2- 172.21.15.7:0/2137681695 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f392c0ce8f0 0x7f392c1f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f392bfff700 1 -- 172.21.15.7:0/2137681695 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f392c0ce8f0 msgr2=0x7f392c1f17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f392bfff700 1 --2- 172.21.15.7:0/2137681695 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f392c0ce8f0 0x7f392c1f17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f392bfff700 1 -- 172.21.15.7:0/2137681695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f392c15ce20 msgr2=0x7f392c1f5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f392bfff700 1 --2- 172.21.15.7:0/2137681695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f392c15ce20 0x7f392c1f5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f392bfff700 1 -- 172.21.15.7:0/2137681695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3920009570 con 0x7f392c06b530 2024-09-13T18:54:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f392b7fe700 1 --2- 172.21.15.7:0/2137681695 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f392c0ce8f0 0x7f392c1f17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:58.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.322+0000 7f392bfff700 1 --2- 172.21.15.7:0/2137681695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c06b530 0x7f392c0c9170 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3920000c00 tx=0x7f3920004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.326+0000 7f3928ff9700 1 -- 172.21.15.7:0/2137681695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3920004610 con 0x7f392c06b530 2024-09-13T18:54:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.326+0000 7f3928ff9700 1 -- 172.21.15.7:0/2137681695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392001e070 con 0x7f392c06b530 2024-09-13T18:54:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.326+0000 7f3928ff9700 1 -- 172.21.15.7:0/2137681695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39200038e0 con 0x7f392c06b530 2024-09-13T18:54:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.326+0000 7f393132b700 1 -- 172.21.15.7:0/2137681695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392c14fb70 con 0x7f392c06b530 2024-09-13T18:54:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.326+0000 7f393132b700 1 -- 172.21.15.7:0/2137681695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392c150080 con 0x7f392c06b530 2024-09-13T18:54:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.326+0000 7f393132b700 1 -- 172.21.15.7:0/2137681695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f392c0d1210 con 0x7f392c06b530 2024-09-13T18:54:58.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.326+0000 7f3928ff9700 1 -- 172.21.15.7:0/2137681695 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3920003a40 con 0x7f392c06b530 2024-09-13T18:54:58.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.330+0000 7f3928ff9700 1 --2- 172.21.15.7:0/2137681695 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3918066730 0x7f3918068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.330+0000 7f3928ff9700 1 -- 172.21.15.7:0/2137681695 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f3920084a80 con 0x7f392c06b530 2024-09-13T18:54:58.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.330+0000 7f3928ff9700 1 --2- 172.21.15.7:0/2137681695 >> [v2:172.21.15.7:6800/2775757779,v1:172.21.15.7:6801/2775757779] conn(0x7f391806c630 0x7f391806ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.330+0000 7f3928ff9700 1 -- 172.21.15.7:0/2137681695 --> [v2:172.21.15.7:6800/2775757779,v1:172.21.15.7:6801/2775757779] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f391806f100 con 0x7f391806c630 2024-09-13T18:54:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.330+0000 7f3928ff9700 1 -- 172.21.15.7:0/2137681695 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39200892a0 con 0x7f392c06b530 2024-09-13T18:54:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.330+0000 7f392b7fe700 1 --2- 172.21.15.7:0/2137681695 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3918066730 0x7f3918068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.330+0000 7f3930b2a700 1 --2- 172.21.15.7:0/2137681695 >> [v2:172.21.15.7:6800/2775757779,v1:172.21.15.7:6801/2775757779] conn(0x7f391806c630 0x7f391806ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.330+0000 7f392b7fe700 1 --2- 172.21.15.7:0/2137681695 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3918066730 0x7f3918068bf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f392c15c800 tx=0x7f391c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:58.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.330+0000 7f3930b2a700 1 --2- 172.21.15.7:0/2137681695 >> [v2:172.21.15.7:6800/2775757779,v1:172.21.15.7:6801/2775757779] conn(0x7f391806c630 0x7f391806ea50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:58.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.330+0000 7f3928ff9700 1 -- 172.21.15.7:0/2137681695 <== osd.3 v2:172.21.15.7:6800/2775757779 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f391806f100 con 0x7f391806c630 2024-09-13T18:54:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.350+0000 7f393132b700 1 -- 172.21.15.7:0/2137681695 --> [v2:172.21.15.7:6800/2775757779,v1:172.21.15.7:6801/2775757779] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f392c1f1f70 con 0x7f391806c630 2024-09-13T18:54:58.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.350+0000 7f3928ff9700 1 -- 172.21.15.7:0/2137681695 <== osd.3 v2:172.21.15.7:6800/2775757779 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f392c1f1f70 con 0x7f391806c630 2024-09-13T18:54:58.357 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:54:58.357 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:54:58.357 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:54:58.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.350+0000 7f393132b700 1 -- 172.21.15.7:0/2137681695 >> [v2:172.21.15.7:6800/2775757779,v1:172.21.15.7:6801/2775757779] conn(0x7f391806c630 msgr2=0x7f391806ea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.350+0000 7f393132b700 1 --2- 172.21.15.7:0/2137681695 >> [v2:172.21.15.7:6800/2775757779,v1:172.21.15.7:6801/2775757779] conn(0x7f391806c630 0x7f391806ea50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.350+0000 7f393132b700 1 -- 172.21.15.7:0/2137681695 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3918066730 msgr2=0x7f3918068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.350+0000 7f393132b700 1 --2- 172.21.15.7:0/2137681695 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3918066730 0x7f3918068bf0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f392c15c800 tx=0x7f391c009040 comp rx=0 tx=0).stop 2024-09-13T18:54:58.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.350+0000 7f393132b700 1 -- 172.21.15.7:0/2137681695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c06b530 msgr2=0x7f392c0c9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.350+0000 7f393132b700 1 --2- 172.21.15.7:0/2137681695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c06b530 0x7f392c0c9170 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3920000c00 tx=0x7f3920004770 comp rx=0 tx=0).stop 2024-09-13T18:54:58.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.350+0000 7f3930b2a700 1 -- 172.21.15.7:0/2137681695 reap_dead start 2024-09-13T18:54:58.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.350+0000 7f393132b700 1 -- 172.21.15.7:0/2137681695 shutdown_connections 2024-09-13T18:54:58.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.350+0000 7f393132b700 1 -- 172.21.15.7:0/2137681695 >> 172.21.15.7:0/2137681695 conn(0x7f392c0d2c30 msgr2=0x7f392c0b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:58.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.350+0000 7f393132b700 1 -- 172.21.15.7:0/2137681695 shutdown_connections 2024-09-13T18:54:58.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.350+0000 7f393132b700 1 -- 172.21.15.7:0/2137681695 wait complete. 2024-09-13T18:54:58.370 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T18:54:58.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.482+0000 7f736e4c9700 1 Processor -- start 2024-09-13T18:54:58.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.482+0000 7f736e4c9700 1 -- start start 2024-09-13T18:54:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736e4c9700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f73680b3130 0x7f73680b3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736e4c9700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73680b3ac0 con 0x7f7368159130 2024-09-13T18:54:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736e4c9700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73680cc090 con 0x7f7368071750 2024-09-13T18:54:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736e4c9700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7368159510 con 0x7f73680b3130 2024-09-13T18:54:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736d4c7700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7368159130 0x7f73680cb980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58064/0 (socket says 172.21.15.7:58064) 2024-09-13T18:54:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736d4c7700 1 -- 172.21.15.7:0/773342490 learned_addr learned my addr 172.21.15.7:0/773342490 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:58.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736dcc8700 1 --2- 172.21.15.7:0/773342490 >> v2:172.21.15.171:3300/0 conn(0x7f73680b3130 0x7f73680b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f735ffff700 1 -- 172.21.15.7:0/773342490 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936978220 0 0) 0x7f7364001110 con 0x7f7368159130 2024-09-13T18:54:58.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f735ffff700 1 -- 172.21.15.7:0/773342490 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73540035d0 con 0x7f7368159130 2024-09-13T18:54:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f735ffff700 1 -- 172.21.15.7:0/773342490 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3438981343 0 0) 0x7f73540035d0 con 0x7f7368159130 2024-09-13T18:54:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f735ffff700 1 -- 172.21.15.7:0/773342490 >> v2:172.21.15.171:3300/0 conn(0x7f73680b3130 msgr2=0x7f73680b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f735ffff700 1 --2- 172.21.15.7:0/773342490 >> v2:172.21.15.171:3300/0 conn(0x7f73680b3130 0x7f73680b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f735ffff700 1 -- 172.21.15.7:0/773342490 >> v1:172.21.15.171:6789/0 conn(0x7f7368071750 legacy=0x7f736806ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f735ffff700 1 -- 172.21.15.7:0/773342490 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73680cc2f0 con 0x7f7368159130 2024-09-13T18:54:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f735ffff700 1 -- 172.21.15.7:0/773342490 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3004512194 0 0) 0x7f7364001800 con 0x7f7368159130 2024-09-13T18:54:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f735ffff700 1 -- 172.21.15.7:0/773342490 >> v1:172.21.15.7:6789/0 conn(0x7f7368159130 legacy=0x7f73680cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f735ffff700 1 --2- 172.21.15.7:0/773342490 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7354003a20 0x7f7354003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f735ffff700 1 --2- 172.21.15.7:0/773342490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7354004420 0x7f73540048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f735ffff700 1 --2- 172.21.15.7:0/773342490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73540050f0 0x7f735400ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f735ffff700 1 -- 172.21.15.7:0/773342490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7368159510 con 0x7f7354004420 2024-09-13T18:54:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f735ffff700 1 -- 172.21.15.7:0/773342490 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735400c050 con 0x7f73540050f0 2024-09-13T18:54:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f735ffff700 1 -- 172.21.15.7:0/773342490 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735400c210 con 0x7f7354003a20 2024-09-13T18:54:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736ccc6700 1 --2- 172.21.15.7:0/773342490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7354004420 0x7f73540048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736d4c7700 1 --2- 172.21.15.7:0/773342490 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7354003a20 0x7f7354003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736dcc8700 1 --2- 172.21.15.7:0/773342490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73540050f0 0x7f735400ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736ccc6700 1 -- 172.21.15.7:0/773342490 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7354003a20 msgr2=0x7f7354003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736ccc6700 1 --2- 172.21.15.7:0/773342490 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7354003a20 0x7f7354003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736ccc6700 1 -- 172.21.15.7:0/773342490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73540050f0 msgr2=0x7f735400ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736ccc6700 1 --2- 172.21.15.7:0/773342490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73540050f0 0x7f735400ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736dcc8700 1 -- 172.21.15.7:0/773342490 reap_dead start 2024-09-13T18:54:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736ccc6700 1 -- 172.21.15.7:0/773342490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73540035d0 con 0x7f7354004420 2024-09-13T18:54:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736ccc6700 1 --2- 172.21.15.7:0/773342490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7354004420 0x7f73540048f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7368071b70 tx=0x7f73540058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13ca20d227737ee server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f735ffff700 1 -- 172.21.15.7:0/773342490 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358001390 con 0x7f7354004420 2024-09-13T18:54:58.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f735ffff700 1 -- 172.21.15.7:0/773342490 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73580052f0 con 0x7f7354004420 2024-09-13T18:54:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f735ffff700 1 -- 172.21.15.7:0/773342490 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73580036a0 con 0x7f7354004420 2024-09-13T18:54:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736e4c9700 1 -- 172.21.15.7:0/773342490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7354004420 msgr2=0x7f73540048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736e4c9700 1 --2- 172.21.15.7:0/773342490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7354004420 0x7f73540048f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7368071b70 tx=0x7f73540058a0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736e4c9700 1 -- 172.21.15.7:0/773342490 shutdown_connections 2024-09-13T18:54:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736e4c9700 1 --2- 172.21.15.7:0/773342490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7354004420 0x7f73540048f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736e4c9700 1 -- 172.21.15.7:0/773342490 >> 172.21.15.7:0/773342490 conn(0x7f73680cf090 msgr2=0x7f73680cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736e4c9700 1 -- 172.21.15.7:0/773342490 shutdown_connections 2024-09-13T18:54:58.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.486+0000 7f736e4c9700 1 -- 172.21.15.7:0/773342490 wait complete. 2024-09-13T18:54:58.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.490+0000 7f736e4c9700 1 Processor -- start 2024-09-13T18:54:58.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.490+0000 7f736e4c9700 1 -- start start 2024-09-13T18:54:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.490+0000 7f736e4c9700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7354004420 0x7f73681e8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.490+0000 7f736e4c9700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736806b750 0x7f73681598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.490+0000 7f736e4c9700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73680707c0 0x7f7368070c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.490+0000 7f736e4c9700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7368071180 con 0x7f73680707c0 2024-09-13T18:54:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.490+0000 7f736e4c9700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736813f620 con 0x7f736806b750 2024-09-13T18:54:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.490+0000 7f736e4c9700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73681407a0 con 0x7f7354004420 2024-09-13T18:54:58.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.490+0000 7f736dcc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73680707c0 0x7f7368070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.490+0000 7f736dcc8700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73680707c0 0x7f7368070c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40954/0 (socket says 172.21.15.7:40954) 2024-09-13T18:54:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.490+0000 7f736dcc8700 1 -- 172.21.15.7:0/1219671807 learned_addr learned my addr 172.21.15.7:0/1219671807 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.490+0000 7f736ccc6700 1 --2- 172.21.15.7:0/1219671807 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736806b750 0x7f73681598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.490+0000 7f736d4c7700 1 --2- 172.21.15.7:0/1219671807 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7354004420 0x7f73681e8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.494+0000 7f736dcc8700 1 -- 172.21.15.7:0/1219671807 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7354004420 msgr2=0x7f73681e8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.494+0000 7f736dcc8700 1 --2- 172.21.15.7:0/1219671807 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7354004420 0x7f73681e8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.494+0000 7f736dcc8700 1 -- 172.21.15.7:0/1219671807 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736806b750 msgr2=0x7f73681598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.494+0000 7f736dcc8700 1 --2- 172.21.15.7:0/1219671807 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736806b750 0x7f73681598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.494+0000 7f736dcc8700 1 -- 172.21.15.7:0/1219671807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736813f8b0 con 0x7f73680707c0 2024-09-13T18:54:58.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.494+0000 7f736dcc8700 1 --2- 172.21.15.7:0/1219671807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73680707c0 0x7f7368070c40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7360007cd0 tx=0x7f73600111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:58.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.494+0000 7f735dffb700 1 -- 172.21.15.7:0/1219671807 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736000e070 con 0x7f73680707c0 2024-09-13T18:54:58.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.494+0000 7f735dffb700 1 -- 172.21.15.7:0/1219671807 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7360003d10 con 0x7f73680707c0 2024-09-13T18:54:58.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.494+0000 7f736e4c9700 1 -- 172.21.15.7:0/1219671807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f736813fb90 con 0x7f73680707c0 2024-09-13T18:54:58.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.494+0000 7f735dffb700 1 -- 172.21.15.7:0/1219671807 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73600117c0 con 0x7f73680707c0 2024-09-13T18:54:58.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.494+0000 7f736e4c9700 1 -- 172.21.15.7:0/1219671807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7368140150 con 0x7f73680707c0 2024-09-13T18:54:58.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.494+0000 7f736e4c9700 1 -- 172.21.15.7:0/1219671807 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f736806e210 con 0x7f73680707c0 2024-09-13T18:54:58.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.494+0000 7f735dffb700 1 -- 172.21.15.7:0/1219671807 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f7360011920 con 0x7f73680707c0 2024-09-13T18:54:58.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.494+0000 7f735dffb700 1 --2- 172.21.15.7:0/1219671807 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f734c066760 0x7f734c068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.494+0000 7f735dffb700 1 -- 172.21.15.7:0/1219671807 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f7360084390 con 0x7f73680707c0 2024-09-13T18:54:58.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.498+0000 7f736d4c7700 1 --2- 172.21.15.7:0/1219671807 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f734c066760 0x7f734c068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.498+0000 7f736d4c7700 1 --2- 172.21.15.7:0/1219671807 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f734c066760 0x7f734c068c20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7364001390 tx=0x7f7364000ef0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:58.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.502+0000 7f735dffb700 1 -- 172.21.15.7:0/1219671807 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f73600519a0 con 0x7f73680707c0 2024-09-13T18:54:58.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.666+0000 7f736e4c9700 1 -- 172.21.15.7:0/1219671807 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f73680ccb90 con 0x7f734c066760 2024-09-13T18:54:58.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.666+0000 7f735dffb700 1 -- 172.21.15.7:0/1219671807 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f73640070c0 con 0x7f734c066760 2024-09-13T18:54:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.670+0000 7f736e4c9700 1 -- 172.21.15.7:0/1219671807 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f734c066760 msgr2=0x7f734c068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.670+0000 7f736e4c9700 1 --2- 172.21.15.7:0/1219671807 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f734c066760 0x7f734c068c20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7364001390 tx=0x7f7364000ef0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.670+0000 7f736e4c9700 1 -- 172.21.15.7:0/1219671807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73680707c0 msgr2=0x7f7368070c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.670+0000 7f736e4c9700 1 --2- 172.21.15.7:0/1219671807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73680707c0 0x7f7368070c40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7360007cd0 tx=0x7f73600111d0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.670+0000 7f736e4c9700 1 -- 172.21.15.7:0/1219671807 shutdown_connections 2024-09-13T18:54:58.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.670+0000 7f736e4c9700 1 --2- 172.21.15.7:0/1219671807 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f734c066760 0x7f734c068c20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.670+0000 7f736e4c9700 1 --2- 172.21.15.7:0/1219671807 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73680707c0 0x7f7368070c40 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.670+0000 7f736e4c9700 1 --2- 172.21.15.7:0/1219671807 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f736806b750 0x7f73681598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.670+0000 7f736e4c9700 1 --2- 172.21.15.7:0/1219671807 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f7354004420 0x7f73681e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.670+0000 7f736e4c9700 1 -- 172.21.15.7:0/1219671807 >> 172.21.15.7:0/1219671807 conn(0x7f73680cf090 msgr2=0x7f7368156320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.670+0000 7f736e4c9700 1 -- 172.21.15.7:0/1219671807 shutdown_connections 2024-09-13T18:54:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.670+0000 7f736e4c9700 1 -- 172.21.15.7:0/1219671807 wait complete. 2024-09-13T18:54:58.677 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T18:54:58.689 INFO:tasks.ceph:Scrubbing osd.4 2024-09-13T18:54:58.689 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:54:58.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.806+0000 7f0996d69700 1 Processor -- start 2024-09-13T18:54:58.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.806+0000 7f0996d69700 1 -- start start 2024-09-13T18:54:58.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0996d69700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09900cc630 0x7f09900cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0996d69700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09900ce270 con 0x7f099006bb60 2024-09-13T18:54:58.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0996d69700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09900ce550 con 0x7f0990158f20 2024-09-13T18:54:58.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0996d69700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099006da90 con 0x7f09900cc630 2024-09-13T18:54:58.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0996568700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09900cc630 0x7f09900cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0995566700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f099006bb60 0x7f09900b2430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58076/0 (socket says 172.21.15.7:58076) 2024-09-13T18:54:58.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0995566700 1 -- 172.21.15.7:0/2688173367 learned_addr learned my addr 172.21.15.7:0/2688173367 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:58.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0996568700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09900cc630 0x7f09900cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40966/0 (socket says 172.21.15.7:40966) 2024-09-13T18:54:58.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0996568700 1 -- 172.21.15.7:0/2688173367 >> v1:172.21.15.171:6789/0 conn(0x7f0990158f20 legacy=0x7f09900cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0996568700 1 -- 172.21.15.7:0/2688173367 >> v1:172.21.15.7:6789/0 conn(0x7f099006bb60 legacy=0x7f09900b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0996568700 1 -- 172.21.15.7:0/2688173367 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09900cd770 con 0x7f09900cc630 2024-09-13T18:54:58.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0994d65700 1 -- 172.21.15.7:0/2688173367 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804432097 0 0) 0x7f09900ce270 con 0x7f099006bb60 2024-09-13T18:54:58.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0996568700 1 --2- 172.21.15.7:0/2688173367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09900cc630 0x7f09900cb590 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0988009d90 tx=0x7f098800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0c892b71d14572 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:58.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0994d65700 1 -- 172.21.15.7:0/2688173367 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098800e070 con 0x7f09900cc630 2024-09-13T18:54:58.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0994d65700 1 -- 172.21.15.7:0/2688173367 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098800bbd0 con 0x7f09900cc630 2024-09-13T18:54:58.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0994d65700 1 -- 172.21.15.7:0/2688173367 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0988012430 con 0x7f09900cc630 2024-09-13T18:54:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0996d69700 1 -- 172.21.15.7:0/2688173367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09900cc630 msgr2=0x7f09900cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0996d69700 1 --2- 172.21.15.7:0/2688173367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09900cc630 0x7f09900cb590 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0988009d90 tx=0x7f098800b330 comp rx=0 tx=0).stop 2024-09-13T18:54:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0996d69700 1 -- 172.21.15.7:0/2688173367 shutdown_connections 2024-09-13T18:54:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0996d69700 1 --2- 172.21.15.7:0/2688173367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09900cc630 0x7f09900cb590 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0996d69700 1 -- 172.21.15.7:0/2688173367 >> 172.21.15.7:0/2688173367 conn(0x7f09900b0cc0 msgr2=0x7f099006fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0996d69700 1 -- 172.21.15.7:0/2688173367 shutdown_connections 2024-09-13T18:54:58.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.810+0000 7f0996d69700 1 -- 172.21.15.7:0/2688173367 wait complete. 2024-09-13T18:54:58.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.814+0000 7f0996d69700 1 Processor -- start 2024-09-13T18:54:58.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.814+0000 7f0996d69700 1 -- start start 2024-09-13T18:54:58.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.814+0000 7f0996d69700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09900cc630 0x7f099014ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.814+0000 7f0996d69700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0990158f20 0x7f09901552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.814+0000 7f0996d69700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f099015aa00 0x7f0990153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.814+0000 7f0996d69700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099014f430 con 0x7f09900cc630 2024-09-13T18:54:58.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.814+0000 7f0996d69700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09901557e0 con 0x7f0990158f20 2024-09-13T18:54:58.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.814+0000 7f0996d69700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0990157a20 con 0x7f099015aa00 2024-09-13T18:54:58.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.814+0000 7f0995d67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09900cc630 0x7f099014ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.814+0000 7f0995d67700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09900cc630 0x7f099014ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40974/0 (socket says 172.21.15.7:40974) 2024-09-13T18:54:58.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.814+0000 7f0995d67700 1 -- 172.21.15.7:0/3911268523 learned_addr learned my addr 172.21.15.7:0/3911268523 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:58.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.814+0000 7f0995566700 1 --2- 172.21.15.7:0/3911268523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0990158f20 0x7f09901552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.814+0000 7f0996568700 1 --2- 172.21.15.7:0/3911268523 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f099015aa00 0x7f0990153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.818+0000 7f0995d67700 1 -- 172.21.15.7:0/3911268523 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f099015aa00 msgr2=0x7f0990153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.818+0000 7f0995d67700 1 --2- 172.21.15.7:0/3911268523 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f099015aa00 0x7f0990153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.818+0000 7f0995d67700 1 -- 172.21.15.7:0/3911268523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0990158f20 msgr2=0x7f09901552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.818+0000 7f0995d67700 1 --2- 172.21.15.7:0/3911268523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0990158f20 0x7f09901552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.818+0000 7f0995d67700 1 -- 172.21.15.7:0/3911268523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0988009a70 con 0x7f09900cc630 2024-09-13T18:54:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.818+0000 7f0995566700 1 --2- 172.21.15.7:0/3911268523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0990158f20 0x7f09901552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:54:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.818+0000 7f0995d67700 1 --2- 172.21.15.7:0/3911268523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09900cc630 0x7f099014ec60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f098c00c500 tx=0x7f098c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.818+0000 7f09867fc700 1 -- 172.21.15.7:0/3911268523 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c0052d0 con 0x7f09900cc630 2024-09-13T18:54:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.818+0000 7f09867fc700 1 -- 172.21.15.7:0/3911268523 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c005430 con 0x7f09900cc630 2024-09-13T18:54:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.818+0000 7f0996d69700 1 -- 172.21.15.7:0/3911268523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0990157170 con 0x7f09900cc630 2024-09-13T18:54:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.818+0000 7f09867fc700 1 -- 172.21.15.7:0/3911268523 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c0055a0 con 0x7f09900cc630 2024-09-13T18:54:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.818+0000 7f0996d69700 1 -- 172.21.15.7:0/3911268523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09901559f0 con 0x7f09900cc630 2024-09-13T18:54:58.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.818+0000 7f0996d69700 1 -- 172.21.15.7:0/3911268523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f09900cdf40 con 0x7f09900cc630 2024-09-13T18:54:58.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.818+0000 7f09867fc700 1 -- 172.21.15.7:0/3911268523 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f098c0038e0 con 0x7f09900cc630 2024-09-13T18:54:58.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.822+0000 7f09867fc700 1 --2- 172.21.15.7:0/3911268523 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0974066730 0x7f0974068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.822+0000 7f09867fc700 1 -- 172.21.15.7:0/3911268523 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f098c083a00 con 0x7f09900cc630 2024-09-13T18:54:58.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.822+0000 7f0995566700 1 --2- 172.21.15.7:0/3911268523 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0974066730 0x7f0974068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.822+0000 7f09867fc700 1 --2- 172.21.15.7:0/3911268523 >> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] conn(0x7f097406c630 0x7f097406ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.822+0000 7f09867fc700 1 -- 172.21.15.7:0/3911268523 --> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f097406f100 con 0x7f097406c630 2024-09-13T18:54:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.822+0000 7f09867fc700 1 -- 172.21.15.7:0/3911268523 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f098c083e50 con 0x7f09900cc630 2024-09-13T18:54:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.822+0000 7f0996568700 1 --2- 172.21.15.7:0/3911268523 >> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] conn(0x7f097406c630 0x7f097406ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.822+0000 7f0995566700 1 --2- 172.21.15.7:0/3911268523 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0974066730 0x7f0974068bf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0990164070 tx=0x7f0980009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:58.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.822+0000 7f0996568700 1 --2- 172.21.15.7:0/3911268523 >> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] conn(0x7f097406c630 0x7f097406ea50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:58.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.822+0000 7f09867fc700 1 -- 172.21.15.7:0/3911268523 <== osd.4 v2:172.21.15.171:6816/1731498321 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f097406f100 con 0x7f097406c630 2024-09-13T18:54:58.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.842+0000 7f0996d69700 1 -- 172.21.15.7:0/3911268523 --> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f09900ccba0 con 0x7f097406c630 2024-09-13T18:54:58.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.842+0000 7f09867fc700 1 -- 172.21.15.7:0/3911268523 <== osd.4 v2:172.21.15.171:6816/1731498321 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f09880098f0 con 0x7f097406c630 2024-09-13T18:54:58.850 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:54:58.850 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:54:58.850 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:54:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.846+0000 7f0996d69700 1 -- 172.21.15.7:0/3911268523 >> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] conn(0x7f097406c630 msgr2=0x7f097406ea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.846+0000 7f0996d69700 1 --2- 172.21.15.7:0/3911268523 >> [v2:172.21.15.171:6816/1731498321,v1:172.21.15.171:6818/1731498321] conn(0x7f097406c630 0x7f097406ea50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.846+0000 7f0996d69700 1 -- 172.21.15.7:0/3911268523 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0974066730 msgr2=0x7f0974068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.846+0000 7f0996d69700 1 --2- 172.21.15.7:0/3911268523 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f0974066730 0x7f0974068bf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0990164070 tx=0x7f0980009040 comp rx=0 tx=0).stop 2024-09-13T18:54:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.846+0000 7f0996d69700 1 -- 172.21.15.7:0/3911268523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09900cc630 msgr2=0x7f099014ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.846+0000 7f0996d69700 1 --2- 172.21.15.7:0/3911268523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09900cc630 0x7f099014ec60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f098c00c500 tx=0x7f098c00c8c0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.846+0000 7f0996568700 1 -- 172.21.15.7:0/3911268523 reap_dead start 2024-09-13T18:54:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.846+0000 7f0996d69700 1 -- 172.21.15.7:0/3911268523 shutdown_connections 2024-09-13T18:54:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.846+0000 7f0996d69700 1 -- 172.21.15.7:0/3911268523 >> 172.21.15.7:0/3911268523 conn(0x7f09900b0cc0 msgr2=0x7f09900cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.846+0000 7f0996d69700 1 -- 172.21.15.7:0/3911268523 shutdown_connections 2024-09-13T18:54:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.846+0000 7f0996d69700 1 -- 172.21.15.7:0/3911268523 wait complete. 2024-09-13T18:54:58.863 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-13T18:54:58.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.974+0000 7f857e211700 1 Processor -- start 2024-09-13T18:54:58.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.974+0000 7f857e211700 1 -- start start 2024-09-13T18:54:58.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.974+0000 7f857e211700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f85781444c0 0x7f85780a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.974+0000 7f857e211700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857815c3d0 con 0x7f857815b870 2024-09-13T18:54:58.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.974+0000 7f857e211700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857815c6b0 con 0x7f857806b2b0 2024-09-13T18:54:58.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.974+0000 7f857e211700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f857815d050 con 0x7f85781444c0 2024-09-13T18:54:58.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.974+0000 7f857ca0e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f857815b870 0x7f857806aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58086/0 (socket says 172.21.15.7:58086) 2024-09-13T18:54:58.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.974+0000 7f857ca0e700 1 -- 172.21.15.7:0/3628705581 learned_addr learned my addr 172.21.15.7:0/3628705581 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:58.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.974+0000 7f857d20f700 1 --2- 172.21.15.7:0/3628705581 >> v2:172.21.15.171:3300/0 conn(0x7f85781444c0 0x7f85780a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.974+0000 7f8577fff700 1 -- 172.21.15.7:0/3628705581 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 811010981 0 0) 0x7f857815c3d0 con 0x7f857815b870 2024-09-13T18:54:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.974+0000 7f8577fff700 1 -- 172.21.15.7:0/3628705581 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85600035d0 con 0x7f857815b870 2024-09-13T18:54:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.978+0000 7f857d20f700 1 -- 172.21.15.7:0/3628705581 >> v1:172.21.15.171:6789/0 conn(0x7f857806b2b0 legacy=0x7f857815d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.978+0000 7f857d20f700 1 -- 172.21.15.7:0/3628705581 >> v1:172.21.15.7:6789/0 conn(0x7f857815b870 legacy=0x7f857806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.978+0000 7f857d20f700 1 -- 172.21.15.7:0/3628705581 --> v2:172.21.15.171:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857815c990 con 0x7f85781444c0 2024-09-13T18:54:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.978+0000 7f857d20f700 1 --2- 172.21.15.7:0/3628705581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85781444c0 0x7f85780a84b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8570009850 tx=0x7f8570009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d445064aae4b4ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.978+0000 7f8577fff700 1 -- 172.21.15.7:0/3628705581 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8570011830 con 0x7f85781444c0 2024-09-13T18:54:58.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.978+0000 7f8577fff700 1 -- 172.21.15.7:0/3628705581 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8570011990 con 0x7f85781444c0 2024-09-13T18:54:58.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.978+0000 7f8577fff700 1 -- 172.21.15.7:0/3628705581 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8570011d00 con 0x7f85781444c0 2024-09-13T18:54:58.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.978+0000 7f857e211700 1 -- 172.21.15.7:0/3628705581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85781444c0 msgr2=0x7f85780a84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.978+0000 7f857e211700 1 --2- 172.21.15.7:0/3628705581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85781444c0 0x7f85780a84b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8570009850 tx=0x7f8570009b60 comp rx=0 tx=0).stop 2024-09-13T18:54:58.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.978+0000 7f857e211700 1 -- 172.21.15.7:0/3628705581 shutdown_connections 2024-09-13T18:54:58.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.978+0000 7f857e211700 1 --2- 172.21.15.7:0/3628705581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85781444c0 0x7f85780a84b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.978+0000 7f857e211700 1 -- 172.21.15.7:0/3628705581 >> 172.21.15.7:0/3628705581 conn(0x7f85780b14c0 msgr2=0x7f85780b18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:58.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.978+0000 7f857e211700 1 -- 172.21.15.7:0/3628705581 shutdown_connections 2024-09-13T18:54:58.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.978+0000 7f857e211700 1 -- 172.21.15.7:0/3628705581 wait complete. 2024-09-13T18:54:58.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.982+0000 7f857e211700 1 Processor -- start 2024-09-13T18:54:58.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.982+0000 7f857e211700 1 -- start start 2024-09-13T18:54:58.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.982+0000 7f857e211700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f857806b2b0 0x7f8578154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.982+0000 7f857e211700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85781444c0 0x7f8578142a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.982+0000 7f857e211700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f857815b870 0x7f857813fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.982+0000 7f857e211700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85780c5a10 con 0x7f857806b2b0 2024-09-13T18:54:58.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.982+0000 7f857e211700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85780c5cf0 con 0x7f85781444c0 2024-09-13T18:54:58.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.982+0000 7f857e211700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85780c5fd0 con 0x7f857815b870 2024-09-13T18:54:58.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.982+0000 7f857d20f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f857806b2b0 0x7f8578154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.982+0000 7f857d20f700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f857806b2b0 0x7f8578154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40980/0 (socket says 172.21.15.7:40980) 2024-09-13T18:54:58.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.982+0000 7f857d20f700 1 -- 172.21.15.7:0/3522778345 learned_addr learned my addr 172.21.15.7:0/3522778345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:58.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.982+0000 7f857ca0e700 1 --2- 172.21.15.7:0/3522778345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85781444c0 0x7f8578142a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.982+0000 7f857da10700 1 --2- 172.21.15.7:0/3522778345 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f857815b870 0x7f857813fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.986+0000 7f857d20f700 1 -- 172.21.15.7:0/3522778345 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f857815b870 msgr2=0x7f857813fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.986+0000 7f857d20f700 1 --2- 172.21.15.7:0/3522778345 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f857815b870 0x7f857813fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.986+0000 7f857d20f700 1 -- 172.21.15.7:0/3522778345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85781444c0 msgr2=0x7f8578142a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:58.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.986+0000 7f857d20f700 1 --2- 172.21.15.7:0/3522778345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85781444c0 0x7f8578142a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:58.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.986+0000 7f857d20f700 1 -- 172.21.15.7:0/3522778345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85700094b0 con 0x7f857806b2b0 2024-09-13T18:54:58.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.986+0000 7f857da10700 1 --2- 172.21.15.7:0/3522778345 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f857815b870 0x7f857813fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:54:58.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.986+0000 7f857d20f700 1 --2- 172.21.15.7:0/3522778345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f857806b2b0 0x7f8578154050 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8570009ba0 tx=0x7f8570004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:58.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.986+0000 7f8575ffb700 1 -- 172.21.15.7:0/3522778345 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85700043c0 con 0x7f857806b2b0 2024-09-13T18:54:58.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.986+0000 7f8575ffb700 1 -- 172.21.15.7:0/3522778345 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857001f070 con 0x7f857806b2b0 2024-09-13T18:54:58.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.986+0000 7f857e211700 1 -- 172.21.15.7:0/3522778345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85780c62b0 con 0x7f857806b2b0 2024-09-13T18:54:58.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.986+0000 7f8575ffb700 1 -- 172.21.15.7:0/3522778345 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85700036c0 con 0x7f857806b2b0 2024-09-13T18:54:58.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.986+0000 7f857e211700 1 -- 172.21.15.7:0/3522778345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8578154590 con 0x7f857806b2b0 2024-09-13T18:54:58.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.986+0000 7f857e211700 1 -- 172.21.15.7:0/3522778345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85780a84b0 con 0x7f857806b2b0 2024-09-13T18:54:58.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.986+0000 7f8575ffb700 1 -- 172.21.15.7:0/3522778345 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f8570021030 con 0x7f857806b2b0 2024-09-13T18:54:58.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.986+0000 7f8575ffb700 1 --2- 172.21.15.7:0/3522778345 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8564066730 0x7f8564068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:58.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.986+0000 7f8575ffb700 1 -- 172.21.15.7:0/3522778345 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f8570084eb0 con 0x7f857806b2b0 2024-09-13T18:54:58.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.986+0000 7f857ca0e700 1 --2- 172.21.15.7:0/3522778345 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8564066730 0x7f8564068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:58.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.990+0000 7f857ca0e700 1 --2- 172.21.15.7:0/3522778345 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8564066730 0x7f8564068bf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f857815aad0 tx=0x7f856c007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:58.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:58.994+0000 7f8575ffb700 1 -- 172.21.15.7:0/3522778345 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8570052670 con 0x7f857806b2b0 2024-09-13T18:54:59.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.162+0000 7f857e211700 1 -- 172.21.15.7:0/3522778345 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f857815bd00 con 0x7f8564066730 2024-09-13T18:54:59.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.162+0000 7f8575ffb700 1 -- 172.21.15.7:0/3522778345 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f856c014450 con 0x7f8564066730 2024-09-13T18:54:59.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.166+0000 7f857e211700 1 -- 172.21.15.7:0/3522778345 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8564066730 msgr2=0x7f8564068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.166+0000 7f857e211700 1 --2- 172.21.15.7:0/3522778345 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8564066730 0x7f8564068bf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f857815aad0 tx=0x7f856c007de0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.166+0000 7f857e211700 1 -- 172.21.15.7:0/3522778345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f857806b2b0 msgr2=0x7f8578154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.166+0000 7f857e211700 1 --2- 172.21.15.7:0/3522778345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f857806b2b0 0x7f8578154050 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8570009ba0 tx=0x7f8570004770 comp rx=0 tx=0).stop 2024-09-13T18:54:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.166+0000 7f857e211700 1 -- 172.21.15.7:0/3522778345 shutdown_connections 2024-09-13T18:54:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.166+0000 7f857e211700 1 --2- 172.21.15.7:0/3522778345 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8564066730 0x7f8564068bf0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.166+0000 7f857e211700 1 --2- 172.21.15.7:0/3522778345 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f857815b870 0x7f857813fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.166+0000 7f857e211700 1 --2- 172.21.15.7:0/3522778345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f85781444c0 0x7f8578142a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.166+0000 7f857e211700 1 --2- 172.21.15.7:0/3522778345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f857806b2b0 0x7f8578154050 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.166+0000 7f857e211700 1 -- 172.21.15.7:0/3522778345 >> 172.21.15.7:0/3522778345 conn(0x7f85780b14c0 msgr2=0x7f85780b18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.166+0000 7f857e211700 1 -- 172.21.15.7:0/3522778345 shutdown_connections 2024-09-13T18:54:59.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.166+0000 7f857e211700 1 -- 172.21.15.7:0/3522778345 wait complete. 2024-09-13T18:54:59.173 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 4 to deep-scrub 2024-09-13T18:54:59.184 INFO:tasks.ceph:Scrubbing osd.5 2024-09-13T18:54:59.184 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:54:59.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.294+0000 7f8240ac2700 1 Processor -- start 2024-09-13T18:54:59.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.294+0000 7f8240ac2700 1 -- start start 2024-09-13T18:54:59.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f8240ac2700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f823c0ceaf0 0x7f823c0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f8240ac2700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f823c160d90 0x7f823c1611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f8240ac2700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823c06aa30 con 0x7f823c0ce710 2024-09-13T18:54:59.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f8240ac2700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823c0d0660 con 0x7f823c0ceaf0 2024-09-13T18:54:59.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f8240ac2700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f823c0b1e60 con 0x7f823c160d90 2024-09-13T18:54:59.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f823affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f823c0ceaf0 0x7f823c0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f823affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f823c0ceaf0 0x7f823c0b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40994/0 (socket says 172.21.15.7:40994) 2024-09-13T18:54:59.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f823affd700 1 -- 172.21.15.7:0/3834273611 learned_addr learned my addr 172.21.15.7:0/3834273611 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:59.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f823bfff700 1 --2- 172.21.15.7:0/3834273611 >> v2:172.21.15.171:3300/0 conn(0x7f823c160d90 0x7f823c1611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f823affd700 1 -- 172.21.15.7:0/3834273611 >> v2:172.21.15.171:3300/0 conn(0x7f823c160d90 msgr2=0x7f823c1611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:59.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f823affd700 1 --2- 172.21.15.7:0/3834273611 >> v2:172.21.15.171:3300/0 conn(0x7f823c160d90 0x7f823c1611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f823affd700 1 -- 172.21.15.7:0/3834273611 >> v1:172.21.15.171:6789/0 conn(0x7f823c0ce710 legacy=0x7f823c0cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f823affd700 1 -- 172.21.15.7:0/3834273611 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c0b18a0 con 0x7f823c0ceaf0 2024-09-13T18:54:59.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f823affd700 1 --2- 172.21.15.7:0/3834273611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0ceaf0 0x7f823c0b5da0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f823c0b8da0 tx=0x7f822c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d39a95e54626be server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:59.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f823a7fc700 1 -- 172.21.15.7:0/3834273611 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c011830 con 0x7f823c0ceaf0 2024-09-13T18:54:59.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f823a7fc700 1 -- 172.21.15.7:0/3834273611 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c011990 con 0x7f823c0ceaf0 2024-09-13T18:54:59.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f823a7fc700 1 -- 172.21.15.7:0/3834273611 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c011d20 con 0x7f823c0ceaf0 2024-09-13T18:54:59.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f8240ac2700 1 -- 172.21.15.7:0/3834273611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0ceaf0 msgr2=0x7f823c0b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f8240ac2700 1 --2- 172.21.15.7:0/3834273611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0ceaf0 0x7f823c0b5da0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f823c0b8da0 tx=0x7f822c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f8240ac2700 1 -- 172.21.15.7:0/3834273611 shutdown_connections 2024-09-13T18:54:59.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f8240ac2700 1 --2- 172.21.15.7:0/3834273611 >> v2:172.21.15.171:3300/0 conn(0x7f823c160d90 0x7f823c1611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f8240ac2700 1 --2- 172.21.15.7:0/3834273611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c0ceaf0 0x7f823c0b5da0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f8240ac2700 1 -- 172.21.15.7:0/3834273611 >> 172.21.15.7:0/3834273611 conn(0x7f823c0d3260 msgr2=0x7f823c0cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:59.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f8240ac2700 1 -- 172.21.15.7:0/3834273611 shutdown_connections 2024-09-13T18:54:59.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.298+0000 7f8240ac2700 1 -- 172.21.15.7:0/3834273611 wait complete. 2024-09-13T18:54:59.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.302+0000 7f8240ac2700 1 Processor -- start 2024-09-13T18:54:59.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.302+0000 7f8240ac2700 1 -- start start 2024-09-13T18:54:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f8240ac2700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f823c0ce710 0x7f823c0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f8240ac2700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f823c0ceaf0 0x7f823c157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f8240ac2700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c160d90 0x7f823c15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f8240ac2700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c0ccbe0 con 0x7f823c160d90 2024-09-13T18:54:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f8240ac2700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823c1592f0 con 0x7f823c0ceaf0 2024-09-13T18:54:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f8240ac2700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823c1595d0 con 0x7f823c0ce710 2024-09-13T18:54:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f823bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c160d90 0x7f823c15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f823affd700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f823c0ceaf0 0x7f823c157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f823bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c160d90 0x7f823c15a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40996/0 (socket says 172.21.15.7:40996) 2024-09-13T18:54:59.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f823bfff700 1 -- 172.21.15.7:0/2483145298 learned_addr learned my addr 172.21.15.7:0/2483145298 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:59.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f823b7fe700 1 --2- 172.21.15.7:0/2483145298 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f823c0ce710 0x7f823c0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f823affd700 1 -- 172.21.15.7:0/2483145298 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f823c0ce710 msgr2=0x7f823c0cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f823affd700 1 --2- 172.21.15.7:0/2483145298 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f823c0ce710 0x7f823c0cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f823affd700 1 -- 172.21.15.7:0/2483145298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c160d90 msgr2=0x7f823c15a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f823affd700 1 --2- 172.21.15.7:0/2483145298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f823c160d90 0x7f823c15a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f823affd700 1 -- 172.21.15.7:0/2483145298 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c009570 con 0x7f823c0ceaf0 2024-09-13T18:54:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f823affd700 1 --2- 172.21.15.7:0/2483145298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f823c0ceaf0 0x7f823c157300 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f822c016000 tx=0x7f822c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f8223fff700 1 -- 172.21.15.7:0/2483145298 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c004450 con 0x7f823c0ceaf0 2024-09-13T18:54:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f8223fff700 1 -- 172.21.15.7:0/2483145298 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c01f070 con 0x7f823c0ceaf0 2024-09-13T18:54:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f8240ac2700 1 -- 172.21.15.7:0/2483145298 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823c1598b0 con 0x7f823c0ceaf0 2024-09-13T18:54:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f8223fff700 1 -- 172.21.15.7:0/2483145298 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c003900 con 0x7f823c0ceaf0 2024-09-13T18:54:59.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f8240ac2700 1 -- 172.21.15.7:0/2483145298 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823c152fa0 con 0x7f823c0ceaf0 2024-09-13T18:54:59.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.306+0000 7f8240ac2700 1 -- 172.21.15.7:0/2483145298 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f823c0c9310 con 0x7f823c0ceaf0 2024-09-13T18:54:59.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.310+0000 7f8223fff700 1 -- 172.21.15.7:0/2483145298 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f822c003a60 con 0x7f823c0ceaf0 2024-09-13T18:54:59.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.310+0000 7f8223fff700 1 --2- 172.21.15.7:0/2483145298 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8228066730 0x7f8228068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.310+0000 7f8223fff700 1 -- 172.21.15.7:0/2483145298 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f822c021070 con 0x7f823c0ceaf0 2024-09-13T18:54:59.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.310+0000 7f8223fff700 1 --2- 172.21.15.7:0/2483145298 >> [v2:172.21.15.171:6800/1011659457,v1:172.21.15.171:6801/1011659457] conn(0x7f822806c630 0x7f822806ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.310+0000 7f8223fff700 1 -- 172.21.15.7:0/2483145298 --> [v2:172.21.15.171:6800/1011659457,v1:172.21.15.171:6801/1011659457] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f822806f100 con 0x7f822806c630 2024-09-13T18:54:59.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.310+0000 7f8223fff700 1 -- 172.21.15.7:0/2483145298 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f822c085950 con 0x7f823c0ceaf0 2024-09-13T18:54:59.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.310+0000 7f823b7fe700 1 --2- 172.21.15.7:0/2483145298 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8228066730 0x7f8228068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.310+0000 7f823bfff700 1 --2- 172.21.15.7:0/2483145298 >> [v2:172.21.15.171:6800/1011659457,v1:172.21.15.171:6801/1011659457] conn(0x7f822806c630 0x7f822806ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.310+0000 7f823b7fe700 1 --2- 172.21.15.7:0/2483145298 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8228066730 0x7f8228068bf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f823c1f62b0 tx=0x7f8230009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:59.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.310+0000 7f823bfff700 1 --2- 172.21.15.7:0/2483145298 >> [v2:172.21.15.171:6800/1011659457,v1:172.21.15.171:6801/1011659457] conn(0x7f822806c630 0x7f822806ea50 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:59.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.314+0000 7f8223fff700 1 -- 172.21.15.7:0/2483145298 <== osd.5 v2:172.21.15.171:6800/1011659457 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f822806f100 con 0x7f822806c630 2024-09-13T18:54:59.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.334+0000 7f8240ac2700 1 -- 172.21.15.7:0/2483145298 --> [v2:172.21.15.171:6800/1011659457,v1:172.21.15.171:6801/1011659457] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f823c15da70 con 0x7f822806c630 2024-09-13T18:54:59.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.334+0000 7f8223fff700 1 -- 172.21.15.7:0/2483145298 <== osd.5 v2:172.21.15.171:6800/1011659457 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f823c15da70 con 0x7f822806c630 2024-09-13T18:54:59.341 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:54:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:54:59.341 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:54:59.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.334+0000 7f8240ac2700 1 -- 172.21.15.7:0/2483145298 >> [v2:172.21.15.171:6800/1011659457,v1:172.21.15.171:6801/1011659457] conn(0x7f822806c630 msgr2=0x7f822806ea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.334+0000 7f8240ac2700 1 --2- 172.21.15.7:0/2483145298 >> [v2:172.21.15.171:6800/1011659457,v1:172.21.15.171:6801/1011659457] conn(0x7f822806c630 0x7f822806ea50 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.334+0000 7f8240ac2700 1 -- 172.21.15.7:0/2483145298 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8228066730 msgr2=0x7f8228068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.334+0000 7f8240ac2700 1 --2- 172.21.15.7:0/2483145298 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f8228066730 0x7f8228068bf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f823c1f62b0 tx=0x7f8230009040 comp rx=0 tx=0).stop 2024-09-13T18:54:59.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.334+0000 7f8240ac2700 1 -- 172.21.15.7:0/2483145298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f823c0ceaf0 msgr2=0x7f823c157300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.334+0000 7f8240ac2700 1 --2- 172.21.15.7:0/2483145298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f823c0ceaf0 0x7f823c157300 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f822c016000 tx=0x7f822c004770 comp rx=0 tx=0).stop 2024-09-13T18:54:59.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.334+0000 7f823bfff700 1 -- 172.21.15.7:0/2483145298 reap_dead start 2024-09-13T18:54:59.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.334+0000 7f8240ac2700 1 -- 172.21.15.7:0/2483145298 shutdown_connections 2024-09-13T18:54:59.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.334+0000 7f8240ac2700 1 -- 172.21.15.7:0/2483145298 >> 172.21.15.7:0/2483145298 conn(0x7f823c0d3260 msgr2=0x7f823c15e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:59.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.334+0000 7f8240ac2700 1 -- 172.21.15.7:0/2483145298 shutdown_connections 2024-09-13T18:54:59.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.334+0000 7f8240ac2700 1 -- 172.21.15.7:0/2483145298 wait complete. 2024-09-13T18:54:59.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-13T18:54:59.423 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:54:59.426 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:54:59.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.462+0000 7f6239004700 1 Processor -- start 2024-09-13T18:54:59.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.462+0000 7f6239004700 1 -- start start 2024-09-13T18:54:59.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.462+0000 7f6239004700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6234145e10 0x7f62340cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.462+0000 7f6239004700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62340cc250 0x7f62340b3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.462+0000 7f6239004700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62341461f0 con 0x7f62340b37b0 2024-09-13T18:54:59.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.462+0000 7f6239004700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62341466f0 con 0x7f62340cc250 2024-09-13T18:54:59.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.462+0000 7f6239004700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62340cc650 con 0x7f6234145e10 2024-09-13T18:54:59.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.462+0000 7f62337fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62340cc250 0x7f62340b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.462+0000 7f62337fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62340cc250 0x7f62340b3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41004/0 (socket says 172.21.15.7:41004) 2024-09-13T18:54:59.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.462+0000 7f62337fe700 1 -- 172.21.15.7:0/2510334450 learned_addr learned my addr 172.21.15.7:0/2510334450 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:59.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.462+0000 7f6233fff700 1 --2- 172.21.15.7:0/2510334450 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6234145e10 0x7f62340cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.462+0000 7f62337fe700 1 -- 172.21.15.7:0/2510334450 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6234145e10 msgr2=0x7f62340cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:59.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.462+0000 7f62337fe700 1 --2- 172.21.15.7:0/2510334450 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6234145e10 0x7f62340cbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.462+0000 7f62337fe700 1 -- 172.21.15.7:0/2510334450 >> v1:172.21.15.171:6789/0 conn(0x7f62340b37b0 legacy=0x7f62340ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:59.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.462+0000 7f62337fe700 1 -- 172.21.15.7:0/2510334450 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62340b3bf0 con 0x7f62340cc250 2024-09-13T18:54:59.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.462+0000 7f6233fff700 1 --2- 172.21.15.7:0/2510334450 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6234145e10 0x7f62340cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-13T18:54:59.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.466+0000 7f62337fe700 1 --2- 172.21.15.7:0/2510334450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62340cc250 0x7f62340b3270 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f621c009910 tx=0x7f621c009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d9b31a2f95c021 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:59.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.466+0000 7f6232ffd700 1 -- 172.21.15.7:0/2510334450 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c004230 con 0x7f62340cc250 2024-09-13T18:54:59.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.466+0000 7f6232ffd700 1 -- 172.21.15.7:0/2510334450 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621c004390 con 0x7f62340cc250 2024-09-13T18:54:59.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.466+0000 7f6232ffd700 1 -- 172.21.15.7:0/2510334450 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c009e60 con 0x7f62340cc250 2024-09-13T18:54:59.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.466+0000 7f6239004700 1 -- 172.21.15.7:0/2510334450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62340cc250 msgr2=0x7f62340b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.466+0000 7f6239004700 1 --2- 172.21.15.7:0/2510334450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62340cc250 0x7f62340b3270 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f621c009910 tx=0x7f621c009c20 comp rx=0 tx=0).stop 2024-09-13T18:54:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.466+0000 7f6239004700 1 -- 172.21.15.7:0/2510334450 shutdown_connections 2024-09-13T18:54:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.466+0000 7f6239004700 1 --2- 172.21.15.7:0/2510334450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62340cc250 0x7f62340b3270 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.466+0000 7f6239004700 1 --2- 172.21.15.7:0/2510334450 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f6234145e10 0x7f62340cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.466+0000 7f6239004700 1 -- 172.21.15.7:0/2510334450 >> 172.21.15.7:0/2510334450 conn(0x7f62340b0e10 msgr2=0x7f62340b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.466+0000 7f6239004700 1 -- 172.21.15.7:0/2510334450 shutdown_connections 2024-09-13T18:54:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.466+0000 7f6239004700 1 -- 172.21.15.7:0/2510334450 wait complete. 2024-09-13T18:54:59.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.466+0000 7f6239004700 1 Processor -- start 2024-09-13T18:54:59.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.466+0000 7f6239004700 1 -- start start 2024-09-13T18:54:59.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f6239004700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f62340b37b0 0x7f6234140010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f6239004700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62340cc250 0x7f623413dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f6239004700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6234145e10 0x7f623406fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f6239004700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623413fbf0 con 0x7f62340cc250 2024-09-13T18:54:59.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f6239004700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6234140550 con 0x7f6234145e10 2024-09-13T18:54:59.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f6239004700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623413e010 con 0x7f62340b37b0 2024-09-13T18:54:59.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f62337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62340cc250 0x7f623413dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f62337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62340cc250 0x7f623413dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41016/0 (socket says 172.21.15.7:41016) 2024-09-13T18:54:59.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f62337fe700 1 -- 172.21.15.7:0/3479576934 learned_addr learned my addr 172.21.15.7:0/3479576934 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:59.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f6233fff700 1 --2- 172.21.15.7:0/3479576934 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f62340b37b0 0x7f6234140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f6238803700 1 --2- 172.21.15.7:0/3479576934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6234145e10 0x7f623406fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f62337fe700 1 -- 172.21.15.7:0/3479576934 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f62340b37b0 msgr2=0x7f6234140010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f62337fe700 1 --2- 172.21.15.7:0/3479576934 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f62340b37b0 0x7f6234140010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f62337fe700 1 -- 172.21.15.7:0/3479576934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6234145e10 msgr2=0x7f623406fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f62337fe700 1 --2- 172.21.15.7:0/3479576934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6234145e10 0x7f623406fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f62337fe700 1 -- 172.21.15.7:0/3479576934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c009570 con 0x7f62340cc250 2024-09-13T18:54:59.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f62337fe700 1 --2- 172.21.15.7:0/3479576934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62340cc250 0x7f623413dad0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f62340ad940 tx=0x7f621c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:59.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f6230ff9700 1 -- 172.21.15.7:0/3479576934 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c00bc80 con 0x7f62340cc250 2024-09-13T18:54:59.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f6230ff9700 1 -- 172.21.15.7:0/3479576934 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621c01f070 con 0x7f62340cc250 2024-09-13T18:54:59.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f6239004700 1 -- 172.21.15.7:0/3479576934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623413ef70 con 0x7f62340cc250 2024-09-13T18:54:59.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f6230ff9700 1 -- 172.21.15.7:0/3479576934 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c0036c0 con 0x7f62340cc250 2024-09-13T18:54:59.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f6239004700 1 -- 172.21.15.7:0/3479576934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623413f530 con 0x7f62340cc250 2024-09-13T18:54:59.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.470+0000 7f6239004700 1 -- 172.21.15.7:0/3479576934 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62340b3270 con 0x7f62340cc250 2024-09-13T18:54:59.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.474+0000 7f6230ff9700 1 -- 172.21.15.7:0/3479576934 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f621c021030 con 0x7f62340cc250 2024-09-13T18:54:59.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.474+0000 7f6230ff9700 1 --2- 172.21.15.7:0/3479576934 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6214066730 0x7f6214068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.474+0000 7f6230ff9700 1 -- 172.21.15.7:0/3479576934 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f621c0854c0 con 0x7f62340cc250 2024-09-13T18:54:59.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.474+0000 7f6233fff700 1 --2- 172.21.15.7:0/3479576934 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6214066730 0x7f6214068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.474+0000 7f6233fff700 1 --2- 172.21.15.7:0/3479576934 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6214066730 0x7f6214068bf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f622400d620 tx=0x7f6224007810 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:59.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.482+0000 7f6230ff9700 1 -- 172.21.15.7:0/3479576934 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f621c052b60 con 0x7f62340cc250 2024-09-13T18:54:59.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.654+0000 7f6239004700 1 -- 172.21.15.7:0/3479576934 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f623415d800 con 0x7f6214066730 2024-09-13T18:54:59.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.654+0000 7f6230ff9700 1 -- 172.21.15.7:0/3479576934 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6224007d70 con 0x7f6214066730 2024-09-13T18:54:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.654+0000 7f6239004700 1 -- 172.21.15.7:0/3479576934 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6214066730 msgr2=0x7f6214068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.654+0000 7f6239004700 1 --2- 172.21.15.7:0/3479576934 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6214066730 0x7f6214068bf0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f622400d620 tx=0x7f6224007810 comp rx=0 tx=0).stop 2024-09-13T18:54:59.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.654+0000 7f6239004700 1 -- 172.21.15.7:0/3479576934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62340cc250 msgr2=0x7f623413dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.654+0000 7f6239004700 1 --2- 172.21.15.7:0/3479576934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62340cc250 0x7f623413dad0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f62340ad940 tx=0x7f621c004770 comp rx=0 tx=0).stop 2024-09-13T18:54:59.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.658+0000 7f6239004700 1 -- 172.21.15.7:0/3479576934 shutdown_connections 2024-09-13T18:54:59.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.658+0000 7f6239004700 1 --2- 172.21.15.7:0/3479576934 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f6214066730 0x7f6214068bf0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.658+0000 7f6239004700 1 --2- 172.21.15.7:0/3479576934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6234145e10 0x7f623406fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.658+0000 7f6239004700 1 --2- 172.21.15.7:0/3479576934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62340cc250 0x7f623413dad0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.658+0000 7f6239004700 1 --2- 172.21.15.7:0/3479576934 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f62340b37b0 0x7f6234140010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.658+0000 7f6239004700 1 -- 172.21.15.7:0/3479576934 >> 172.21.15.7:0/3479576934 conn(0x7f62340b0e10 msgr2=0x7f62340ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:59.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.658+0000 7f6239004700 1 -- 172.21.15.7:0/3479576934 shutdown_connections 2024-09-13T18:54:59.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.658+0000 7f6239004700 1 -- 172.21.15.7:0/3479576934 wait complete. 2024-09-13T18:54:59.665 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 5 to deep-scrub 2024-09-13T18:54:59.675 INFO:tasks.ceph:Scrubbing osd.6 2024-09-13T18:54:59.675 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:54:59.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.786+0000 7f3187fff700 1 Processor -- start 2024-09-13T18:54:59.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.786+0000 7f3187fff700 1 -- start start 2024-09-13T18:54:59.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.786+0000 7f3187fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f31880ceaf0 0x7f31880b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.786+0000 7f3187fff700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f3188160d90 0x7f31881611c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.786+0000 7f3187fff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f318806aa30 con 0x7f31880ce710 2024-09-13T18:54:59.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.786+0000 7f3187fff700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31880d0660 con 0x7f3188160d90 2024-09-13T18:54:59.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.786+0000 7f3187fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31880b1e60 con 0x7f31880ceaf0 2024-09-13T18:54:59.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.786+0000 7f31867fc700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f31880ceaf0 0x7f31880b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.786+0000 7f31877fe700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f3188160d90 0x7f31881611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.786+0000 7f3186ffd700 1 --1- >> v1:172.21.15.171:6789/0 conn(0x7f31880ce710 0x7f31880cff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:53458/0 (socket says 172.21.15.7:53458) 2024-09-13T18:54:59.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.786+0000 7f31867fc700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f31880ceaf0 0x7f31880b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:41572/0 (socket says 172.21.15.7:41572) 2024-09-13T18:54:59.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.786+0000 7f31877fe700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f3188160d90 0x7f31881611c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:53850/0 (socket says 172.21.15.7:53850) 2024-09-13T18:54:59.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.786+0000 7f3186ffd700 1 -- 172.21.15.7:0/1860345102 learned_addr learned my addr 172.21.15.7:0/1860345102 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:59.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.790+0000 7f31867fc700 1 -- 172.21.15.7:0/1860345102 >> v2:172.21.15.171:3300/0 conn(0x7f3188160d90 msgr2=0x7f31881611c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:54:59.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.790+0000 7f31867fc700 1 --2- 172.21.15.7:0/1860345102 >> v2:172.21.15.171:3300/0 conn(0x7f3188160d90 0x7f31881611c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.790+0000 7f31867fc700 1 -- 172.21.15.7:0/1860345102 >> v1:172.21.15.171:6789/0 conn(0x7f31880ce710 legacy=0x7f31880cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.790+0000 7f31867fc700 1 -- 172.21.15.7:0/1860345102 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31880b18a0 con 0x7f31880ceaf0 2024-09-13T18:54:59.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.790+0000 7f31867fc700 1 --2- 172.21.15.7:0/1860345102 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f31880ceaf0 0x7f31880b5da0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3178009850 tx=0x7f3178009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=431ebacda07a7359 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:59.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.790+0000 7f3185ffb700 1 -- 172.21.15.7:0/1860345102 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178011830 con 0x7f31880ceaf0 2024-09-13T18:54:59.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.790+0000 7f3185ffb700 1 -- 172.21.15.7:0/1860345102 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3178011990 con 0x7f31880ceaf0 2024-09-13T18:54:59.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.790+0000 7f3185ffb700 1 -- 172.21.15.7:0/1860345102 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178011d40 con 0x7f31880ceaf0 2024-09-13T18:54:59.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.790+0000 7f3187fff700 1 -- 172.21.15.7:0/1860345102 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f31880ceaf0 msgr2=0x7f31880b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.790+0000 7f3187fff700 1 --2- 172.21.15.7:0/1860345102 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f31880ceaf0 0x7f31880b5da0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3178009850 tx=0x7f3178009b60 comp rx=0 tx=0).stop 2024-09-13T18:54:59.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.790+0000 7f3187fff700 1 -- 172.21.15.7:0/1860345102 shutdown_connections 2024-09-13T18:54:59.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.790+0000 7f3187fff700 1 --2- 172.21.15.7:0/1860345102 >> v2:172.21.15.171:3300/0 conn(0x7f3188160d90 0x7f31881611c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.790+0000 7f3187fff700 1 --2- 172.21.15.7:0/1860345102 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f31880ceaf0 0x7f31880b5da0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.790+0000 7f3187fff700 1 -- 172.21.15.7:0/1860345102 >> 172.21.15.7:0/1860345102 conn(0x7f31880d3260 msgr2=0x7f31880cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:59.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.790+0000 7f3187fff700 1 -- 172.21.15.7:0/1860345102 shutdown_connections 2024-09-13T18:54:59.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.790+0000 7f3187fff700 1 -- 172.21.15.7:0/1860345102 wait complete. 2024-09-13T18:54:59.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f3187fff700 1 Processor -- start 2024-09-13T18:54:59.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f3187fff700 1 -- start start 2024-09-13T18:54:59.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f3187fff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f31880ce710 0x7f31881e6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f3187fff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31880ceaf0 0x7f31880cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f3187fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3188160d90 0x7f31881537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f3187fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31881e6fb0 con 0x7f3188160d90 2024-09-13T18:54:59.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f3187fff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31881e7290 con 0x7f31880ceaf0 2024-09-13T18:54:59.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f3187fff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31881e7570 con 0x7f31880ce710 2024-09-13T18:54:59.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f31877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3188160d90 0x7f31881537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f3186ffd700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f31880ce710 0x7f31881e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f31867fc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31880ceaf0 0x7f31880cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f31867fc700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31880ceaf0 0x7f31880cab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:53856/0 (socket says 172.21.15.7:53856) 2024-09-13T18:54:59.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f3186ffd700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f31880ce710 0x7f31881e6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3301/0 says I am v2:172.21.15.7:41582/0 (socket says 172.21.15.7:41582) 2024-09-13T18:54:59.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f3186ffd700 1 -- 172.21.15.7:0/934277011 learned_addr learned my addr 172.21.15.7:0/934277011 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:59.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f31877fe700 1 --2- 172.21.15.7:0/934277011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3188160d90 0x7f31881537d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41026/0 (socket says 172.21.15.7:41026) 2024-09-13T18:54:59.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f31867fc700 1 -- 172.21.15.7:0/934277011 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f31880ce710 msgr2=0x7f31881e6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f31867fc700 1 --2- 172.21.15.7:0/934277011 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f31880ce710 0x7f31881e6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f31867fc700 1 -- 172.21.15.7:0/934277011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3188160d90 msgr2=0x7f31881537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f31867fc700 1 --2- 172.21.15.7:0/934277011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3188160d90 0x7f31881537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f31867fc700 1 -- 172.21.15.7:0/934277011 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31780094b0 con 0x7f31880ceaf0 2024-09-13T18:54:59.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f31877fe700 1 --2- 172.21.15.7:0/934277011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3188160d90 0x7f31881537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:59.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f3186ffd700 1 --2- 172.21.15.7:0/934277011 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f31880ce710 0x7f31881e6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:54:59.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f31867fc700 1 --2- 172.21.15.7:0/934277011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31880ceaf0 0x7f31880cab90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3178016000 tx=0x7f3178004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:59.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f316f7fe700 1 -- 172.21.15.7:0/934277011 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31780036c0 con 0x7f31880ceaf0 2024-09-13T18:54:59.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f316f7fe700 1 -- 172.21.15.7:0/934277011 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3178003840 con 0x7f31880ceaf0 2024-09-13T18:54:59.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f3187fff700 1 -- 172.21.15.7:0/934277011 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31881e7850 con 0x7f31880ceaf0 2024-09-13T18:54:59.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f316f7fe700 1 -- 172.21.15.7:0/934277011 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178003bf0 con 0x7f31880ceaf0 2024-09-13T18:54:59.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.794+0000 7f3187fff700 1 -- 172.21.15.7:0/934277011 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31881e6cc0 con 0x7f31880ceaf0 2024-09-13T18:54:59.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.798+0000 7f3187fff700 1 -- 172.21.15.7:0/934277011 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f318806a6c0 con 0x7f31880ceaf0 2024-09-13T18:54:59.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.798+0000 7f316f7fe700 1 -- 172.21.15.7:0/934277011 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f3178003d50 con 0x7f31880ceaf0 2024-09-13T18:54:59.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.798+0000 7f316f7fe700 1 --2- 172.21.15.7:0/934277011 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3174066730 0x7f3174068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.798+0000 7f316f7fe700 1 -- 172.21.15.7:0/934277011 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f3178021030 con 0x7f31880ceaf0 2024-09-13T18:54:59.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.798+0000 7f316f7fe700 1 --2- 172.21.15.7:0/934277011 >> [v2:172.21.15.171:6808/2277626206,v1:172.21.15.171:6809/2277626206] conn(0x7f317406c630 0x7f317406ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.798+0000 7f316f7fe700 1 -- 172.21.15.7:0/934277011 --> [v2:172.21.15.171:6808/2277626206,v1:172.21.15.171:6809/2277626206] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f317406f100 con 0x7f317406c630 2024-09-13T18:54:59.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.798+0000 7f316f7fe700 1 -- 172.21.15.7:0/934277011 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3178085b20 con 0x7f31880ceaf0 2024-09-13T18:54:59.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.802+0000 7f3186ffd700 1 --2- 172.21.15.7:0/934277011 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3174066730 0x7f3174068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.802+0000 7f31877fe700 1 --2- 172.21.15.7:0/934277011 >> [v2:172.21.15.171:6808/2277626206,v1:172.21.15.171:6809/2277626206] conn(0x7f317406c630 0x7f317406ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.802+0000 7f3186ffd700 1 --2- 172.21.15.7:0/934277011 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3174066730 0x7f3174068bf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f317c005930 tx=0x7f317c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:59.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.802+0000 7f31877fe700 1 --2- 172.21.15.7:0/934277011 >> [v2:172.21.15.171:6808/2277626206,v1:172.21.15.171:6809/2277626206] conn(0x7f317406c630 0x7f317406ea50 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:59.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.802+0000 7f316f7fe700 1 -- 172.21.15.7:0/934277011 <== osd.6 v2:172.21.15.171:6808/2277626206 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f317406f100 con 0x7f317406c630 2024-09-13T18:54:59.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.822+0000 7f3187fff700 1 -- 172.21.15.7:0/934277011 --> [v2:172.21.15.171:6808/2277626206,v1:172.21.15.171:6809/2277626206] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f31881e7d70 con 0x7f317406c630 2024-09-13T18:54:59.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.822+0000 7f316f7fe700 1 -- 172.21.15.7:0/934277011 <== osd.6 v2:172.21.15.171:6808/2277626206 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f31881e7d70 con 0x7f317406c630 2024-09-13T18:54:59.829 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:54:59.829 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:54:59.829 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:54:59.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.822+0000 7f3187fff700 1 -- 172.21.15.7:0/934277011 >> [v2:172.21.15.171:6808/2277626206,v1:172.21.15.171:6809/2277626206] conn(0x7f317406c630 msgr2=0x7f317406ea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.822+0000 7f3187fff700 1 --2- 172.21.15.7:0/934277011 >> [v2:172.21.15.171:6808/2277626206,v1:172.21.15.171:6809/2277626206] conn(0x7f317406c630 0x7f317406ea50 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.822+0000 7f3187fff700 1 -- 172.21.15.7:0/934277011 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3174066730 msgr2=0x7f3174068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.822+0000 7f3187fff700 1 --2- 172.21.15.7:0/934277011 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f3174066730 0x7f3174068bf0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f317c005930 tx=0x7f317c009040 comp rx=0 tx=0).stop 2024-09-13T18:54:59.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.822+0000 7f3187fff700 1 -- 172.21.15.7:0/934277011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31880ceaf0 msgr2=0x7f31880cab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.822+0000 7f3187fff700 1 --2- 172.21.15.7:0/934277011 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31880ceaf0 0x7f31880cab90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3178016000 tx=0x7f3178004770 comp rx=0 tx=0).stop 2024-09-13T18:54:59.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.822+0000 7f31877fe700 1 -- 172.21.15.7:0/934277011 reap_dead start 2024-09-13T18:54:59.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.822+0000 7f3187fff700 1 -- 172.21.15.7:0/934277011 shutdown_connections 2024-09-13T18:54:59.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.822+0000 7f3187fff700 1 -- 172.21.15.7:0/934277011 >> 172.21.15.7:0/934277011 conn(0x7f31880d3260 msgr2=0x7f31880c9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:59.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.822+0000 7f3187fff700 1 -- 172.21.15.7:0/934277011 shutdown_connections 2024-09-13T18:54:59.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.822+0000 7f3187fff700 1 -- 172.21.15.7:0/934277011 wait complete. 2024-09-13T18:54:59.843 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-13T18:54:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.954+0000 7f31ebfff700 1 Processor -- start 2024-09-13T18:54:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.954+0000 7f31ebfff700 1 -- start start 2024-09-13T18:54:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.954+0000 7f31ebfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f31ec0b3980 0x7f31ec167740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.954+0000 7f31ebfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31ec1461f0 con 0x7f31ec0cc420 2024-09-13T18:54:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.954+0000 7f31ebfff700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31ec15d240 con 0x7f31ec145e10 2024-09-13T18:54:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.954+0000 7f31ebfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31ec1466f0 con 0x7f31ec0b3980 2024-09-13T18:54:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.954+0000 7f31eb7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f31ec0b3980 0x7f31ec167740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.954+0000 7f31ea7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f31ec0cc420 0x7f31ec0b3270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58098/0 (socket says 172.21.15.7:58098) 2024-09-13T18:54:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.954+0000 7f31ea7fc700 1 -- 172.21.15.7:0/3196681190 learned_addr learned my addr 172.21.15.7:0/3196681190 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:54:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.958+0000 7f31e9ffb700 1 -- 172.21.15.7:0/3196681190 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 809143270 0 0) 0x7f31ec1461f0 con 0x7f31ec0cc420 2024-09-13T18:54:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.958+0000 7f31e9ffb700 1 -- 172.21.15.7:0/3196681190 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31d80035d0 con 0x7f31ec0cc420 2024-09-13T18:54:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.958+0000 7f31eb7fe700 1 -- 172.21.15.7:0/3196681190 >> v1:172.21.15.171:6789/0 conn(0x7f31ec145e10 legacy=0x7f31ec0cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.958+0000 7f31eb7fe700 1 -- 172.21.15.7:0/3196681190 >> v1:172.21.15.7:6789/0 conn(0x7f31ec0cc420 legacy=0x7f31ec0b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.958+0000 7f31eb7fe700 1 -- 172.21.15.7:0/3196681190 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec15d520 con 0x7f31ec0b3980 2024-09-13T18:54:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.958+0000 7f31eb7fe700 1 --2- 172.21.15.7:0/3196681190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31ec0b3980 0x7f31ec167740 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f31e4009850 tx=0x7f31e4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84591a93da9563d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.958+0000 7f31e9ffb700 1 -- 172.21.15.7:0/3196681190 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e4011830 con 0x7f31ec0b3980 2024-09-13T18:54:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.958+0000 7f31e9ffb700 1 -- 172.21.15.7:0/3196681190 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e4011990 con 0x7f31ec0b3980 2024-09-13T18:54:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.958+0000 7f31e9ffb700 1 -- 172.21.15.7:0/3196681190 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e4011d40 con 0x7f31ec0b3980 2024-09-13T18:54:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.958+0000 7f31ebfff700 1 -- 172.21.15.7:0/3196681190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31ec0b3980 msgr2=0x7f31ec167740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.958+0000 7f31ebfff700 1 --2- 172.21.15.7:0/3196681190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31ec0b3980 0x7f31ec167740 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f31e4009850 tx=0x7f31e4009b60 comp rx=0 tx=0).stop 2024-09-13T18:54:59.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.958+0000 7f31ebfff700 1 -- 172.21.15.7:0/3196681190 shutdown_connections 2024-09-13T18:54:59.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.958+0000 7f31ebfff700 1 --2- 172.21.15.7:0/3196681190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31ec0b3980 0x7f31ec167740 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.958+0000 7f31ebfff700 1 -- 172.21.15.7:0/3196681190 >> 172.21.15.7:0/3196681190 conn(0x7f31ec0b0e10 msgr2=0x7f31ec0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:54:59.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.958+0000 7f31ebfff700 1 -- 172.21.15.7:0/3196681190 shutdown_connections 2024-09-13T18:54:59.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.958+0000 7f31ebfff700 1 -- 172.21.15.7:0/3196681190 wait complete. 2024-09-13T18:54:59.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.958+0000 7f31ebfff700 1 Processor -- start 2024-09-13T18:54:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.958+0000 7f31ebfff700 1 -- start start 2024-09-13T18:54:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31ec0b3980 0x7f31ec15f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31ebfff700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31ec0cc420 0x7f31ec143d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31ebfff700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f31ec145e10 0x7f31ec0c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31ebfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec0c4630 con 0x7f31ec0b3980 2024-09-13T18:54:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31ebfff700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec0c4840 con 0x7f31ec0cc420 2024-09-13T18:54:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31ebfff700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec1f4110 con 0x7f31ec145e10 2024-09-13T18:54:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31eaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31ec0b3980 0x7f31ec15f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31eaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31ec0b3980 0x7f31ec15f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41042/0 (socket says 172.21.15.7:41042) 2024-09-13T18:54:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31eaffd700 1 -- 172.21.15.7:0/505764568 learned_addr learned my addr 172.21.15.7:0/505764568 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:54:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31ea7fc700 1 --2- 172.21.15.7:0/505764568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31ec0cc420 0x7f31ec143d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31eb7fe700 1 --2- 172.21.15.7:0/505764568 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f31ec145e10 0x7f31ec0c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31eaffd700 1 -- 172.21.15.7:0/505764568 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f31ec145e10 msgr2=0x7f31ec0c40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31eaffd700 1 --2- 172.21.15.7:0/505764568 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f31ec145e10 0x7f31ec0c40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31eaffd700 1 -- 172.21.15.7:0/505764568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31ec0cc420 msgr2=0x7f31ec143d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:54:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31eaffd700 1 --2- 172.21.15.7:0/505764568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31ec0cc420 0x7f31ec143d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:54:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31eaffd700 1 -- 172.21.15.7:0/505764568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e40094b0 con 0x7f31ec0b3980 2024-09-13T18:54:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31eaffd700 1 --2- 172.21.15.7:0/505764568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31ec0b3980 0x7f31ec15f7e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f31d400c500 tx=0x7f31d400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31e3fff700 1 -- 172.21.15.7:0/505764568 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d40052d0 con 0x7f31ec0b3980 2024-09-13T18:54:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31e3fff700 1 -- 172.21.15.7:0/505764568 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d4005430 con 0x7f31ec0b3980 2024-09-13T18:54:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31ebfff700 1 -- 172.21.15.7:0/505764568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31ec1f43f0 con 0x7f31ec0b3980 2024-09-13T18:54:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31e3fff700 1 -- 172.21.15.7:0/505764568 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d40055a0 con 0x7f31ec0b3980 2024-09-13T18:54:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31ebfff700 1 -- 172.21.15.7:0/505764568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31ec1f49b0 con 0x7f31ec0b3980 2024-09-13T18:54:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31ebfff700 1 -- 172.21.15.7:0/505764568 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31ec0ccbe0 con 0x7f31ec0b3980 2024-09-13T18:54:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.962+0000 7f31e3fff700 1 -- 172.21.15.7:0/505764568 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f31d40038e0 con 0x7f31ec0b3980 2024-09-13T18:54:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.966+0000 7f31e3fff700 1 --2- 172.21.15.7:0/505764568 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f31cc066730 0x7f31cc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:54:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.966+0000 7f31e3fff700 1 -- 172.21.15.7:0/505764568 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f31d4083280 con 0x7f31ec0b3980 2024-09-13T18:54:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.966+0000 7f31ea7fc700 1 --2- 172.21.15.7:0/505764568 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f31cc066730 0x7f31cc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:54:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.966+0000 7f31ea7fc700 1 --2- 172.21.15.7:0/505764568 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f31cc066730 0x7f31cc068bf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f31ec0cd260 tx=0x7f31dc001a90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:54:59.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:54:59.974+0000 7f31e3fff700 1 -- 172.21.15.7:0/505764568 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f31d4050810 con 0x7f31ec0b3980 2024-09-13T18:55:00.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.142+0000 7f31ebfff700 1 -- 172.21.15.7:0/505764568 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f31ec1f4280 con 0x7f31cc066730 2024-09-13T18:55:00.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.146+0000 7f31e3fff700 1 -- 172.21.15.7:0/505764568 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f31ec1f4280 con 0x7f31cc066730 2024-09-13T18:55:00.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.146+0000 7f31ebfff700 1 -- 172.21.15.7:0/505764568 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f31cc066730 msgr2=0x7f31cc068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.146+0000 7f31ebfff700 1 --2- 172.21.15.7:0/505764568 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f31cc066730 0x7f31cc068bf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f31ec0cd260 tx=0x7f31dc001a90 comp rx=0 tx=0).stop 2024-09-13T18:55:00.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.146+0000 7f31ebfff700 1 -- 172.21.15.7:0/505764568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31ec0b3980 msgr2=0x7f31ec15f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.146+0000 7f31ebfff700 1 --2- 172.21.15.7:0/505764568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31ec0b3980 0x7f31ec15f7e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f31d400c500 tx=0x7f31d400c810 comp rx=0 tx=0).stop 2024-09-13T18:55:00.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.146+0000 7f31ebfff700 1 -- 172.21.15.7:0/505764568 shutdown_connections 2024-09-13T18:55:00.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.146+0000 7f31ebfff700 1 --2- 172.21.15.7:0/505764568 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f31cc066730 0x7f31cc068bf0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.146+0000 7f31ebfff700 1 --2- 172.21.15.7:0/505764568 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f31ec145e10 0x7f31ec0c40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.146+0000 7f31ebfff700 1 --2- 172.21.15.7:0/505764568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f31ec0cc420 0x7f31ec143d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.146+0000 7f31ebfff700 1 --2- 172.21.15.7:0/505764568 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31ec0b3980 0x7f31ec15f7e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.146+0000 7f31ebfff700 1 -- 172.21.15.7:0/505764568 >> 172.21.15.7:0/505764568 conn(0x7f31ec0b0e10 msgr2=0x7f31ec0cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:00.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.146+0000 7f31ebfff700 1 -- 172.21.15.7:0/505764568 shutdown_connections 2024-09-13T18:55:00.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.146+0000 7f31ebfff700 1 -- 172.21.15.7:0/505764568 wait complete. 2024-09-13T18:55:00.154 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 6 to deep-scrub 2024-09-13T18:55:00.165 INFO:tasks.ceph:Scrubbing osd.7 2024-09-13T18:55:00.165 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-13T18:55:00.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.282+0000 7fd443134700 1 Processor -- start 2024-09-13T18:55:00.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.282+0000 7fd443134700 1 -- start start 2024-09-13T18:55:00.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.282+0000 7fd443134700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd43c154180 0x7fd43c0b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.282+0000 7fd443134700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd43c15aa30 0x7fd43c0b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.282+0000 7fd443134700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43c0b4ef0 con 0x7fd43c0b4ae0 2024-09-13T18:55:00.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.282+0000 7fd443134700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43c0b3d70 con 0x7fd43c15aa30 2024-09-13T18:55:00.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.282+0000 7fd443134700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43c06ccc0 con 0x7fd43c154180 2024-09-13T18:55:00.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.282+0000 7fd43bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd43c15aa30 0x7fd43c0b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd440ed0700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7fd43c154180 0x7fd43c0b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd43bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd43c15aa30 0x7fd43c0b45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41056/0 (socket says 172.21.15.7:41056) 2024-09-13T18:55:00.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd43bfff700 1 -- 172.21.15.7:0/963766347 learned_addr learned my addr 172.21.15.7:0/963766347 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:55:00.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd4416d1700 1 --1- 172.21.15.7:0/963766347 >> v1:172.21.15.171:6789/0 conn(0x7fd43c0b4ae0 0x7fd43c15d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.171:6789/0 says I am v1:172.21.15.7:53474/0 (socket says 172.21.15.7:53474) 2024-09-13T18:55:00.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd43bfff700 1 -- 172.21.15.7:0/963766347 >> v2:172.21.15.171:3300/0 conn(0x7fd43c154180 msgr2=0x7fd43c0b2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:00.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd43bfff700 1 --2- 172.21.15.7:0/963766347 >> v2:172.21.15.171:3300/0 conn(0x7fd43c154180 0x7fd43c0b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd43bfff700 1 -- 172.21.15.7:0/963766347 >> v1:172.21.15.171:6789/0 conn(0x7fd43c0b4ae0 legacy=0x7fd43c15d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd43bfff700 1 -- 172.21.15.7:0/963766347 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c0b30f0 con 0x7fd43c15aa30 2024-09-13T18:55:00.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd43bfff700 1 --2- 172.21.15.7:0/963766347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c15aa30 0x7fd43c0b45a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd43c066d00 tx=0x7fd42400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3807bfc2295eb27f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:00.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd43b7fe700 1 -- 172.21.15.7:0/963766347 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42400dd30 con 0x7fd43c15aa30 2024-09-13T18:55:00.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd43b7fe700 1 -- 172.21.15.7:0/963766347 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42400de90 con 0x7fd43c15aa30 2024-09-13T18:55:00.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd43b7fe700 1 -- 172.21.15.7:0/963766347 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd424010690 con 0x7fd43c15aa30 2024-09-13T18:55:00.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd443134700 1 -- 172.21.15.7:0/963766347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c15aa30 msgr2=0x7fd43c0b45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd443134700 1 --2- 172.21.15.7:0/963766347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c15aa30 0x7fd43c0b45a0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd43c066d00 tx=0x7fd42400be90 comp rx=0 tx=0).stop 2024-09-13T18:55:00.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd443134700 1 -- 172.21.15.7:0/963766347 shutdown_connections 2024-09-13T18:55:00.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd443134700 1 --2- 172.21.15.7:0/963766347 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c15aa30 0x7fd43c0b45a0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd443134700 1 --2- 172.21.15.7:0/963766347 >> v2:172.21.15.171:3300/0 conn(0x7fd43c154180 0x7fd43c0b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd443134700 1 -- 172.21.15.7:0/963766347 >> 172.21.15.7:0/963766347 conn(0x7fd43c070d00 msgr2=0x7fd43c071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:00.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd443134700 1 -- 172.21.15.7:0/963766347 shutdown_connections 2024-09-13T18:55:00.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd443134700 1 -- 172.21.15.7:0/963766347 wait complete. 2024-09-13T18:55:00.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd443134700 1 Processor -- start 2024-09-13T18:55:00.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.286+0000 7fd443134700 1 -- start start 2024-09-13T18:55:00.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd443134700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c0b4ae0 0x7fd43c1edaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd443134700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd43c154180 0x7fd43c0cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd443134700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd43c15aa30 0x7fd43c0ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd443134700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c154560 con 0x7fd43c0b4ae0 2024-09-13T18:55:00.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd443134700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c155380 con 0x7fd43c154180 2024-09-13T18:55:00.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd443134700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c154770 con 0x7fd43c15aa30 2024-09-13T18:55:00.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd440ed0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c0b4ae0 0x7fd43c1edaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd440ed0700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c0b4ae0 0x7fd43c1edaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41058/0 (socket says 172.21.15.7:41058) 2024-09-13T18:55:00.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd440ed0700 1 -- 172.21.15.7:0/1877621597 learned_addr learned my addr 172.21.15.7:0/1877621597 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:55:00.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd4416d1700 1 --2- 172.21.15.7:0/1877621597 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd43c15aa30 0x7fd43c0ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd43bfff700 1 --2- 172.21.15.7:0/1877621597 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd43c154180 0x7fd43c0cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd440ed0700 1 -- 172.21.15.7:0/1877621597 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd43c15aa30 msgr2=0x7fd43c0ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd440ed0700 1 --2- 172.21.15.7:0/1877621597 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fd43c15aa30 0x7fd43c0ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd440ed0700 1 -- 172.21.15.7:0/1877621597 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd43c154180 msgr2=0x7fd43c0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd440ed0700 1 --2- 172.21.15.7:0/1877621597 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd43c154180 0x7fd43c0cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd440ed0700 1 -- 172.21.15.7:0/1877621597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42400b810 con 0x7fd43c0b4ae0 2024-09-13T18:55:00.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd440ed0700 1 --2- 172.21.15.7:0/1877621597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c0b4ae0 0x7fd43c1edaf0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00e9d0 tx=0x7fd42c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:00.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd4397fa700 1 -- 172.21.15.7:0/1877621597 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42c009720 con 0x7fd43c0b4ae0 2024-09-13T18:55:00.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd4397fa700 1 -- 172.21.15.7:0/1877621597 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42c00ee70 con 0x7fd43c0b4ae0 2024-09-13T18:55:00.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd443134700 1 -- 172.21.15.7:0/1877621597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd43c154a50 con 0x7fd43c0b4ae0 2024-09-13T18:55:00.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd4397fa700 1 -- 172.21.15.7:0/1877621597 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42c00ca50 con 0x7fd43c0b4ae0 2024-09-13T18:55:00.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd443134700 1 -- 172.21.15.7:0/1877621597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd43c155010 con 0x7fd43c0b4ae0 2024-09-13T18:55:00.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.290+0000 7fd443134700 1 -- 172.21.15.7:0/1877621597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd43c0b3990 con 0x7fd43c0b4ae0 2024-09-13T18:55:00.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.294+0000 7fd4397fa700 1 -- 172.21.15.7:0/1877621597 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fd42c00cbb0 con 0x7fd43c0b4ae0 2024-09-13T18:55:00.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.294+0000 7fd4397fa700 1 --2- 172.21.15.7:0/1877621597 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd41c066730 0x7fd41c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.294+0000 7fd4397fa700 1 -- 172.21.15.7:0/1877621597 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fd42c084350 con 0x7fd43c0b4ae0 2024-09-13T18:55:00.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.294+0000 7fd4397fa700 1 --2- 172.21.15.7:0/1877621597 >> [v2:172.21.15.171:6817/2886154603,v1:172.21.15.171:6819/2886154603] conn(0x7fd41c06c630 0x7fd41c06ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.294+0000 7fd4397fa700 1 -- 172.21.15.7:0/1877621597 --> [v2:172.21.15.171:6817/2886154603,v1:172.21.15.171:6819/2886154603] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd41c06f100 con 0x7fd41c06c630 2024-09-13T18:55:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.294+0000 7fd4397fa700 1 -- 172.21.15.7:0/1877621597 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd42c0846c0 con 0x7fd43c0b4ae0 2024-09-13T18:55:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.294+0000 7fd43bfff700 1 --2- 172.21.15.7:0/1877621597 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd41c066730 0x7fd41c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.294+0000 7fd4416d1700 1 --2- 172.21.15.7:0/1877621597 >> [v2:172.21.15.171:6817/2886154603,v1:172.21.15.171:6819/2886154603] conn(0x7fd41c06c630 0x7fd41c06ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.294+0000 7fd4416d1700 1 --2- 172.21.15.7:0/1877621597 >> [v2:172.21.15.171:6817/2886154603,v1:172.21.15.171:6819/2886154603] conn(0x7fd41c06c630 0x7fd41c06ea50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:00.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.294+0000 7fd43bfff700 1 --2- 172.21.15.7:0/1877621597 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd41c066730 0x7fd41c068bf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd424000c00 tx=0x7fd424013000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:00.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.298+0000 7fd4397fa700 1 -- 172.21.15.7:0/1877621597 <== osd.7 v2:172.21.15.171:6817/2886154603 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fd41c06f100 con 0x7fd41c06c630 2024-09-13T18:55:00.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.314+0000 7fd443134700 1 -- 172.21.15.7:0/1877621597 --> [v2:172.21.15.171:6817/2886154603,v1:172.21.15.171:6819/2886154603] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd43c0b36b0 con 0x7fd41c06c630 2024-09-13T18:55:00.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.314+0000 7fd4397fa700 1 -- 172.21.15.7:0/1877621597 <== osd.7 v2:172.21.15.171:6817/2886154603 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd434007240 con 0x7fd41c06c630 2024-09-13T18:55:00.320 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-13T18:55:00.320 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T18:55:00.320 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-13T18:55:00.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.314+0000 7fd443134700 1 -- 172.21.15.7:0/1877621597 >> [v2:172.21.15.171:6817/2886154603,v1:172.21.15.171:6819/2886154603] conn(0x7fd41c06c630 msgr2=0x7fd41c06ea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.314+0000 7fd443134700 1 --2- 172.21.15.7:0/1877621597 >> [v2:172.21.15.171:6817/2886154603,v1:172.21.15.171:6819/2886154603] conn(0x7fd41c06c630 0x7fd41c06ea50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.314+0000 7fd443134700 1 -- 172.21.15.7:0/1877621597 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd41c066730 msgr2=0x7fd41c068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.314+0000 7fd443134700 1 --2- 172.21.15.7:0/1877621597 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fd41c066730 0x7fd41c068bf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd424000c00 tx=0x7fd424013000 comp rx=0 tx=0).stop 2024-09-13T18:55:00.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.314+0000 7fd443134700 1 -- 172.21.15.7:0/1877621597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c0b4ae0 msgr2=0x7fd43c1edaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.314+0000 7fd443134700 1 --2- 172.21.15.7:0/1877621597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43c0b4ae0 0x7fd43c1edaf0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00e9d0 tx=0x7fd42c00ece0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.314+0000 7fd4416d1700 1 -- 172.21.15.7:0/1877621597 reap_dead start 2024-09-13T18:55:00.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.314+0000 7fd443134700 1 -- 172.21.15.7:0/1877621597 shutdown_connections 2024-09-13T18:55:00.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.314+0000 7fd443134700 1 -- 172.21.15.7:0/1877621597 >> 172.21.15.7:0/1877621597 conn(0x7fd43c070d00 msgr2=0x7fd43c0b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:00.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.314+0000 7fd443134700 1 -- 172.21.15.7:0/1877621597 shutdown_connections 2024-09-13T18:55:00.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.314+0000 7fd443134700 1 -- 172.21.15.7:0/1877621597 wait complete. 2024-09-13T18:55:00.334 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-13T18:55:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e3b441700 1 Processor -- start 2024-09-13T18:55:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e3b441700 1 -- start start 2024-09-13T18:55:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e3b441700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7f2e3406ed60 0x7f2e340cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e3b441700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2e340cfa10 0x7f2e34164a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e3b441700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e34165410 con 0x7f2e34164fc0 2024-09-13T18:55:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e3b441700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e340ce650 con 0x7f2e340cfa10 2024-09-13T18:55:00.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e3b441700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e340cdad0 con 0x7f2e3406ed60 2024-09-13T18:55:00.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e39c3e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2e340cfa10 0x7f2e34164a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e39c3e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2e340cfa10 0x7f2e34164a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41066/0 (socket says 172.21.15.7:41066) 2024-09-13T18:55:00.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e39c3e700 1 -- 172.21.15.7:0/1627517117 learned_addr learned my addr 172.21.15.7:0/1627517117 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:55:00.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e3a43f700 1 --2- 172.21.15.7:0/1627517117 >> v2:172.21.15.171:3300/0 conn(0x7f2e3406ed60 0x7f2e340cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e39c3e700 1 -- 172.21.15.7:0/1627517117 >> v2:172.21.15.171:3300/0 conn(0x7f2e3406ed60 msgr2=0x7f2e340cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e39c3e700 1 --2- 172.21.15.7:0/1627517117 >> v2:172.21.15.171:3300/0 conn(0x7f2e3406ed60 0x7f2e340cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e39c3e700 1 -- 172.21.15.7:0/1627517117 >> v1:172.21.15.171:6789/0 conn(0x7f2e34164fc0 legacy=0x7f2e34161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e39c3e700 1 -- 172.21.15.7:0/1627517117 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e340cddb0 con 0x7f2e340cfa10 2024-09-13T18:55:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e3a43f700 1 --2- 172.21.15.7:0/1627517117 >> v2:172.21.15.171:3300/0 conn(0x7f2e3406ed60 0x7f2e340cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T18:55:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e39c3e700 1 --2- 172.21.15.7:0/1627517117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e340cfa10 0x7f2e34164a80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2e34071870 tx=0x7f2e1c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62842cac1d4fe50 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e3943d700 1 -- 172.21.15.7:0/1627517117 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c011830 con 0x7f2e340cfa10 2024-09-13T18:55:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e3943d700 1 -- 172.21.15.7:0/1627517117 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c011990 con 0x7f2e340cfa10 2024-09-13T18:55:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.450+0000 7f2e3943d700 1 -- 172.21.15.7:0/1627517117 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c011d20 con 0x7f2e340cfa10 2024-09-13T18:55:00.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.454+0000 7f2e3b441700 1 -- 172.21.15.7:0/1627517117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e340cfa10 msgr2=0x7f2e34164a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.454+0000 7f2e3b441700 1 --2- 172.21.15.7:0/1627517117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e340cfa10 0x7f2e34164a80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2e34071870 tx=0x7f2e1c009bf0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.454+0000 7f2e3b441700 1 -- 172.21.15.7:0/1627517117 shutdown_connections 2024-09-13T18:55:00.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.454+0000 7f2e3b441700 1 --2- 172.21.15.7:0/1627517117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e340cfa10 0x7f2e34164a80 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.454+0000 7f2e3b441700 1 --2- 172.21.15.7:0/1627517117 >> v2:172.21.15.171:3300/0 conn(0x7f2e3406ed60 0x7f2e340cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.454+0000 7f2e3b441700 1 -- 172.21.15.7:0/1627517117 >> 172.21.15.7:0/1627517117 conn(0x7f2e34071d00 msgr2=0x7f2e340720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:00.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.454+0000 7f2e3b441700 1 -- 172.21.15.7:0/1627517117 shutdown_connections 2024-09-13T18:55:00.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.454+0000 7f2e3b441700 1 -- 172.21.15.7:0/1627517117 wait complete. 2024-09-13T18:55:00.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.454+0000 7f2e3b441700 1 Processor -- start 2024-09-13T18:55:00.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.454+0000 7f2e3b441700 1 -- start start 2024-09-13T18:55:00.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e3b441700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2e3406ed60 0x7f2e34145960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e3b441700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e340cfa10 0x7f2e34161490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e3b441700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e34164fc0 0x7f2e341619d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e3b441700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e341595e0 con 0x7f2e34164fc0 2024-09-13T18:55:00.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e3b441700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e34158fd0 con 0x7f2e340cfa10 2024-09-13T18:55:00.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e3b441700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3415d270 con 0x7f2e3406ed60 2024-09-13T18:55:00.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e39c3e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e340cfa10 0x7f2e34161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e3ac40700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e34164fc0 0x7f2e341619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e39c3e700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e340cfa10 0x7f2e34161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.7:53888/0 (socket says 172.21.15.7:53888) 2024-09-13T18:55:00.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e39c3e700 1 -- 172.21.15.7:0/3797328103 learned_addr learned my addr 172.21.15.7:0/3797328103 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:55:00.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e3a43f700 1 --2- 172.21.15.7:0/3797328103 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2e3406ed60 0x7f2e34145960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e39c3e700 1 -- 172.21.15.7:0/3797328103 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2e3406ed60 msgr2=0x7f2e34145960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e39c3e700 1 --2- 172.21.15.7:0/3797328103 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2e3406ed60 0x7f2e34145960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e39c3e700 1 -- 172.21.15.7:0/3797328103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e34164fc0 msgr2=0x7f2e341619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e39c3e700 1 --2- 172.21.15.7:0/3797328103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e34164fc0 0x7f2e341619d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e39c3e700 1 -- 172.21.15.7:0/3797328103 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c009a70 con 0x7f2e340cfa10 2024-09-13T18:55:00.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e39c3e700 1 --2- 172.21.15.7:0/3797328103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e340cfa10 0x7f2e34161490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c000c00 tx=0x7f2e1c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:00.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e32ffd700 1 -- 172.21.15.7:0/3797328103 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c004510 con 0x7f2e340cfa10 2024-09-13T18:55:00.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e32ffd700 1 -- 172.21.15.7:0/3797328103 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c01e070 con 0x7f2e340cfa10 2024-09-13T18:55:00.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e3b441700 1 -- 172.21.15.7:0/3797328103 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e1c009570 con 0x7f2e340cfa10 2024-09-13T18:55:00.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e32ffd700 1 -- 172.21.15.7:0/3797328103 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0038e0 con 0x7f2e340cfa10 2024-09-13T18:55:00.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.458+0000 7f2e3b441700 1 -- 172.21.15.7:0/3797328103 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3415d550 con 0x7f2e340cfa10 2024-09-13T18:55:00.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.462+0000 7f2e3b441700 1 -- 172.21.15.7:0/3797328103 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e34159750 con 0x7f2e340cfa10 2024-09-13T18:55:00.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.462+0000 7f2e32ffd700 1 -- 172.21.15.7:0/3797328103 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f2e1c003a40 con 0x7f2e340cfa10 2024-09-13T18:55:00.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.462+0000 7f2e32ffd700 1 --2- 172.21.15.7:0/3797328103 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2e14066730 0x7f2e14068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.462+0000 7f2e32ffd700 1 -- 172.21.15.7:0/3797328103 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f2e1c084910 con 0x7f2e340cfa10 2024-09-13T18:55:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.462+0000 7f2e3a43f700 1 --2- 172.21.15.7:0/3797328103 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2e14066730 0x7f2e14068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.462+0000 7f2e3a43f700 1 --2- 172.21.15.7:0/3797328103 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2e14066730 0x7f2e14068bf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0114e0 tx=0x7f2e2c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:00.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.474+0000 7f2e32ffd700 1 -- 172.21.15.7:0/3797328103 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2e1c052030 con 0x7f2e340cfa10 2024-09-13T18:55:00.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.642+0000 7f2e3b441700 1 -- 172.21.15.7:0/3797328103 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e3406f140 con 0x7f2e14066730 2024-09-13T18:55:00.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.646+0000 7f2e32ffd700 1 -- 172.21.15.7:0/3797328103 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2e2c00f040 con 0x7f2e14066730 2024-09-13T18:55:00.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.646+0000 7f2e3b441700 1 -- 172.21.15.7:0/3797328103 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2e14066730 msgr2=0x7f2e14068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.646+0000 7f2e3b441700 1 --2- 172.21.15.7:0/3797328103 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2e14066730 0x7f2e14068bf0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0114e0 tx=0x7f2e2c00b040 comp rx=0 tx=0).stop 2024-09-13T18:55:00.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.646+0000 7f2e3b441700 1 -- 172.21.15.7:0/3797328103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e340cfa10 msgr2=0x7f2e34161490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.646+0000 7f2e3b441700 1 --2- 172.21.15.7:0/3797328103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e340cfa10 0x7f2e34161490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c000c00 tx=0x7f2e1c004770 comp rx=0 tx=0).stop 2024-09-13T18:55:00.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.646+0000 7f2e3b441700 1 -- 172.21.15.7:0/3797328103 shutdown_connections 2024-09-13T18:55:00.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.646+0000 7f2e3b441700 1 --2- 172.21.15.7:0/3797328103 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f2e14066730 0x7f2e14068bf0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.646+0000 7f2e3b441700 1 --2- 172.21.15.7:0/3797328103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e34164fc0 0x7f2e341619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.646+0000 7f2e3b441700 1 --2- 172.21.15.7:0/3797328103 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e340cfa10 0x7f2e34161490 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.646+0000 7f2e3b441700 1 --2- 172.21.15.7:0/3797328103 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f2e3406ed60 0x7f2e34145960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.646+0000 7f2e3b441700 1 -- 172.21.15.7:0/3797328103 >> 172.21.15.7:0/3797328103 conn(0x7f2e34071d00 msgr2=0x7f2e3415fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:00.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.646+0000 7f2e3b441700 1 -- 172.21.15.7:0/3797328103 shutdown_connections 2024-09-13T18:55:00.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.646+0000 7f2e3b441700 1 -- 172.21.15.7:0/3797328103 wait complete. 2024-09-13T18:55:00.655 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 7 to deep-scrub 2024-09-13T18:55:00.666 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:55:00.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.774+0000 7fc4710be700 1 Processor -- start 2024-09-13T18:55:00.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.774+0000 7fc4710be700 1 -- start start 2024-09-13T18:55:00.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.774+0000 7fc4710be700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc46c0b38d0 0x7fc46c15d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.774+0000 7fc4710be700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc46c06b5b0 0x7fc46c15b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.774+0000 7fc4710be700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc46c15a630 con 0x7fc46c159060 2024-09-13T18:55:00.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.774+0000 7fc4710be700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc46c15a450 con 0x7fc46c0b38d0 2024-09-13T18:55:00.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.774+0000 7fc4710be700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c06bd30 con 0x7fc46c06b5b0 2024-09-13T18:55:00.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.774+0000 7fc46b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc46c0b38d0 0x7fc46c15d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.774+0000 7fc46b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc46c0b38d0 0x7fc46c15d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41092/0 (socket says 172.21.15.7:41092) 2024-09-13T18:55:00.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.774+0000 7fc46b7fe700 1 -- 172.21.15.7:0/3637364190 learned_addr learned my addr 172.21.15.7:0/3637364190 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:55:00.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.778+0000 7fc4708bd700 1 --2- 172.21.15.7:0/3637364190 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc46c06b5b0 0x7fc46c15b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.778+0000 7fc46b7fe700 1 -- 172.21.15.7:0/3637364190 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc46c06b5b0 msgr2=0x7fc46c15b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:00.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.778+0000 7fc46b7fe700 1 --2- 172.21.15.7:0/3637364190 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc46c06b5b0 0x7fc46c15b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.778+0000 7fc46b7fe700 1 -- 172.21.15.7:0/3637364190 >> v1:172.21.15.171:6789/0 conn(0x7fc46c159060 legacy=0x7fc46c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.778+0000 7fc46b7fe700 1 -- 172.21.15.7:0/3637364190 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46c06c010 con 0x7fc46c0b38d0 2024-09-13T18:55:00.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.778+0000 7fc46b7fe700 1 --2- 172.21.15.7:0/3637364190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46c0b38d0 0x7fc46c15d5b0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc460006fd0 tx=0x7fc46000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132c62e6c1712118 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:00.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.778+0000 7fc46affd700 1 -- 172.21.15.7:0/3637364190 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46000dc40 con 0x7fc46c0b38d0 2024-09-13T18:55:00.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.778+0000 7fc46affd700 1 -- 172.21.15.7:0/3637364190 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46000dda0 con 0x7fc46c0b38d0 2024-09-13T18:55:00.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.778+0000 7fc46affd700 1 -- 172.21.15.7:0/3637364190 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc460004250 con 0x7fc46c0b38d0 2024-09-13T18:55:00.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.778+0000 7fc4710be700 1 -- 172.21.15.7:0/3637364190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46c0b38d0 msgr2=0x7fc46c15d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.778+0000 7fc4710be700 1 --2- 172.21.15.7:0/3637364190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46c0b38d0 0x7fc46c15d5b0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc460006fd0 tx=0x7fc46000bea0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.778+0000 7fc4710be700 1 -- 172.21.15.7:0/3637364190 shutdown_connections 2024-09-13T18:55:00.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.778+0000 7fc4710be700 1 --2- 172.21.15.7:0/3637364190 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc46c06b5b0 0x7fc46c15b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.778+0000 7fc4710be700 1 --2- 172.21.15.7:0/3637364190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46c0b38d0 0x7fc46c15d5b0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.778+0000 7fc4710be700 1 -- 172.21.15.7:0/3637364190 >> 172.21.15.7:0/3637364190 conn(0x7fc46c06d900 msgr2=0x7fc46c06dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:00.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.778+0000 7fc4710be700 1 -- 172.21.15.7:0/3637364190 shutdown_connections 2024-09-13T18:55:00.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.778+0000 7fc4710be700 1 -- 172.21.15.7:0/3637364190 wait complete. 2024-09-13T18:55:00.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4710be700 1 Processor -- start 2024-09-13T18:55:00.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4710be700 1 -- start start 2024-09-13T18:55:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4710be700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc46c06b5b0 0x7fc46c148df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4710be700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc46c0b38d0 0x7fc46c0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4710be700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46c159060 0x7fc46c0717d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4710be700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c0cda40 con 0x7fc46c159060 2024-09-13T18:55:00.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4710be700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c0cdd20 con 0x7fc46c0b38d0 2024-09-13T18:55:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4710be700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c14a490 con 0x7fc46c06b5b0 2024-09-13T18:55:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4708bd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46c159060 0x7fc46c0717d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4708bd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46c159060 0x7fc46c0717d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41100/0 (socket says 172.21.15.7:41100) 2024-09-13T18:55:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4708bd700 1 -- 172.21.15.7:0/1461971395 learned_addr learned my addr 172.21.15.7:0/1461971395 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:55:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc46bfff700 1 --2- 172.21.15.7:0/1461971395 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc46c06b5b0 0x7fc46c148df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc46b7fe700 1 --2- 172.21.15.7:0/1461971395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc46c0b38d0 0x7fc46c0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4708bd700 1 -- 172.21.15.7:0/1461971395 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc46c06b5b0 msgr2=0x7fc46c148df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4708bd700 1 --2- 172.21.15.7:0/1461971395 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc46c06b5b0 0x7fc46c148df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4708bd700 1 -- 172.21.15.7:0/1461971395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc46c0b38d0 msgr2=0x7fc46c0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4708bd700 1 --2- 172.21.15.7:0/1461971395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc46c0b38d0 0x7fc46c0cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4708bd700 1 -- 172.21.15.7:0/1461971395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46000b820 con 0x7fc46c159060 2024-09-13T18:55:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc46bfff700 1 --2- 172.21.15.7:0/1461971395 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc46c06b5b0 0x7fc46c148df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:55:00.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4708bd700 1 --2- 172.21.15.7:0/1461971395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46c159060 0x7fc46c0717d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc464007e40 tx=0x7fc46400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc468ff9700 1 -- 172.21.15.7:0/1461971395 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46400ede0 con 0x7fc46c159060 2024-09-13T18:55:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc468ff9700 1 -- 172.21.15.7:0/1461971395 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc464014420 con 0x7fc46c159060 2024-09-13T18:55:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4710be700 1 -- 172.21.15.7:0/1461971395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc46c14a770 con 0x7fc46c159060 2024-09-13T18:55:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc468ff9700 1 -- 172.21.15.7:0/1461971395 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4640147d0 con 0x7fc46c159060 2024-09-13T18:55:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.782+0000 7fc4710be700 1 -- 172.21.15.7:0/1461971395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46c0cd660 con 0x7fc46c159060 2024-09-13T18:55:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.786+0000 7fc4710be700 1 -- 172.21.15.7:0/1461971395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46c14ac90 con 0x7fc46c159060 2024-09-13T18:55:00.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.786+0000 7fc468ff9700 1 -- 172.21.15.7:0/1461971395 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fc464014930 con 0x7fc46c159060 2024-09-13T18:55:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.786+0000 7fc468ff9700 1 --2- 172.21.15.7:0/1461971395 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc458066730 0x7fc458068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.786+0000 7fc468ff9700 1 -- 172.21.15.7:0/1461971395 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fc464084630 con 0x7fc46c159060 2024-09-13T18:55:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.786+0000 7fc46bfff700 1 --2- 172.21.15.7:0/1461971395 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc458066730 0x7fc458068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:00.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.786+0000 7fc46bfff700 1 --2- 172.21.15.7:0/1461971395 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc458066730 0x7fc458068bf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc45c000f60 tx=0x7fc45c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:00.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.794+0000 7fc468ff9700 1 -- 172.21.15.7:0/1461971395 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc46c14ac90 con 0x7fc46c159060 2024-09-13T18:55:00.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.958+0000 7fc4710be700 1 -- 172.21.15.7:0/1461971395 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc46c06ba40 con 0x7fc458066730 2024-09-13T18:55:00.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.966+0000 7fc468ff9700 1 -- 172.21.15.7:0/1461971395 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49330 (secure 0 0 0) 0x7fc45c0050d0 con 0x7fc458066730 2024-09-13T18:55:00.973 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:55:00.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.966+0000 7fc4710be700 1 -- 172.21.15.7:0/1461971395 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc458066730 msgr2=0x7fc458068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.966+0000 7fc4710be700 1 --2- 172.21.15.7:0/1461971395 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc458066730 0x7fc458068bf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc45c000f60 tx=0x7fc45c009040 comp rx=0 tx=0).stop 2024-09-13T18:55:00.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.966+0000 7fc4710be700 1 -- 172.21.15.7:0/1461971395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46c159060 msgr2=0x7fc46c0717d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:00.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.966+0000 7fc4710be700 1 --2- 172.21.15.7:0/1461971395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46c159060 0x7fc46c0717d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fc464007e40 tx=0x7fc46400e3c0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.970+0000 7fc4710be700 1 -- 172.21.15.7:0/1461971395 shutdown_connections 2024-09-13T18:55:00.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.970+0000 7fc4710be700 1 --2- 172.21.15.7:0/1461971395 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fc458066730 0x7fc458068bf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.970+0000 7fc4710be700 1 --2- 172.21.15.7:0/1461971395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc46c159060 0x7fc46c0717d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.970+0000 7fc4710be700 1 --2- 172.21.15.7:0/1461971395 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc46c0b38d0 0x7fc46c0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.970+0000 7fc4710be700 1 --2- 172.21.15.7:0/1461971395 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fc46c06b5b0 0x7fc46c148df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:00.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.970+0000 7fc4710be700 1 -- 172.21.15.7:0/1461971395 >> 172.21.15.7:0/1461971395 conn(0x7fc46c06d900 msgr2=0x7fc46c0b3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:00.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.970+0000 7fc4710be700 1 -- 172.21.15.7:0/1461971395 shutdown_connections 2024-09-13T18:55:00.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:00.970+0000 7fc4710be700 1 -- 172.21.15.7:0/1461971395 wait complete. 2024-09-13T18:55:00.976 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-13T18:55:00.989 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-09-13T18:55:00.265587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":247420,"kb_used_data":6664,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754727300,"statfs":{"total":773094113280,"available":772840755200,"internally_reserved":0,"allocated":6823936,"data_stored":3537896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":246402617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1,"num_write_kb":-1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.419904"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":189,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T18:54:49.877544+0000","last_change":"2024-09-13T18:52:15.080080+0000","last_active":"2024-09-13T18:54:49.877544+0000","last_peered":"2024-09-13T18:54:49.877544+0000","last_clean":"2024-09-13T18:54:49.877544+0000","last_became_active":"2024-09-13T18:52:15.078100+0000","last_became_peered":"2024-09-13T18:52:15.078100+0000","last_unstale":"2024-09-13T18:54:49.877544+0000","last_undegraded":"2024-09-13T18:54:49.877544+0000","last_fullsized":"2024-09-13T18:54:49.877544+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T18:52:13.196473+0000","last_clean_scrub_stamp":"2024-09-13T18:52:13.196473+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:29:59.452757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542180,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28288,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343552,"statfs":{"total":96636764160,"available":96607797248,"internally_reserved":0,"allocated":1298432,"data_stored":885013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.85999999999999999,"15min":0.85499999999999998},"min":{"1min":0.58899999999999997,"5min":0.311,"15min":0.311},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.83199999999999996,"15min":0.84099999999999997},"min":{"1min":0.48299999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1,"5min":1.161,"15min":1.161},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.83299999999999996,"15min":0.82599999999999996},"min":{"1min":0.53400000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.49099999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.161,"5min":1.2,"15min":1.2},"last":0.76100000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.90300000000000002,"15min":0.91500000000000004},"min":{"1min":0.52500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.022,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.90600000000000003,"15min":0.91800000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1439999999999999,"5min":1.232,"15min":1.232},"last":0.65500000000000003}]},{"osd":3,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.90900000000000003,"15min":0.91200000000000003},"min":{"1min":0.65000000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94899999999999995,"15min":0.95699999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2589999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.69399999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.89600000000000002,"15min":0.89700000000000002},"min":{"1min":0.50800000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0309999999999999,"15min":1.0389999999999999},"min":{"1min":0.65000000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.97899999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":1.0469999999999999,"15min":1.069},"min":{"1min":0.60499999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2130000000000001,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.998,"5min":1.0269999999999999,"15min":1.032},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3260000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.93200000000000005}]},{"osd":6,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.095,"15min":1.103},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.379,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.127,"15min":1.139},"min":{"1min":0.76500000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.0840000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542180,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":294645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.91900000000000004,"15min":0.93100000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.171,"5min":1.706,"15min":1.706},"last":1.163},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.90800000000000003,"15min":0.90200000000000002},"min":{"1min":0.56299999999999994,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2869999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.86199999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.94599999999999995,"15min":0.95799999999999996},"min":{"1min":0.60599999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.153,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.91700000000000004,"15min":0.89900000000000002},"min":{"1min":0.66400000000000003,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.371,"5min":1.643,"15min":1.643},"last":0.93000000000000005}]},{"osd":2,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.84999999999999998,"15min":0.84099999999999997},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.96099999999999997,"15min":0.96299999999999997},"min":{"1min":0.78300000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.196,"5min":1.579,"15min":1.579},"last":1.0369999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.92400000000000004,"15min":0.91800000000000004},"min":{"1min":0.73199999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.444,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.91700000000000004,"15min":0.90400000000000003},"min":{"1min":0.67700000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":0.63300000000000001}]},{"osd":4,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.94999999999999996,"15min":0.95799999999999996},"min":{"1min":0.68200000000000005,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.171,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.002,"15min":1.004},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4870000000000001,"5min":1.879,"15min":1.879},"last":1.1890000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.753,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.1020000000000001,"15min":1.117},"min":{"1min":0.75700000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.2589999999999999,"5min":2.093,"15min":2.093},"last":1.113}]},{"osd":7,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1799999999999999,"15min":1.1879999999999999},"min":{"1min":0.94499999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.5269999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.099,"15min":1.0960000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5840000000000001,"5min":1.843,"15min":1.843},"last":0.84099999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542181,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":294645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.91100000000000003,"15min":0.91500000000000004},"min":{"1min":0.58199999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.103,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.94599999999999995,"15min":0.94999999999999996},"min":{"1min":0.66600000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2529999999999999,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.91400000000000003}]},{"osd":1,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.89700000000000002,"15min":0.90500000000000003},"min":{"1min":0.59599999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.3440000000000001,"5min":2.145,"15min":2.145},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.999,"15min":1.0029999999999999},"min":{"1min":0.70399999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.2330000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.96099999999999997}]},{"osd":2,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.85499999999999998,"15min":0.84999999999999998},"min":{"1min":0.69399999999999995,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0760000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.056},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.93500000000000005,"15min":0.92700000000000005},"min":{"1min":0.65800000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.208,"5min":1.427,"15min":1.427},"last":1.024}]},{"osd":3,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.86599999999999999,"15min":0.872},"min":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0720000000000001,"5min":1.704,"15min":1.704},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.92500000000000004,"15min":0.91800000000000004},"min":{"1min":0.69299999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1479999999999999,"5min":1.335,"15min":1.335},"last":0.72099999999999997}]},{"osd":4,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.004,"15min":1.0269999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.198,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.165},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.032,"15min":1.044},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.3,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.0009999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0569999999999999,"15min":1.0649999999999999},"min":{"1min":0.69099999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.264,"5min":1.857,"15min":1.857},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.05,"5min":1.139,"15min":1.1539999999999999},"min":{"1min":0.78000000000000003,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.2749999999999999,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.087}]},{"osd":7,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0780000000000001,"15min":1.0920000000000001},"min":{"1min":0.70099999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.2529999999999999,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.236,"15min":1.254},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.325,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":1.143}]}]},{"osd":4,"up_from":14,"seq":60129542180,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31804,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":704512,"data_stored":294645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.76900000000000002,"15min":0.749},"min":{"1min":0.48999999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.81999999999999995,"15min":0.80900000000000005},"min":{"1min":0.66700000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.69799999999999995}]},{"osd":1,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.84299999999999997,"15min":0.83799999999999997},"min":{"1min":0.63,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.026},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.82699999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.121,"5min":1.242,"15min":1.242},"last":1.3320000000000001}]},{"osd":2,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.52400000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":1.3120000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.274,"5min":1.694,"15min":1.694},"last":1.1639999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.91800000000000004,"15min":0.91300000000000003},"min":{"1min":0.76100000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":1.282},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":1.099}]},{"osd":5,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.86899999999999999,"15min":0.86199999999999999},"min":{"1min":0.68400000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.006,"5min":0.96599999999999997,"15min":0.95899999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3400000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.137}]},{"osd":6,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.997,"15min":1.0089999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0740000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0209999999999999,"15min":1.0169999999999999},"min":{"1min":0.79300000000000004,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.1830000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.2529999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.036,"15min":1.0249999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.302,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.0960000000000001,"15min":1.095},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.198}]}]},{"osd":3,"up_from":14,"seq":60129542180,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31808,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":704512,"data_stored":294645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65200000000000002,"15min":0.65100000000000002},"min":{"1min":0.51600000000000001,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.67600000000000005,"15min":0.68000000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93999999999999995,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.64700000000000002}]},{"osd":1,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.66500000000000004,"15min":0.65500000000000003},"min":{"1min":0.47699999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.0169999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.498},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.78000000000000003,"15min":0.78200000000000003},"min":{"1min":0.60699999999999998,"5min":0.433,"15min":0.433},"max":{"1min":1.147,"5min":1.181,"15min":1.181},"last":0.61599999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.82799999999999996,"15min":0.84799999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.98299999999999998,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.82299999999999995,"15min":0.81899999999999995},"min":{"1min":0.59799999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.167,"5min":1.571,"15min":1.571},"last":0.67300000000000004}]},{"osd":4,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":1.0189999999999999,"15min":1.0309999999999999},"min":{"1min":0.78500000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.27,"5min":1.837,"15min":1.837},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.3680000000000001,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.048}]},{"osd":5,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.0620000000000001,"15min":1.0680000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.3700000000000001,"5min":1.893,"15min":1.893},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.083,"5min":1.115,"15min":1.1200000000000001},"min":{"1min":0.86199999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.325,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.995}]},{"osd":6,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.1180000000000001,"15min":1.1160000000000001},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.3959999999999999,"5min":1.421,"15min":1.421},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.04,"5min":1.163,"15min":1.1830000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.2130000000000001,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.109}]},{"osd":7,"last update":"Fri Sep 13 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.2529999999999999,"15min":1.278},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.3029999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.079},{"interface":"front","average":{"1min":1.177,"5min":1.159,"15min":1.1559999999999999},"min":{"1min":0.89800000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.635,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.746}]}]},{"osd":2,"up_from":14,"seq":60129542180,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35900,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335940,"statfs":{"total":96636764160,"available":96600002560,"internally_reserved":0,"allocated":704512,"data_stored":294645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0609999999999999,"5min":1.073,"15min":1.073},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.83299999999999996,"15min":0.85099999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.091,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.48399999999999999}]},{"osd":1,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.78800000000000003},"min":{"1min":0.55500000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.98299999999999998,"5min":1.359,"15min":1.359},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.80700000000000005,"15min":0.81799999999999995},"min":{"1min":0.46500000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.042,"5min":1.696,"15min":1.696},"last":0.55200000000000005}]},{"osd":3,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.66200000000000003,"15min":0.65200000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89800000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.91500000000000004,"15min":0.91700000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0900000000000001,"5min":1.288,"15min":1.288},"last":0.58899999999999997}]},{"osd":4,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0680000000000001,"15min":1.0780000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.1850000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.014,"5min":1.0640000000000001,"15min":1.0720000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.238,"5min":1.879,"15min":1.879},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.0900000000000001,"15min":1.1040000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.387,"5min":1.796,"15min":1.796},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.038,"15min":1.0429999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2729999999999999,"5min":1.516,"15min":1.516},"last":0.67300000000000004}]},{"osd":6,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0800000000000001,"15min":1.0920000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.286,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.747},{"interface":"front","average":{"1min":1.119,"5min":1.1719999999999999,"15min":1.181},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.71999999999999997}]},{"osd":7,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.171,"15min":1.1910000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.3360000000000001,"5min":1.579,"15min":1.579},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.123,"5min":1.143,"15min":1.1459999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.3580000000000001,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.70299999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542180,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31808,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":704512,"data_stored":294645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.51900000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86899999999999999,"5min":2.786,"15min":2.786},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.629,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0009999999999999,"5min":1.159,"15min":1.159},"last":0.60399999999999998}]},{"osd":2,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.78100000000000003,"15min":0.79700000000000004},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2030000000000001,"5min":3.113,"15min":3.113},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.232,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.63500000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84599999999999997,"15min":0.84799999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.117,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81999999999999995,"15min":0.81899999999999995},"min":{"1min":0.626,"5min":0.502,"15min":0.502},"max":{"1min":1.1619999999999999,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.67600000000000005}]},{"osd":4,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.99199999999999999,"15min":1.004},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2070000000000001,"5min":2.387,"15min":2.387},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0589999999999999,"15min":1.0620000000000001},"min":{"1min":0.83799999999999997,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.51,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.83799999999999997}]},{"osd":5,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0469999999999999,"15min":1.0529999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4139999999999999,"5min":1.419,"15min":1.419},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0680000000000001,"15min":1.0860000000000001},"min":{"1min":0.74099999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2789999999999999,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":0.78800000000000003}]},{"osd":6,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.135,"15min":1.139},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.6020000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.141,"15min":1.151},"min":{"1min":0.82899999999999996,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.5760000000000001,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.0329999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.2030000000000001,"15min":1.2150000000000001},"min":{"1min":0.89700000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.659,"5min":1.825,"15min":1.825},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.111,"5min":1.1599999999999999,"15min":1.1679999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.7350000000000001,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.878}]}]},{"osd":0,"up_from":14,"seq":60129542181,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1298432,"data_stored":885013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.95099999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.62},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68899999999999995,"15min":0.68799999999999994},"min":{"1min":0.438,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.74299999999999999}]},{"osd":2,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.83599999999999997,"15min":0.82399999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":1.2529999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.75900000000000001,"15min":0.74299999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.5859999999999999,"5min":2.5859999999999999,"15min":2.5859999999999999},"last":1.077},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.79300000000000004,"15min":0.76900000000000002},"min":{"1min":0.42799999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.82899999999999996}]},{"osd":4,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.018,"15min":1.0169999999999999},"min":{"1min":0.68100000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.018,"15min":1.03},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.3009999999999999,"5min":1.579,"15min":1.579},"last":1.004}]},{"osd":5,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.97199999999999998,"15min":0.96599999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.073,"15min":1.075},"min":{"1min":0.68600000000000005,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.91800000000000004}]},{"osd":6,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.083,"15min":1.093},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3839999999999999,"5min":1.504,"15min":1.504},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.044,"15min":1.0369999999999999},"min":{"1min":0.79800000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.1100000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1759999999999999,"15min":1.1839999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.452,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0860000000000001,"15min":1.081},"min":{"1min":0.64800000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.226}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:55:00.993 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T18:52:13.196473+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=52, tm_sec=13, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=18, tm_min=54, tm_sec=55, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T18:55:00.993 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T18:55:20.995 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T18:55:21.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.113+0000 7f8408826700 1 Processor -- start 2024-09-13T18:55:21.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.113+0000 7f8408826700 1 -- start start 2024-09-13T18:55:21.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.113+0000 7f8408826700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f840406cf90 0x7f840406aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.113+0000 7f8408826700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840406d440 con 0x7f84040b3ae0 2024-09-13T18:55:21.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.113+0000 7f8408826700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840406d690 con 0x7f8404159720 2024-09-13T18:55:21.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.113+0000 7f8408826700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84040cbfe0 con 0x7f840406cf90 2024-09-13T18:55:21.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.113+0000 7f8403fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f840406cf90 0x7f840406aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.113+0000 7f8402ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f84040b3ae0 0x7f84040ccac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41642/0 (socket says 172.21.15.7:41642) 2024-09-13T18:55:21.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.113+0000 7f8402ffd700 1 -- 172.21.15.7:0/1523192011 learned_addr learned my addr 172.21.15.7:0/1523192011 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:55:21.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.117+0000 7f8403fff700 1 -- 172.21.15.7:0/1523192011 >> v1:172.21.15.171:6789/0 conn(0x7f8404159720 legacy=0x7f84040b33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.117+0000 7f8403fff700 1 -- 172.21.15.7:0/1523192011 >> v1:172.21.15.7:6789/0 conn(0x7f84040b3ae0 legacy=0x7f84040ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.117+0000 7f8403fff700 1 -- 172.21.15.7:0/1523192011 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84040cc2c0 con 0x7f840406cf90 2024-09-13T18:55:21.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.117+0000 7f8403fff700 1 --2- 172.21.15.7:0/1523192011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f840406cf90 0x7f840406aa90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009d90 tx=0x7f83fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8487c4f9c90cf01b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:21.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.117+0000 7f84027fc700 1 -- 172.21.15.7:0/1523192011 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc00e070 con 0x7f840406cf90 2024-09-13T18:55:21.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.117+0000 7f84027fc700 1 -- 172.21.15.7:0/1523192011 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83fc00bbd0 con 0x7f840406cf90 2024-09-13T18:55:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.117+0000 7f84027fc700 1 -- 172.21.15.7:0/1523192011 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc012430 con 0x7f840406cf90 2024-09-13T18:55:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.117+0000 7f8408826700 1 -- 172.21.15.7:0/1523192011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f840406cf90 msgr2=0x7f840406aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.117+0000 7f8408826700 1 --2- 172.21.15.7:0/1523192011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f840406cf90 0x7f840406aa90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009d90 tx=0x7f83fc00b330 comp rx=0 tx=0).stop 2024-09-13T18:55:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.117+0000 7f8408826700 1 -- 172.21.15.7:0/1523192011 shutdown_connections 2024-09-13T18:55:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.117+0000 7f8408826700 1 --2- 172.21.15.7:0/1523192011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f840406cf90 0x7f840406aa90 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.117+0000 7f8408826700 1 -- 172.21.15.7:0/1523192011 >> 172.21.15.7:0/1523192011 conn(0x7f84040b0f00 msgr2=0x7f84040b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.117+0000 7f8408826700 1 -- 172.21.15.7:0/1523192011 shutdown_connections 2024-09-13T18:55:21.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.117+0000 7f8408826700 1 -- 172.21.15.7:0/1523192011 wait complete. 2024-09-13T18:55:21.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.117+0000 7f8408826700 1 Processor -- start 2024-09-13T18:55:21.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.117+0000 7f8408826700 1 -- start start 2024-09-13T18:55:21.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8408826700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f840406cf90 0x7f840413e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8408826700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84040b3ae0 0x7f840414ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8408826700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8404159720 0x7f840414e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8408826700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840413e700 con 0x7f8404159720 2024-09-13T18:55:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8408826700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84041531b0 con 0x7f840406cf90 2024-09-13T18:55:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8408826700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840414f800 con 0x7f84040b3ae0 2024-09-13T18:55:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8403fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8404159720 0x7f840414e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8403fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8404159720 0x7f840414e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53080/0 (socket says 172.21.15.7:53080) 2024-09-13T18:55:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8403fff700 1 -- 172.21.15.7:0/1557666435 learned_addr learned my addr 172.21.15.7:0/1557666435 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:55:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8402ffd700 1 --2- 172.21.15.7:0/1557666435 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84040b3ae0 0x7f840414ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f84037fe700 1 --2- 172.21.15.7:0/1557666435 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f840406cf90 0x7f840413e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8403fff700 1 -- 172.21.15.7:0/1557666435 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84040b3ae0 msgr2=0x7f840414ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8403fff700 1 --2- 172.21.15.7:0/1557666435 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84040b3ae0 0x7f840414ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8403fff700 1 -- 172.21.15.7:0/1557666435 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f840406cf90 msgr2=0x7f840413e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8403fff700 1 --2- 172.21.15.7:0/1557666435 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f840406cf90 0x7f840413e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8403fff700 1 -- 172.21.15.7:0/1557666435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc009a70 con 0x7f8404159720 2024-09-13T18:55:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8403fff700 1 --2- 172.21.15.7:0/1557666435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8404159720 0x7f840414e410 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8404071e20 tx=0x7f83fc004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f83fbfff700 1 -- 172.21.15.7:0/1557666435 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc012d20 con 0x7f8404159720 2024-09-13T18:55:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f83fbfff700 1 -- 172.21.15.7:0/1557666435 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83fc020070 con 0x7f8404159720 2024-09-13T18:55:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8408826700 1 -- 172.21.15.7:0/1557666435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84041500a0 con 0x7f8404159720 2024-09-13T18:55:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f83fbfff700 1 -- 172.21.15.7:0/1557666435 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc012ea0 con 0x7f8404159720 2024-09-13T18:55:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8408826700 1 -- 172.21.15.7:0/1557666435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840414fa50 con 0x7f8404159720 2024-09-13T18:55:21.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.121+0000 7f8408826700 1 -- 172.21.15.7:0/1557666435 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8404150230 con 0x7f8404159720 2024-09-13T18:55:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.125+0000 7f83fbfff700 1 -- 172.21.15.7:0/1557666435 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7f83fc004280 con 0x7f8404159720 2024-09-13T18:55:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.125+0000 7f83fbfff700 1 --2- 172.21.15.7:0/1557666435 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f83e4066730 0x7f83e4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.125+0000 7f83fbfff700 1 -- 172.21.15.7:0/1557666435 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f83fc022070 con 0x7f8404159720 2024-09-13T18:55:21.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.125+0000 7f84037fe700 1 --2- 172.21.15.7:0/1557666435 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f83e4066730 0x7f83e4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.125+0000 7f84037fe700 1 --2- 172.21.15.7:0/1557666435 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f83e4066730 0x7f83e4068bf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f83f4000f60 tx=0x7f83f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:21.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.133+0000 7f83fbfff700 1 -- 172.21.15.7:0/1557666435 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f83fc08c080 con 0x7f8404159720 2024-09-13T18:55:21.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.301+0000 7f8408826700 1 -- 172.21.15.7:0/1557666435 --> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8404159b00 con 0x7f83e4066730 2024-09-13T18:55:21.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.305+0000 7f83fbfff700 1 -- 172.21.15.7:0/1557666435 <== mgr.4103 v2:172.21.15.171:6832/2829153556 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49365 (secure 0 0 0) 0x7f83f4001540 con 0x7f83e4066730 2024-09-13T18:55:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:55:21.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.309+0000 7f8408826700 1 -- 172.21.15.7:0/1557666435 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f83e4066730 msgr2=0x7f83e4068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.309+0000 7f8408826700 1 --2- 172.21.15.7:0/1557666435 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f83e4066730 0x7f83e4068bf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f83f4000f60 tx=0x7f83f4009040 comp rx=0 tx=0).stop 2024-09-13T18:55:21.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.309+0000 7f8408826700 1 -- 172.21.15.7:0/1557666435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8404159720 msgr2=0x7f840414e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.309+0000 7f8408826700 1 --2- 172.21.15.7:0/1557666435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8404159720 0x7f840414e410 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8404071e20 tx=0x7f83fc004af0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.309+0000 7f8408826700 1 -- 172.21.15.7:0/1557666435 shutdown_connections 2024-09-13T18:55:21.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.309+0000 7f8408826700 1 --2- 172.21.15.7:0/1557666435 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7f83e4066730 0x7f83e4068bf0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.309+0000 7f8408826700 1 --2- 172.21.15.7:0/1557666435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8404159720 0x7f840414e410 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.309+0000 7f8408826700 1 --2- 172.21.15.7:0/1557666435 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7f84040b3ae0 0x7f840414ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.309+0000 7f8408826700 1 --2- 172.21.15.7:0/1557666435 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f840406cf90 0x7f840413e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.309+0000 7f8408826700 1 -- 172.21.15.7:0/1557666435 >> 172.21.15.7:0/1557666435 conn(0x7f84040b0f00 msgr2=0x7f840415bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:21.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.309+0000 7f8408826700 1 -- 172.21.15.7:0/1557666435 shutdown_connections 2024-09-13T18:55:21.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.309+0000 7f8408826700 1 -- 172.21.15.7:0/1557666435 wait complete. 2024-09-13T18:55:21.318 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-13T18:55:21.330 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-13T18:55:20.270823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":247408,"kb_used_data":6664,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754727312,"statfs":{"total":773094113280,"available":772840767488,"internally_reserved":0,"allocated":6823936,"data_stored":3537896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":246402617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003273"},"pg_stats":[{"pgid":"1.0","version":"59'119","reported_seq":203,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T18:55:02.579429+0000","last_change":"2024-09-13T18:55:02.579730+0000","last_active":"2024-09-13T18:55:02.579429+0000","last_peered":"2024-09-13T18:55:02.579429+0000","last_clean":"2024-09-13T18:55:02.579429+0000","last_became_active":"2024-09-13T18:52:15.078100+0000","last_became_peered":"2024-09-13T18:52:15.078100+0000","last_unstale":"2024-09-13T18:55:02.579429+0000","last_undegraded":"2024-09-13T18:55:02.579429+0000","last_fullsized":"2024-09-13T18:55:02.579429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'119","last_scrub_stamp":"2024-09-13T18:55:02.579355+0000","last_deep_scrub":"59'119","last_deep_scrub_stamp":"2024-09-13T18:55:02.579355+0000","last_clean_scrub_stamp":"2024-09-13T18:55:02.579355+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:01.575867+0000","scrub_duration":0.010338428,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542184,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28284,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343556,"statfs":{"total":96636764160,"available":96607801344,"internally_reserved":0,"allocated":1298432,"data_stored":885013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.85999999999999999,"15min":0.85499999999999998},"min":{"1min":0.58899999999999997,"5min":0.311,"15min":0.311},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.83199999999999996,"15min":0.84099999999999997},"min":{"1min":0.48299999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1,"5min":1.161,"15min":1.161},"last":0.76200000000000001}]},{"osd":1,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.83299999999999996,"15min":0.82599999999999996},"min":{"1min":0.53400000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.49099999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.161,"5min":1.2,"15min":1.2},"last":1.105}]},{"osd":2,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.90300000000000002,"15min":0.91500000000000004},"min":{"1min":0.52500000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.022,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.90600000000000003,"15min":0.91800000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.1439999999999999,"5min":1.232,"15min":1.232},"last":1.1639999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.90900000000000003,"15min":0.91200000000000003},"min":{"1min":0.65000000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.002},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94899999999999995,"15min":0.95699999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2589999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.0329999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.89600000000000002,"15min":0.89700000000000002},"min":{"1min":0.50800000000000001,"5min":0.502,"15min":0.502},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0309999999999999,"15min":1.0389999999999999},"min":{"1min":0.65000000000000002,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.71799999999999997}]},{"osd":5,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":1.0469999999999999,"15min":1.069},"min":{"1min":0.60499999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2130000000000001,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.998,"5min":1.0269999999999999,"15min":1.032},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3260000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.82899999999999996}]},{"osd":6,"last update":"Fri Sep 13 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.095,"15min":1.103},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.379,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.127,"15min":1.139},"min":{"1min":0.76500000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.91600000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":294645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.91900000000000004,"15min":0.93100000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.171,"5min":1.706,"15min":1.706},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.90800000000000003,"15min":0.90200000000000002},"min":{"1min":0.56299999999999994,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2869999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.70499999999999996}]},{"osd":1,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.94599999999999995,"15min":0.95799999999999996},"min":{"1min":0.60599999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.153,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.129},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.91700000000000004,"15min":0.89900000000000002},"min":{"1min":0.66400000000000003,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.371,"5min":1.643,"15min":1.643},"last":0.83299999999999996}]},{"osd":2,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.84999999999999998,"15min":0.84099999999999997},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.96099999999999997,"15min":0.96299999999999997},"min":{"1min":0.78300000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.196,"5min":1.579,"15min":1.579},"last":0.93500000000000005}]},{"osd":3,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.92400000000000004,"15min":0.91800000000000004},"min":{"1min":0.73199999999999998,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.444,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.91700000000000004,"15min":0.90400000000000003},"min":{"1min":0.67700000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.45,"5min":1.45,"15min":1.45},"last":1.0609999999999999}]},{"osd":4,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.94999999999999996,"15min":0.95799999999999996},"min":{"1min":0.68200000000000005,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.171,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.002,"15min":1.004},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.4870000000000001,"5min":1.879,"15min":1.879},"last":1.101}]},{"osd":5,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.753,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.0129999999999999,"5min":1.1020000000000001,"15min":1.117},"min":{"1min":0.75700000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.2589999999999999,"5min":2.093,"15min":2.093},"last":1.0089999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1799999999999999,"15min":1.1879999999999999},"min":{"1min":0.94499999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.5269999999999999,"5min":3.0859999999999999,"15min":3.0859999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.099,"15min":1.0960000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5840000000000001,"5min":1.843,"15min":1.843},"last":0.90100000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27712,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":704512,"data_stored":294645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.91100000000000003,"15min":0.91500000000000004},"min":{"1min":0.58199999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.103,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.94599999999999995,"15min":0.94999999999999996},"min":{"1min":0.66600000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2529999999999999,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.93999999999999995}]},{"osd":1,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.89700000000000002,"15min":0.90500000000000003},"min":{"1min":0.59599999999999997,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.3440000000000001,"5min":2.145,"15min":2.145},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.999,"15min":1.0029999999999999},"min":{"1min":0.70399999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.2330000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.121}]},{"osd":2,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.85499999999999998,"15min":0.84999999999999998},"min":{"1min":0.69399999999999995,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.0760000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.93500000000000005,"15min":0.92700000000000005},"min":{"1min":0.65800000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.208,"5min":1.427,"15min":1.427},"last":0.96799999999999997}]},{"osd":3,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.86599999999999999,"15min":0.872},"min":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0720000000000001,"5min":1.704,"15min":1.704},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.92500000000000004,"15min":0.91800000000000004},"min":{"1min":0.69299999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1479999999999999,"5min":1.335,"15min":1.335},"last":1.222}]},{"osd":4,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":1.004,"15min":1.0269999999999999},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.198,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.147},{"interface":"front","average":{"1min":0.96299999999999997,"5min":1.032,"15min":1.044},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.3,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.76500000000000001}]},{"osd":6,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0569999999999999,"15min":1.0649999999999999},"min":{"1min":0.69099999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.264,"5min":1.857,"15min":1.857},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.139,"15min":1.1539999999999999},"min":{"1min":0.78000000000000003,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.2749999999999999,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.1990000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0780000000000001,"15min":1.0920000000000001},"min":{"1min":0.70099999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.2529999999999999,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.236,"15min":1.254},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.325,"5min":3.4369999999999998,"15min":3.4369999999999998},"last":1.2629999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31804,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":704512,"data_stored":294645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.76900000000000002,"15min":0.749},"min":{"1min":0.48999999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.81999999999999995,"15min":0.80900000000000005},"min":{"1min":0.66700000000000004,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.96299999999999997}]},{"osd":1,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.84299999999999997,"15min":0.83799999999999997},"min":{"1min":0.63,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.82699999999999996},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.121,"5min":1.242,"15min":1.242},"last":0.81299999999999994}]},{"osd":2,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78800000000000003,"15min":0.78400000000000003},"min":{"1min":0.52400000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":1.04},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.274,"5min":1.694,"15min":1.694},"last":0.73599999999999999}]},{"osd":3,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.91800000000000004,"15min":0.91300000000000003},"min":{"1min":0.76100000000000001,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.88700000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.86899999999999999,"15min":0.86199999999999999},"min":{"1min":0.68400000000000005,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.006,"5min":0.96599999999999997,"15min":0.95899999999999996},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.3400000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.86199999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.997,"15min":1.0089999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.0740000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0209999999999999,"15min":1.0169999999999999},"min":{"1min":0.79300000000000004,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.1830000000000001,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":1.018}]},{"osd":7,"last update":"Fri Sep 13 18:54:15 2024","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.036,"15min":1.0249999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.302,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.998},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.0960000000000001,"15min":1.095},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":1.1140000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31808,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":704512,"data_stored":294645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":31848905},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:55:13 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67800000000000005,"15min":0.68100000000000005},"min":{"1min":0.45600000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.85899999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.82799999999999996}]},{"osd":1,"last update":"Fri Sep 13 18:55:13 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66800000000000004,"15min":0.65600000000000003},"min":{"1min":0.45600000000000002,"5min":0.443,"15min":0.443},"max":{"1min":1.0289999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.57199999999999995,"5min":0.433,"15min":0.433},"max":{"1min":0.98199999999999998,"5min":1.181,"15min":1.181},"last":0.67700000000000005}]},{"osd":2,"last update":"Fri Sep 13 18:55:13 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.80000000000000004,"15min":0.83799999999999997},"min":{"1min":0.47299999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.94499999999999995,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.81599999999999995,"15min":0.81699999999999995},"min":{"1min":0.60199999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0269999999999999,"5min":1.571,"15min":1.571},"last":0.91500000000000004}]},{"osd":4,"last update":"Fri Sep 13 18:55:13 2024","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.995,"15min":1.0229999999999999},"min":{"1min":0.76300000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.046,"5min":1.837,"15min":1.837},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.97199999999999998,"5min":1.0449999999999999,"15min":1.0569999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.2949999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.1180000000000001}]},{"osd":5,"last update":"Fri Sep 13 18:55:13 2024","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.048,"15min":1.0629999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.4079999999999999,"5min":1.893,"15min":1.893},"last":1.008},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.0860000000000001,"15min":1.1100000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.335,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.95099999999999996}]},{"osd":6,"last update":"Fri Sep 13 18:55:13 2024","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":1.095,"15min":1.109},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.2629999999999999,"5min":1.421,"15min":1.421},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1439999999999999,"15min":1.177},"min":{"1min":0.92600000000000005,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.3720000000000001,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.036}]},{"osd":7,"last update":"Fri Sep 13 18:55:13 2024","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.218,"15min":1.266},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.45,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1479999999999999,"15min":1.1519999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.359,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.3280000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35900,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335940,"statfs":{"total":96636764160,"available":96600002560,"internally_reserved":0,"allocated":704512,"data_stored":294645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:55:15 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.39000000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.80900000000000005,"15min":0.84299999999999997},"min":{"1min":0.48399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0089999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.79000000000000004}]},{"osd":1,"last update":"Fri Sep 13 18:55:15 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.75700000000000001,"15min":0.78100000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97799999999999998,"5min":1.359,"15min":1.359},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.79300000000000004,"15min":0.81299999999999994},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.2629999999999999,"5min":1.696,"15min":1.696},"last":0.64700000000000002}]},{"osd":3,"last update":"Fri Sep 13 18:55:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67200000000000004,"15min":0.65600000000000003},"min":{"1min":0.5,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82899999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.90500000000000003,"15min":0.91400000000000003},"min":{"1min":0.58099999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.05,"5min":1.288,"15min":1.288},"last":0.88}]},{"osd":4,"last update":"Fri Sep 13 18:55:15 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.0469999999999999,"15min":1.071},"min":{"1min":0.79400000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.137,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.048,"15min":1.0669999999999999},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.1539999999999999,"5min":1.879,"15min":1.879},"last":0.94599999999999995}]},{"osd":5,"last update":"Fri Sep 13 18:55:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.075,"15min":1.099},"min":{"1min":0.80200000000000005,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.268,"5min":1.796,"15min":1.796},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0309999999999999,"15min":1.0409999999999999},"min":{"1min":0.67300000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.319,"5min":1.516,"15min":1.516},"last":0.91700000000000004}]},{"osd":6,"last update":"Fri Sep 13 18:55:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0680000000000001,"15min":1.0880000000000001},"min":{"1min":0.747,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.333,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.1519999999999999,"15min":1.1739999999999999},"min":{"1min":0.71999999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.288,"5min":1.601,"15min":1.601},"last":1.1479999999999999}]},{"osd":7,"last update":"Fri Sep 13 18:55:15 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.1559999999999999,"15min":1.1859999999999999},"min":{"1min":0.78900000000000003,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.2709999999999999,"5min":1.579,"15min":1.579},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.1319999999999999,"15min":1.1419999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.379,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":1.127}]}]},{"osd":1,"up_from":14,"seq":60129542184,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31808,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340032,"statfs":{"total":96636764160,"available":96604192768,"internally_reserved":0,"allocated":704512,"data_stored":294645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.51900000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86899999999999999,"5min":2.786,"15min":2.786},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.629,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0009999999999999,"5min":1.159,"15min":1.159},"last":0.59699999999999998}]},{"osd":2,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.78100000000000003,"15min":0.79700000000000004},"min":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2030000000000001,"5min":3.113,"15min":3.113},"last":1.095},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.232,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.1180000000000001}]},{"osd":3,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84599999999999997,"15min":0.84799999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.117,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81999999999999995,"15min":0.81899999999999995},"min":{"1min":0.626,"5min":0.502,"15min":0.502},"max":{"1min":1.1619999999999999,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.99199999999999999,"15min":1.004},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.2070000000000001,"5min":2.387,"15min":2.387},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":1.0589999999999999,"15min":1.0620000000000001},"min":{"1min":0.83799999999999997,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.51,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.72899999999999998}]},{"osd":5,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0469999999999999,"15min":1.0529999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4139999999999999,"5min":1.419,"15min":1.419},"last":0.873},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.0680000000000001,"15min":1.0860000000000001},"min":{"1min":0.74099999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.2789999999999999,"5min":2.4780000000000002,"15min":2.4780000000000002},"last":0.95199999999999996}]},{"osd":6,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.135,"15min":1.139},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.6020000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.141,"15min":1.151},"min":{"1min":0.82899999999999996,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.5760000000000001,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.147}]},{"osd":7,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.2030000000000001,"15min":1.2150000000000001},"min":{"1min":0.89700000000000002,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.659,"5min":1.825,"15min":1.825},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.1599999999999999,"15min":1.1679999999999999},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.7350000000000001,"5min":2.4409999999999998,"15min":2.4409999999999998},"last":0.89900000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1298432,"data_stored":885013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.95099999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68899999999999995,"15min":0.68799999999999994},"min":{"1min":0.438,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.88}]},{"osd":2,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.64800000000000002,"15min":0.63800000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.83599999999999997,"15min":0.82399999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.72599999999999998}]},{"osd":3,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.75900000000000001,"15min":0.74299999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.5859999999999999,"5min":2.5859999999999999,"15min":2.5859999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.79300000000000004,"15min":0.76900000000000002},"min":{"1min":0.42799999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":0.81599999999999995}]},{"osd":4,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.018,"15min":1.0169999999999999},"min":{"1min":0.68100000000000005,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.94399999999999995,"5min":1.018,"15min":1.03},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.3009999999999999,"5min":1.579,"15min":1.579},"last":1.0069999999999999}]},{"osd":5,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":0.97199999999999998,"15min":0.96599999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.073,"15min":1.075},"min":{"1min":0.68600000000000005,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.98599999999999999}]},{"osd":6,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.083,"15min":1.093},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3839999999999999,"5min":1.504,"15min":1.504},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.044,"15min":1.0369999999999999},"min":{"1min":0.79800000000000004,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.1100000000000001}]},{"osd":7,"last update":"Fri Sep 13 18:54:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1759999999999999,"15min":1.1839999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.452,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0860000000000001,"15min":1.081},"min":{"1min":0.64800000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.0780000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T18:55:21.333 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T18:55:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.445+0000 7fbb102ea700 1 Processor -- start 2024-09-13T18:55:21.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.445+0000 7fbb102ea700 1 -- start start 2024-09-13T18:55:21.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.445+0000 7fbb102ea700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb0815ce90 0x7fbb080ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.445+0000 7fbb102ea700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb080cae50 con 0x7fbb08072460 2024-09-13T18:55:21.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.445+0000 7fbb102ea700 1 -- --> v1:172.21.15.171:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb0815b3a0 con 0x7fbb0815ac40 2024-09-13T18:55:21.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.445+0000 7fbb102ea700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb0815b680 con 0x7fbb0815ce90 2024-09-13T18:55:21.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.445+0000 7fbb0e887700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb0815ce90 0x7fbb080ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.445+0000 7fbb0d885700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbb08072460 0x7fbb080ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41648/0 (socket says 172.21.15.7:41648) 2024-09-13T18:55:21.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.445+0000 7fbb0e887700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb0815ce90 0x7fbb080ca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53094/0 (socket says 172.21.15.7:53094) 2024-09-13T18:55:21.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.445+0000 7fbb0d885700 1 -- 172.21.15.7:0/3469053949 learned_addr learned my addr 172.21.15.7:0/3469053949 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:55:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.445+0000 7fbb0e887700 1 -- 172.21.15.7:0/3469053949 >> v1:172.21.15.171:6789/0 conn(0x7fbb0815ac40 legacy=0x7fbb08071d50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-13T18:55:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.445+0000 7fbb0e887700 1 -- 172.21.15.7:0/3469053949 >> v1:172.21.15.7:6789/0 conn(0x7fbb08072460 legacy=0x7fbb080ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.445+0000 7fbb0e887700 1 -- 172.21.15.7:0/3469053949 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0815b960 con 0x7fbb0815ce90 2024-09-13T18:55:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.449+0000 7fbb0e887700 1 --2- 172.21.15.7:0/3469053949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb0815ce90 0x7fbb080ca7f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009d90 tx=0x7fbb0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df16100ec6a5020 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.449+0000 7fbb0d084700 1 -- 172.21.15.7:0/3469053949 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0000e070 con 0x7fbb0815ce90 2024-09-13T18:55:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.449+0000 7fbb0d084700 1 -- 172.21.15.7:0/3469053949 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0000bb90 con 0x7fbb0815ce90 2024-09-13T18:55:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.449+0000 7fbb0d084700 1 -- 172.21.15.7:0/3469053949 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb00003a40 con 0x7fbb0815ce90 2024-09-13T18:55:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.449+0000 7fbb102ea700 1 -- 172.21.15.7:0/3469053949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb0815ce90 msgr2=0x7fbb080ca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.449+0000 7fbb102ea700 1 --2- 172.21.15.7:0/3469053949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb0815ce90 0x7fbb080ca7f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009d90 tx=0x7fbb0000b330 comp rx=0 tx=0).stop 2024-09-13T18:55:21.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.449+0000 7fbb102ea700 1 -- 172.21.15.7:0/3469053949 shutdown_connections 2024-09-13T18:55:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.449+0000 7fbb102ea700 1 --2- 172.21.15.7:0/3469053949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb0815ce90 0x7fbb080ca7f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.449+0000 7fbb102ea700 1 -- 172.21.15.7:0/3469053949 >> 172.21.15.7:0/3469053949 conn(0x7fbb080cc4a0 msgr2=0x7fbb080cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.449+0000 7fbb102ea700 1 -- 172.21.15.7:0/3469053949 shutdown_connections 2024-09-13T18:55:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.449+0000 7fbb102ea700 1 -- 172.21.15.7:0/3469053949 wait complete. 2024-09-13T18:55:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.449+0000 7fbb102ea700 1 Processor -- start 2024-09-13T18:55:21.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.449+0000 7fbb102ea700 1 -- start start 2024-09-13T18:55:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb102ea700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb08072460 0x7fbb08156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb102ea700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb0815ac40 0x7fbb08149b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb102ea700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb0815ce90 0x7fbb0814a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb102ea700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0814cf10 con 0x7fbb08072460 2024-09-13T18:55:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb102ea700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb08155200 con 0x7fbb0815ce90 2024-09-13T18:55:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb102ea700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb081554a0 con 0x7fbb0815ac40 2024-09-13T18:55:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb0e086700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb08072460 0x7fbb08156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb0e086700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb08072460 0x7fbb08156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53110/0 (socket says 172.21.15.7:53110) 2024-09-13T18:55:21.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb0e086700 1 -- 172.21.15.7:0/2390440314 learned_addr learned my addr 172.21.15.7:0/2390440314 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:55:21.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb0e887700 1 --2- 172.21.15.7:0/2390440314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb0815ce90 0x7fbb0814a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb0d885700 1 --2- 172.21.15.7:0/2390440314 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb0815ac40 0x7fbb08149b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb0e086700 1 -- 172.21.15.7:0/2390440314 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb0815ac40 msgr2=0x7fbb08149b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb0e086700 1 --2- 172.21.15.7:0/2390440314 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb0815ac40 0x7fbb08149b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb0e086700 1 -- 172.21.15.7:0/2390440314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb0815ce90 msgr2=0x7fbb0814a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb0e086700 1 --2- 172.21.15.7:0/2390440314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb0815ce90 0x7fbb0814a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb0e086700 1 -- 172.21.15.7:0/2390440314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb00009a70 con 0x7fbb08072460 2024-09-13T18:55:21.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb0e086700 1 --2- 172.21.15.7:0/2390440314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb08072460 0x7fbb08156990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800b4a0 tx=0x7fbaf800b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbafeffd700 1 -- 172.21.15.7:0/2390440314 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8011830 con 0x7fbb08072460 2024-09-13T18:55:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbafeffd700 1 -- 172.21.15.7:0/2390440314 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf80119b0 con 0x7fbb08072460 2024-09-13T18:55:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbafeffd700 1 -- 172.21.15.7:0/2390440314 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8011d60 con 0x7fbb08072460 2024-09-13T18:55:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb102ea700 1 -- 172.21.15.7:0/2390440314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb08155780 con 0x7fbb08072460 2024-09-13T18:55:21.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.453+0000 7fbb102ea700 1 -- 172.21.15.7:0/2390440314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb08155d40 con 0x7fbb08072460 2024-09-13T18:55:21.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.457+0000 7fbb102ea700 1 -- 172.21.15.7:0/2390440314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb08071d50 con 0x7fbb08072460 2024-09-13T18:55:21.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.457+0000 7fbafeffd700 1 -- 172.21.15.7:0/2390440314 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7fbaf801a400 con 0x7fbb08072460 2024-09-13T18:55:21.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.457+0000 7fbafeffd700 1 --2- 172.21.15.7:0/2390440314 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbaec066730 0x7fbaec068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.457+0000 7fbafeffd700 1 -- 172.21.15.7:0/2390440314 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fbaf8083f90 con 0x7fbb08072460 2024-09-13T18:55:21.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.457+0000 7fbb0d885700 1 --2- 172.21.15.7:0/2390440314 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbaec066730 0x7fbaec068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.457+0000 7fbb0d885700 1 --2- 172.21.15.7:0/2390440314 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbaec066730 0x7fbaec068bf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbb04000f60 tx=0x7fbb04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:21.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.465+0000 7fbafeffd700 1 -- 172.21.15.7:0/2390440314 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbaf8051500 con 0x7fbb08072460 2024-09-13T18:55:21.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.637+0000 7fbb102ea700 1 -- 172.21.15.7:0/2390440314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fbb080b3630 con 0x7fbb08072460 2024-09-13T18:55:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.645+0000 7fbafeffd700 1 -- 172.21.15.7:0/2390440314 <== mon.0 v2:172.21.15.7:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fbaf80092c0 con 0x7fbb08072460 2024-09-13T18:55:21.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.645+0000 7fbafeffd700 1 -- 172.21.15.7:0/2390440314 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7fbaf80150c0 con 0x7fbb08072460 2024-09-13T18:55:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.649+0000 7fbb102ea700 1 -- 172.21.15.7:0/2390440314 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbaec066730 msgr2=0x7fbaec068bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.649+0000 7fbb102ea700 1 --2- 172.21.15.7:0/2390440314 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbaec066730 0x7fbaec068bf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbb04000f60 tx=0x7fbb04009040 comp rx=0 tx=0).stop 2024-09-13T18:55:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.649+0000 7fbb102ea700 1 -- 172.21.15.7:0/2390440314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb08072460 msgr2=0x7fbb08156990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.649+0000 7fbb102ea700 1 --2- 172.21.15.7:0/2390440314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb08072460 0x7fbb08156990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800b4a0 tx=0x7fbaf800b7b0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.649+0000 7fbb102ea700 1 -- 172.21.15.7:0/2390440314 shutdown_connections 2024-09-13T18:55:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.649+0000 7fbb102ea700 1 --2- 172.21.15.7:0/2390440314 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7fbaec066730 0x7fbaec068bf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.649+0000 7fbb102ea700 1 --2- 172.21.15.7:0/2390440314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb0815ce90 0x7fbb0814a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.649+0000 7fbb102ea700 1 --2- 172.21.15.7:0/2390440314 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7fbb0815ac40 0x7fbb08149b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.649+0000 7fbb102ea700 1 --2- 172.21.15.7:0/2390440314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb08072460 0x7fbb08156990 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.649+0000 7fbb102ea700 1 -- 172.21.15.7:0/2390440314 >> 172.21.15.7:0/2390440314 conn(0x7fbb080cc4a0 msgr2=0x7fbb080ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.649+0000 7fbb102ea700 1 -- 172.21.15.7:0/2390440314 shutdown_connections 2024-09-13T18:55:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.649+0000 7fbb102ea700 1 -- 172.21.15.7:0/2390440314 wait complete. 2024-09-13T18:55:21.672 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T18:55:21.672 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T18:55:21.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.781+0000 7ff99308a700 1 Processor -- start 2024-09-13T18:55:21.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.781+0000 7ff99308a700 1 -- start start 2024-09-13T18:55:21.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.781+0000 7ff99308a700 1 --2- >> v2:172.21.15.171:3300/0 conn(0x7ff98c146760 0x7ff98c0ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.781+0000 7ff99308a700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c0d1310 0x7ff98c15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.781+0000 7ff99308a700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98c0cbbd0 con 0x7ff98c0b5bc0 2024-09-13T18:55:21.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.781+0000 7ff99308a700 1 -- --> v2:172.21.15.171:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff98c0cc060 con 0x7ff98c146760 2024-09-13T18:55:21.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.781+0000 7ff99308a700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c0cafc0 con 0x7ff98c0d1310 2024-09-13T18:55:21.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.781+0000 7ff98bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff98c0b5bc0 0x7ff98c0d0c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41650/0 (socket says 172.21.15.7:41650) 2024-09-13T18:55:21.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.781+0000 7ff98bfff700 1 -- 172.21.15.7:0/3296168836 learned_addr learned my addr 172.21.15.7:0/3296168836 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-09-13T18:55:21.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.781+0000 7ff991627700 1 --2- 172.21.15.7:0/3296168836 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c0d1310 0x7ff98c15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.781+0000 7ff990e26700 1 --2- 172.21.15.7:0/3296168836 >> v2:172.21.15.171:3300/0 conn(0x7ff98c146760 0x7ff98c0ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff98b7fe700 1 -- 172.21.15.7:0/3296168836 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302265113 0 0) 0x7ff98c0cbbd0 con 0x7ff98c0b5bc0 2024-09-13T18:55:21.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff98b7fe700 1 -- 172.21.15.7:0/3296168836 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff97c0035d0 con 0x7ff98c0b5bc0 2024-09-13T18:55:21.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff991627700 1 -- 172.21.15.7:0/3296168836 >> v2:172.21.15.171:3300/0 conn(0x7ff98c146760 msgr2=0x7ff98c0ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T18:55:21.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff991627700 1 --2- 172.21.15.7:0/3296168836 >> v2:172.21.15.171:3300/0 conn(0x7ff98c146760 0x7ff98c0ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff991627700 1 -- 172.21.15.7:0/3296168836 >> v1:172.21.15.7:6789/0 conn(0x7ff98c0b5bc0 legacy=0x7ff98c0d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff98b7fe700 1 -- 172.21.15.7:0/3296168836 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3440209494 0 0) 0x7ff97c0035d0 con 0x7ff98c0b5bc0 2024-09-13T18:55:21.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff991627700 1 -- 172.21.15.7:0/3296168836 --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c0cb2a0 con 0x7ff98c0d1310 2024-09-13T18:55:21.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff991627700 1 --2- 172.21.15.7:0/3296168836 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c0d1310 0x7ff98c15d2a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff984009d90 tx=0x7ff98400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9c5494627890da0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:21.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff98b7fe700 1 -- 172.21.15.7:0/3296168836 <== mon.2 v2:172.21.15.171:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98400e070 con 0x7ff98c0d1310 2024-09-13T18:55:21.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff98b7fe700 1 -- 172.21.15.7:0/3296168836 <== mon.2 v2:172.21.15.171:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff984003a40 con 0x7ff98c0d1310 2024-09-13T18:55:21.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff98b7fe700 1 -- 172.21.15.7:0/3296168836 <== mon.2 v2:172.21.15.171:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978001670 con 0x7ff98c0d1310 2024-09-13T18:55:21.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff99308a700 1 -- 172.21.15.7:0/3296168836 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c0d1310 msgr2=0x7ff98c15d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff99308a700 1 --2- 172.21.15.7:0/3296168836 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c0d1310 0x7ff98c15d2a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff984009d90 tx=0x7ff98400b330 comp rx=0 tx=0).stop 2024-09-13T18:55:21.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff99308a700 1 -- 172.21.15.7:0/3296168836 shutdown_connections 2024-09-13T18:55:21.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff99308a700 1 --2- 172.21.15.7:0/3296168836 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c0d1310 0x7ff98c15d2a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff99308a700 1 --2- 172.21.15.7:0/3296168836 >> v2:172.21.15.171:3300/0 conn(0x7ff98c146760 0x7ff98c0ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff99308a700 1 -- 172.21.15.7:0/3296168836 >> 172.21.15.7:0/3296168836 conn(0x7ff98c06a4f0 msgr2=0x7ff98c071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:21.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff99308a700 1 -- 172.21.15.7:0/3296168836 shutdown_connections 2024-09-13T18:55:21.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff99308a700 1 -- 172.21.15.7:0/3296168836 wait complete. 2024-09-13T18:55:21.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff99308a700 1 Processor -- start 2024-09-13T18:55:21.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.785+0000 7ff99308a700 1 -- start start 2024-09-13T18:55:21.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff99308a700 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff98c0b5bc0 0x7ff98c13afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff99308a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff98c0d1310 0x7ff98c1429c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff99308a700 1 --2- >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c146760 0x7ff98c0cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff99308a700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c1f2630 con 0x7ff98c0d1310 2024-09-13T18:55:21.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff99308a700 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c13c1b0 con 0x7ff98c0b5bc0 2024-09-13T18:55:21.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff99308a700 1 -- --> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c13b5a0 con 0x7ff98c146760 2024-09-13T18:55:21.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff98bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff98c0d1310 0x7ff98c1429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff98bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff98c0d1310 0x7ff98c1429c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53122/0 (socket says 172.21.15.7:53122) 2024-09-13T18:55:21.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff98bfff700 1 -- 172.21.15.7:0/2818757722 learned_addr learned my addr 172.21.15.7:0/2818757722 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-13T18:55:21.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff991627700 1 --2- 172.21.15.7:0/2818757722 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c146760 0x7ff98c0cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff990e26700 1 --2- 172.21.15.7:0/2818757722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff98c0b5bc0 0x7ff98c13afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff98bfff700 1 -- 172.21.15.7:0/2818757722 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c146760 msgr2=0x7ff98c0cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff98bfff700 1 --2- 172.21.15.7:0/2818757722 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c146760 0x7ff98c0cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff98bfff700 1 -- 172.21.15.7:0/2818757722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff98c0b5bc0 msgr2=0x7ff98c13afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff98bfff700 1 --2- 172.21.15.7:0/2818757722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff98c0b5bc0 0x7ff98c13afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff98bfff700 1 -- 172.21.15.7:0/2818757722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff984009a70 con 0x7ff98c0d1310 2024-09-13T18:55:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff991627700 1 --2- 172.21.15.7:0/2818757722 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c146760 0x7ff98c0cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:55:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff990e26700 1 --2- 172.21.15.7:0/2818757722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff98c0b5bc0 0x7ff98c13afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-13T18:55:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff98bfff700 1 --2- 172.21.15.7:0/2818757722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff98c0d1310 0x7ff98c1429c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff97800c750 tx=0x7ff97800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff9897fa700 1 -- 172.21.15.7:0/2818757722 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978004ca0 con 0x7ff98c0d1310 2024-09-13T18:55:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff9897fa700 1 -- 172.21.15.7:0/2818757722 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff978004e20 con 0x7ff98c0d1310 2024-09-13T18:55:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff99308a700 1 -- 172.21.15.7:0/2818757722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98c13b880 con 0x7ff98c0d1310 2024-09-13T18:55:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff9897fa700 1 -- 172.21.15.7:0/2818757722 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978007330 con 0x7ff98c0d1310 2024-09-13T18:55:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff99308a700 1 -- 172.21.15.7:0/2818757722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98c13be40 con 0x7ff98c0d1310 2024-09-13T18:55:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.789+0000 7ff99308a700 1 -- 172.21.15.7:0/2818757722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff98c13c320 con 0x7ff98c0d1310 2024-09-13T18:55:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.793+0000 7ff9897fa700 1 -- 172.21.15.7:0/2818757722 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 86092+0+0 (secure 0 0 0) 0x7ff978007490 con 0x7ff98c0d1310 2024-09-13T18:55:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.793+0000 7ff9897fa700 1 --2- 172.21.15.7:0/2818757722 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff970066780 0x7ff970068c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T18:55:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.793+0000 7ff9897fa700 1 -- 172.21.15.7:0/2818757722 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5029+0+0 (secure 0 0 0) 0x7ff978083940 con 0x7ff98c0d1310 2024-09-13T18:55:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.793+0000 7ff990e26700 1 --2- 172.21.15.7:0/2818757722 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff970066780 0x7ff970068c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T18:55:21.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.793+0000 7ff990e26700 1 --2- 172.21.15.7:0/2818757722 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff970066780 0x7ff970068c40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff980009ba0 tx=0x7ff980008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T18:55:21.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:21.797+0000 7ff9897fa700 1 -- 172.21.15.7:0/2818757722 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff978050fa0 con 0x7ff98c0d1310 2024-09-13T18:55:22.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:22.013+0000 7ff99308a700 1 -- 172.21.15.7:0/2818757722 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff98c0ca8e0 con 0x7ff98c0d1310 2024-09-13T18:55:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:22.013+0000 7ff9897fa700 1 -- 172.21.15.7:0/2818757722 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3526 (secure 0 0 0) 0x7ff978018030 con 0x7ff98c0d1310 2024-09-13T18:55:22.020 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-13T18:55:22.020 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/1829974483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":1829974483},{"type":"v1","addr":"172.21.15.171:6835","nonce":1829974483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/1948505526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":1948505526},{"type":"v1","addr":"172.21.15.171:6837","nonce":1948505526}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":31},{"gid":5378,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/3617964311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3617964311},{"type":"v1","addr":"172.21.15.7:6833","nonce":3617964311}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5531,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/338720208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":338720208},{"type":"v1","addr":"172.21.15.7:6835","nonce":338720208}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[]} 2024-09-13T18:55:22.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:22.013+0000 7ff99308a700 1 -- 172.21.15.7:0/2818757722 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff970066780 msgr2=0x7ff970068c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:22.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:22.013+0000 7ff99308a700 1 --2- 172.21.15.7:0/2818757722 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff970066780 0x7ff970068c40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff980009ba0 tx=0x7ff980008040 comp rx=0 tx=0).stop 2024-09-13T18:55:22.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:22.013+0000 7ff99308a700 1 -- 172.21.15.7:0/2818757722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff98c0d1310 msgr2=0x7ff98c1429c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T18:55:22.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:22.013+0000 7ff99308a700 1 --2- 172.21.15.7:0/2818757722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff98c0d1310 0x7ff98c1429c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff97800c750 tx=0x7ff97800cb10 comp rx=0 tx=0).stop 2024-09-13T18:55:22.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:22.013+0000 7ff99308a700 1 -- 172.21.15.7:0/2818757722 shutdown_connections 2024-09-13T18:55:22.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:22.013+0000 7ff99308a700 1 --2- 172.21.15.7:0/2818757722 >> [v2:172.21.15.171:6832/2829153556,v1:172.21.15.171:6833/2829153556] conn(0x7ff970066780 0x7ff970068c40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:22.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:22.013+0000 7ff99308a700 1 --2- 172.21.15.7:0/2818757722 >> [v2:172.21.15.171:3301/0,v1:172.21.15.171:6790/0] conn(0x7ff98c146760 0x7ff98c0cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:22.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:22.013+0000 7ff99308a700 1 --2- 172.21.15.7:0/2818757722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff98c0d1310 0x7ff98c1429c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:22.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:22.013+0000 7ff99308a700 1 --2- 172.21.15.7:0/2818757722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff98c0b5bc0 0x7ff98c13afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T18:55:22.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:22.013+0000 7ff99308a700 1 -- 172.21.15.7:0/2818757722 >> 172.21.15.7:0/2818757722 conn(0x7ff98c06a4f0 msgr2=0x7ff98c0c9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T18:55:22.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:22.013+0000 7ff99308a700 1 -- 172.21.15.7:0/2818757722 shutdown_connections 2024-09-13T18:55:22.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-13T18:55:22.013+0000 7ff99308a700 1 -- 172.21.15.7:0/2818757722 wait complete. 2024-09-13T18:55:22.023 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 38 2024-09-13T18:55:22.034 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-13T18:55:22.034 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T18:55:22.035 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T18:55:22.035 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:55:22.047 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T18:55:22.047 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-13T18:55:22.047 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:55:22.059 INFO:tasks.ceph.mds.c:Stopped 2024-09-13T18:55:22.060 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T18:55:22.060 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:55:22.077 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T18:55:22.077 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-13T18:55:22.078 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:55:22.139 INFO:tasks.ceph.mds.d:Stopped 2024-09-13T18:55:22.139 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T18:55:22.139 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T18:55:22.139 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:55:22.219 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T18:55:22.219 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T18:55:22.220 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:55:24.006 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-13T18:55:28.321 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T18:55:28.321 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T18:55:28.321 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:55:28.421 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T18:55:28.421 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T18:55:28.421 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:55:29.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:55:29.488 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:55:29.915 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-13T18:55:29.915 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-13T18:55:34.522 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T18:55:34.523 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-13T18:55:34.523 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:55:34.619 INFO:tasks.ceph.osd.4:Stopped 2024-09-13T18:55:34.619 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T18:55:34.619 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:55:35.621 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-13T18:55:35.621 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-13T18:55:40.721 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T18:55:40.722 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-13T18:55:40.722 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:55:40.819 INFO:tasks.ceph.osd.6:Stopped 2024-09-13T18:55:40.819 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-13T18:55:40.819 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:55:41.327 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~6s 2024-09-13T18:55:46.832 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-13T18:55:46.832 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-13T18:55:46.921 INFO:tasks.ceph.osd.7:Stopped 2024-09-13T18:55:46.922 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T18:55:46.922 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T18:55:46.922 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:55:46.953 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T18:55:46.954 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T18:55:46.954 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:55:46.985 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T18:55:46.986 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T18:55:46.986 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T18:55:46.986 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:55:47.028 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T18:55:47.028 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T18:55:47.028 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:55:47.074 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T18:55:47.074 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T18:55:47.075 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T18:55:47.123 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T18:55:47.123 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T18:55:47.124 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-13T18:55:47.147 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-13T18:55:47.147 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T18:55:47.504 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-13T18:55:47.504 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T18:55:47.853 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-13T18:55:47.853 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T18:55:48.029 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-13T18:55:48.029 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T18:55:48.237 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T18:55:48.237 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-13T18:55:48.540 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T18:55:48.540 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-13T18:55:48.721 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T18:55:48.721 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-13T18:55:48.903 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T18:55:48.903 DEBUG:teuthology.orchestra.run.smithi171:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-13T18:55:49.095 INFO:tasks.ceph:Archiving mon data... 2024-09-13T18:55:49.097 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7904006/data/mon.a.tgz 2024-09-13T18:55:49.097 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-13T18:55:49.104 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.oldRRYuJ1G 2024-09-13T18:55:49.104 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.oldRRYuJ1G 2024-09-13T18:55:49.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.oldRRYuJ1G 2024-09-13T18:55:49.261 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.oldRRYuJ1G is 349KB 2024-09-13T18:55:49.325 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.oldRRYuJ1G 2024-09-13T18:55:49.331 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7904006/data/mon.b.tgz 2024-09-13T18:55:49.332 DEBUG:teuthology.orchestra.run.smithi171:> mktemp 2024-09-13T18:55:49.338 INFO:teuthology.orchestra.run.smithi171.stdout:/tmp/tmp.HsE5GxfgSp 2024-09-13T18:55:49.339 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.HsE5GxfgSp 2024-09-13T18:55:49.499 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0666 /tmp/tmp.HsE5GxfgSp 2024-09-13T18:55:49.524 DEBUG:teuthology.orchestra.remote:smithi171:/tmp/tmp.HsE5GxfgSp is 461KB 2024-09-13T18:55:49.587 DEBUG:teuthology.orchestra.run.smithi171:> rm -fr /tmp/tmp.HsE5GxfgSp 2024-09-13T18:55:49.593 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7904006/data/mon.c.tgz 2024-09-13T18:55:49.594 DEBUG:teuthology.orchestra.run.smithi171:> mktemp 2024-09-13T18:55:49.643 INFO:teuthology.orchestra.run.smithi171.stdout:/tmp/tmp.Ae2MSuw0ed 2024-09-13T18:55:49.643 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Ae2MSuw0ed 2024-09-13T18:55:49.755 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0666 /tmp/tmp.Ae2MSuw0ed 2024-09-13T18:55:49.776 DEBUG:teuthology.orchestra.remote:smithi171:/tmp/tmp.Ae2MSuw0ed is 461KB 2024-09-13T18:55:49.843 DEBUG:teuthology.orchestra.run.smithi171:> rm -fr /tmp/tmp.Ae2MSuw0ed 2024-09-13T18:55:49.850 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T18:55:49.850 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T18:55:49.852 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T18:55:49.867 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T18:55:49.867 DEBUG:teuthology.orchestra.run.smithi007:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T18:55:49.912 DEBUG:teuthology.orchestra.run.smithi171:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T18:55:49.936 INFO:teuthology.orchestra.run.smithi171.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T18:55:49.938 INFO:teuthology.orchestra.run.smithi007.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T18:55:49.945 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T18:55:49.946 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7904006/remote/smithi007/crash 2024-09-13T18:55:49.947 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T18:55:49.962 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7904006/remote/smithi171/crash 2024-09-13T18:55:49.963 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T18:55:49.978 INFO:tasks.ceph:Shutting down logrotate 2024-09-13T18:55:49.978 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:55:49.981 DEBUG:teuthology.orchestra.run.smithi171:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T18:55:50.074 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-13T18:55:50.090 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-13T18:55:50.106 INFO:tasks.ceph:Compressing logs... 2024-09-13T18:55:50.106 DEBUG:teuthology.orchestra.run.smithi007:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T18:55:50.109 DEBUG:teuthology.orchestra.run.smithi171:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T18:55:51.286 INFO:tasks.ceph:Archiving logs... 2024-09-13T18:55:51.287 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7904006/remote/smithi007/log 2024-09-13T18:55:51.288 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T18:55:51.679 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7904006/remote/smithi171/log 2024-09-13T18:55:51.680 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T18:55:51.840 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-13T18:55:51.845 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T18:55:51.854 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T18:55:51.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T18:55:51.857 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T18:55:51.955 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T18:55:51.965 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T18:55:51.966 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T18:55:51.968 DEBUG:teuthology.orchestra.run.smithi171:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T18:55:51.992 INFO:teuthology.orchestra.run.smithi007.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T18:55:51.992 INFO:teuthology.orchestra.run.smithi007.stdout:============================================================================== 2024-09-13T18:55:51.992 INFO:teuthology.orchestra.run.smithi007.stdout:*hv01.front.sepi 67.205.162.81 3 u 44 64 177 0.085 1.266 1.094 2024-09-13T18:55:51.992 INFO:teuthology.orchestra.run.smithi007.stdout:+hv02.front.sepi 96.245.170.99 3 u 48 64 177 0.088 -3.145 0.880 2024-09-13T18:55:51.992 INFO:teuthology.orchestra.run.smithi007.stdout:+hv03.front.sepi 74.6.168.72 3 u 49 64 177 0.084 0.499 1.500 2024-09-13T18:55:51.992 INFO:teuthology.orchestra.run.smithi007.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T18:55:51.992 INFO:teuthology.orchestra.run.smithi171.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T18:55:51.992 INFO:teuthology.orchestra.run.smithi171.stdout:============================================================================== 2024-09-13T18:55:51.993 INFO:teuthology.orchestra.run.smithi171.stdout:*hv01.front.sepi 67.205.162.81 3 u 48 64 177 0.080 0.054 0.670 2024-09-13T18:55:51.993 INFO:teuthology.orchestra.run.smithi171.stdout:+hv02.front.sepi 96.245.170.99 3 u 41 64 177 0.092 -4.523 0.770 2024-09-13T18:55:51.993 INFO:teuthology.orchestra.run.smithi171.stdout:+hv03.front.sepi 74.6.168.72 3 u 48 64 177 0.062 0.446 0.531 2024-09-13T18:55:51.993 INFO:teuthology.orchestra.run.smithi171.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T18:55:51.994 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T18:55:52.003 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T18:55:52.004 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T18:55:52.028 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T18:55:52.193 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T18:55:52.230 INFO:teuthology.task.internal:Duration was 790.690829 seconds 2024-09-13T18:55:52.231 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T18:55:52.280 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T18:55:52.281 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T18:55:52.283 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T18:55:52.336 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T18:55:52.336 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-09-13T18:55:52.336 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T18:55:52.349 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi171.front.sepia.ceph.com 2024-09-13T18:55:52.349 DEBUG:teuthology.orchestra.run.smithi171:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T18:55:52.359 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T18:55:52.359 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T18:55:52.361 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T18:55:52.399 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T18:55:52.399 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T18:55:52.444 DEBUG:teuthology.orchestra.run.smithi171:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T18:55:52.523 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T18:55:52.532 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T18:55:52.533 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T18:55:52.545 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T18:55:52.560 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T18:55:52.570 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T18:55:52.572 DEBUG:teuthology.orchestra.run.smithi171:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T18:55:52.584 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-09-13T18:55:52.584 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = core 2024-09-13T18:55:52.601 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T18:55:52.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:55:52.648 DEBUG:teuthology.orchestra.run.smithi171:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T18:55:52.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T18:55:52.657 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T18:55:52.669 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T18:55:52.670 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7904006/remote/smithi007 2024-09-13T18:55:52.670 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T18:55:52.695 DEBUG:teuthology.misc:Transferring archived files from smithi171:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7904006/remote/smithi171 2024-09-13T18:55:52.695 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T18:55:52.721 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T18:55:52.721 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T18:55:52.723 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T18:55:52.730 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T18:55:52.743 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T18:55:52.744 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T18:55:52.779 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T18:55:52.779 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T18:55:52.781 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T18:55:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 18:55 /home/ubuntu/cephtest 2024-09-13T18:55:52.789 INFO:teuthology.orchestra.run.smithi171.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 18:55 /home/ubuntu/cephtest 2024-09-13T18:55:52.791 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T18:55:52.805 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T18:55:52.875 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_20.04} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/openfiletable} duration: 790.69082903862 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-13T18:55:52.875 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T18:55:52.969 INFO:teuthology.run:pass